Ollama local LLM client. Runs models (Llama, Mistral, Phi, Qwen, etc.) locally via the Ollama server. Supports text generation, chat sessions with tool calling, structured (JSON schema) output, thinking models, images, batch embeddings and model management. Every request completes before returning -- responses are not streamed, so a long generation produces nothing until it finishes. Compile with -lib ollama.
Reply
A complete reply from /api/generate or /api/chat: the text, any thinking and tool calls, and the timing and token counts. Durations are in nanoseconds.
Example
reply := Completion->GenerateReply("llama3", "Why is the sky blue?", Nil);
if(reply <> Nil) {
reply->GetText()->PrintLine();
"tokens: {$reply->GetEvalCount()}, done: {$reply->GetDoneReason()}"->PrintLine();
};Operations
- GetDoneReason
- GetEvalCount
- GetEvalDuration
- GetLoadDuration
- GetModel
- GetPromptEvalCount
- GetPromptEvalDuration
- GetRole
- GetText
- GetThinking
- GetTokensPerSecond
- GetToolCalls
- GetTotalDuration
- HasToolCalls
- IsDone
- Parse
GetDoneReason #
Get why generation stopped (e.g. "stop", "length", "load")
method : public : GetDoneReason() ~ StringReturn
| Type | Description |
|---|---|
| String | done reason, or Nil |
GetEvalCount #
Get the number of tokens generated
method : public : GetEvalCount() ~ IntReturn
| Type | Description |
|---|---|
| Int | token count |
GetEvalDuration #
Get the time spent generating
method : public : GetEvalDuration() ~ IntReturn
| Type | Description |
|---|---|
| Int | nanoseconds |
GetLoadDuration #
Get the time spent loading the model
method : public : GetLoadDuration() ~ IntReturn
| Type | Description |
|---|---|
| Int | nanoseconds |
GetModel #
Get the model that replied
method : public : GetModel() ~ StringReturn
| Type | Description |
|---|---|
| String | model name |
GetPromptEvalCount #
Get the number of prompt tokens evaluated
method : public : GetPromptEvalCount() ~ IntReturn
| Type | Description |
|---|---|
| Int | token count |
GetPromptEvalDuration #
Get the time spent evaluating the prompt
method : public : GetPromptEvalDuration() ~ IntReturn
| Type | Description |
|---|---|
| Int | nanoseconds |
GetRole #
Get the role of the reply message
method : public : GetRole() ~ StringReturn
| Type | Description |
|---|---|
| String | role (normally "assistant") |
GetText #
Get the generated text (empty when the model only called tools)
method : public : GetText() ~ StringReturn
| Type | Description |
|---|---|
| String | text |
GetThinking #
Get the model's reasoning, for thinking models with think enabled
method : public : GetThinking() ~ StringReturn
| Type | Description |
|---|---|
| String | thinking text, or Nil |
GetTokensPerSecond #
Get the generation speed
method : public : GetTokensPerSecond() ~ FloatReturn
| Type | Description |
|---|---|
| Float | tokens per second, or 0.0 if no timing was reported |
GetToolCalls #
Get the tool calls the model made (chat with tools only)
method : public : GetToolCalls() ~ Vector<ToolCall>Return
| Type | Description |
|---|---|
| Vector<ToolCall> | tool calls, empty if none |
GetTotalDuration #
Get the total request time
method : public : GetTotalDuration() ~ IntReturn
| Type | Description |
|---|---|
| Int | nanoseconds |
HasToolCalls #
Checks for tool calls
method : public : HasToolCalls() ~ BoolReturn
| Type | Description |
|---|---|
| Bool | true if the model called one or more tools |