Phi3Session
Phi-3 / SLM inference session for text generation. Supports autoregressive generation with KV-cache management, temperature-based sampling, and configurable EOS tokens. Example session := Phi3Session->New("phi3-mini/model.onnx"); token_ids := [1, 3148, 1001, 29901, 825, 338, 29871, 29906, 718, 29871, 29906, 29973]; eos_tokens := [2, 32007]; result := session->Generate(token_ids, 128, 0.7, eos_tokens); each(token in result->GetTokens()) { token->PrintLine(); }; session->Close();
Operations
Generate
Generate text tokens from input token IDs
method : public : Generate(token_ids:Int[], max_tokens:Int, temperature:Float, eos_tokens:Int[]) ~ API.Onnx.Phi3ResultParameters
| Name | Type | Description |
|---|---|---|
| token_ids | Int | input token IDs (prompt) |
| max_tokens | Int | maximum number of tokens to generate |
| temperature | Float | sampling temperature (0.0 for greedy) |
| eos_tokens | Int | array of end-of-sequence token IDs |
Return
| Type | Description |
|---|---|
| Phi3Result | generation result with output token IDs |