ONNX Runtime inference library for running pre-trained models. Supports object detection (YOLO), image classification (ResNet), segmentation (DeepLab), pose estimation (OpenPose), face recognition, and Phi-3 text generation. Compile with -lib onnx.
Session
A model session that runs any ONNX graph, without knowing what family it belongs to. The seven session classes above each own one model family: they letterbox or normalise an image, run it, and decode the result into YOloResult, ResNetResult and the rest. Use them when they fit -- they are less work and they are what the examples use. Use this one when they do not: a model with several inputs, a model whose outputs you want to decode yourself, a text or audio model, or anything the eight families do not cover. Preprocessing and decoding are then yours to write, in Objeck, where they can be reused rather than added to the native library as a ninth vertical (docs/INFERENCE_ENGINES_PLAN.md).
Example
session := Session->New("model.onnx");
if(<>session->IsOpen()) {
return;
};
inputs := Vector->New()<Tensor>;
inputs->AddBack(Tensor->New("data", [1, 3, 224, 224], pixels));
outputs := session->Run(inputs);
each(tensor in outputs) {
tensor->ToString()->PrintLine();
};
session->Close();Operations
Close #
Releases the model. Zeroing the handle afterwards is what makes a second Close safe and makes IsOpen report the truth from then on.
method : public : Close() ~ NilGetModelInfo #
What the model takes in and gives back. Read once and cached: a model's signature does not change under us.
method : public : GetModelInfo() ~ ModelInfoReturn
| Type | Description |
|---|---|
| ModelInfo | the signature, or Nil when the session is not open |
IsOpen #
Whether the model loaded. A session that failed to open is not an error at construction -- the constructor returns an object either way, and the reason was written to stderr while it happened. Test this before running: a closed session returns Nil from Run, which is otherwise indistinguishable from a model that produced nothing.
method : public : IsOpen() ~ BoolReturn
| Type | Description |
|---|---|
| Bool | true when the model is loaded and a run can proceed |
New # constructor
Constructor.
New(model:String, config:Map<String,String>)Parameters
| Name | Type | Description |
|---|---|---|
| model | String | model path |
| config | Map<String,String> | session configuration parameters, e.g. ep |
Run #
Runs the model, asking for every output it has.
method : public : Run(inputs:Vector<Tensor>) ~ Vector<Tensor>Parameters
| Name | Type | Description |
|---|---|---|
| inputs | Vector<Tensor> | the model's inputs as named tensors |
Return
| Type | Description |
|---|---|
| Vector<Tensor> | the outputs, or Nil when the run was refused |
Run #
Runs the model for named outputs.
method : public : Run(inputs:Vector<Tensor>, output_names:String[]) ~ Vector<Tensor>Parameters
| Name | Type | Description |
|---|---|---|
| inputs | Vector<Tensor> | the model's inputs as named tensors |
| output_names | String | the outputs wanted; empty for all of them |
Return
| Type | Description |
|---|---|
| Vector<Tensor> | the outputs, or Nil when the run was refused |
RunTensors #
Runs the model and returns the result before it is split into tensors, for a caller reading one output out of many.
method : public : RunTensors(inputs:Vector<Tensor>, output_names:String[]) ~ RunResultParameters
| Name | Type | Description |
|---|---|---|
| inputs | Vector<Tensor> | the model's inputs as named tensors |
| output_names | String | the outputs wanted; empty or Nil for all of them |
Return
| Type | Description |
|---|---|
| RunResult | the raw result, or Nil when the run was refused |