v2026.10.0
All Bundles
Bundle

Client for HTTP model-inference servers. Sends tensors to a running server and reads tensors back, so a model trained in TensorFlow, PyTorch, ONNX or scikit-learn can be used from Objeck without a native binding for each runtime. Two wire protocols, because no single one covers the field: - OPEN_INFERENCE is the KServe v2 protocol, spoken by NVIDIA Triton, KServe, OpenVINO Model Server, Seldon MLServer and others. Triton alone serves TensorFlow, ONNX, PyTorch, TensorRT and OpenVINO behind it, so this one protocol reaches most servers and most runtimes. It is the default. - TF_SERVING is TensorFlow Serving's own REST API, which does not implement KServe v2 and so needs its own request shape. The difference is confined to Client: a Tensor is the same object either way. Choosing the protocol rather than the server is the whole point -- a client written against one server's API reaches one server. This is a client, not a runtime. The server is a separate process the user runs; nothing here loads a model. For in-process inference without a server, API.Onnx runs models directly. Compile with -lib inference.

ModelMetadata

What a server reports about a model: the names, shapes and datatypes it expects and returns. Worth asking for once before inferring. A shape or name mismatch is otherwise reported by the server as a rejected request, and the message rarely says which tensor was wrong.

Operations

GetInputs #

Expected inputs, carrying names, shapes and datatypes but no data. A dimension of -1 means the server accepts any size there, which is usually the batch dimension.

method : public : GetInputs() ~ Vector<Tensor>

Return

TypeDescription
Vector<Tensor>input descriptions

GetOutputs #

Produced outputs, in the same form.

method : public : GetOutputs() ~ Vector<Tensor>

Return

TypeDescription
Vector<Tensor>output descriptions

GetPlatform #

The runtime behind the model, when the server reports one -- "tensorflow", "onnxruntime", "pytorch". Empty for servers that do not say.

method : public : GetPlatform() ~ String

Return

TypeDescription
Stringplatform name