v2026.11.2
All Bundles
Bundle

Client for HTTP model-inference servers. Sends tensors to a running server and reads tensors back, so a model trained in TensorFlow, PyTorch, ONNX or scikit-learn can be used from Objeck without a native binding for each runtime.

Two wire protocols, because no single one covers the field:

- OPEN_INFERENCE is the KServe v2 protocol, spoken by NVIDIA Triton, KServe, OpenVINO Model Server, Seldon MLServer and others. Triton alone serves TensorFlow, ONNX, PyTorch, TensorRT and OpenVINO behind it, so this one protocol reaches most servers and most runtimes. It is the default. - TF_SERVING is TensorFlow Serving's own REST API, which does not implement KServe v2 and so needs its own request shape.

The difference is confined to Client: a Tensor is the same object either way. Choosing the protocol rather than the server is the whole point -- a client written against one server's API reaches one server.

This is a client, not a runtime. The server is a separate process the user runs; nothing here loads a model. For in-process inference without a server, API.Onnx runs models directly. Compile with -lib inference.

Client->Protocol

Which wire protocol the server speaks.

Items