v2026.10.0
All Bundles
Bundle

Client for HTTP model-inference servers. Sends tensors to a running server and reads tensors back, so a model trained in TensorFlow, PyTorch, ONNX or scikit-learn can be used from Objeck without a native binding for each runtime. Two wire protocols, because no single one covers the field: - OPEN_INFERENCE is the KServe v2 protocol, spoken by NVIDIA Triton, KServe, OpenVINO Model Server, Seldon MLServer and others. Triton alone serves TensorFlow, ONNX, PyTorch, TensorRT and OpenVINO behind it, so this one protocol reaches most servers and most runtimes. It is the default. - TF_SERVING is TensorFlow Serving's own REST API, which does not implement KServe v2 and so needs its own request shape. The difference is confined to Client: a Tensor is the same object either way. Choosing the protocol rather than the server is the whole point -- a client written against one server's API reaches one server. This is a client, not a runtime. The server is a separate process the user runs; nothing here loads a model. For in-process inference without a server, API.Onnx runs models directly. Compile with -lib inference.

Wire

Reading what came off the wire. Separate from API.Models.EndPoint, which carries the last error for every backend. Finding an error inside a JSON body is HTTP's business, and a backend-neutral bundle should not have to know what JSON is.

Operations

ParseResponse # function

Parses a response body, setting the last error when it is not JSON. Inference servers answer errors as JSON too -- v2 uses {"error": "..."} -- so a body that will not parse means something else went wrong (an HTML error page from a proxy, a truncated response) and the body itself is the most useful thing to report.

function : ParseResponse(body:String) ~ JsonElement

Parameters

NameTypeDescription
bodyStringresponse text

Return

TypeDescription
JsonElementparsed element, or Nil