v2026.11.2
All Bundles
Bundle

Sentence embeddings from a pretrained BERT-family encoder exported to ONNX: the text is tokenized with WordPieceTokenizer, run through API.Onnx, and pooled into one vector per text.

A library of its own because it needs both System.NLP and API.Onnx, and the compiler resolves every class a linked library names: an embedder inside onnx.obl would make every ONNX program link nlp too, and one inside nlp.obl would make every NLP program need ONNX Runtime (docs/ML_BERT_DESIGN.md). Compile with -lib models,opencv,cipher,json,onnx,nlp,nlp_onnx.

TextClassifier

Text classification with a fine-tuned BERT-family model, as Python's pipeline("text-classification"): a sentiment model, a topic classifier, a malware classifier fine-tuned on Colab.

The model is a Hugging Face model folder (what save_pretrained writes, with an ONNX export beside it), or a Hub repository's files downloaded as they are:

- the model: model.onnx, else onnx/model.onnx; or the file the "model" option names, relative to the folder; - the tokenizer: tokenizer.json (WordPiece), else vocab.txt with tokenizer_config.json's do_lower_case (true when absent, as BertTokenizer defaults); - the labels: config.json's id2label, else LABEL_0, LABEL_1, ...

The model's own signature decides what is sent: input_ids always, attention_mask and token_type_ids only when declared (DistilBERT and RoBERTa exports have no token_type_ids). The output is logits FP32[batch, labels].

Scores follow transformers: SIGMOID when config.json's problem_type is multi_label_classification or there is one label, else SOFTMAX. SetFunction overrides it.

Texts longer than the maximum length are truncated, and the Prediction says so. The maximum length is the smallest of 512, the model's max_position_embeddings and the tokenizer's model_max_length, or the model's fixed sequence axis. Each batch is padded to its longest text. With an fp32 model a text's scores do not depend on which texts share its batch. With a dynamically quantized one (model_quantized.onnx, model_int8.onnx) they do, slightly: activations are quantized with a scale taken from the whole batch. SetBatchSize(1) gives what Python's pipeline gives one text at a time.

With no session options, the build's default provider runs the model, and the CPU when that provider cannot open it: see IsCpuFallback. Any option other than "model" is a session option, e.g. "ep" to "cpu".

Example

classifier := TextClassifier->New("distilbert-sst2");
if(<>classifier->IsOpen()) {
  classifier->GetError()->ErrorLine();
  return;
};
each(p in classifier->Classify(["I loved it", "Not worth the ticket"])) {
  p->ToString()->PrintLine();   # POSITIVE 0.9998, then NEGATIVE 0.9991
};
classifier->Close();

Operations

Classify #

Classifies one text.

method : public : Classify(text:String) ~ Prediction

Parameters

NameTypeDescription
textStringthe text

Return

TypeDescription
Predictionthe prediction; Nil on failure (see GetError)

Classify #

Classifies texts.

method : public : Classify(texts:String[]) ~ Prediction[]

Parameters

NameTypeDescription
textsStringthe texts

Return

TypeDescription
Predictionone prediction per text; Nil on failure (see GetError)

Close #

Releases the model. Safe to call more than once.

method : public : Close() ~ Nil

GetBatchSize #

Texts run through the model at once.

method : public : GetBatchSize() ~ Int

Return

TypeDescription
Intbatch size

GetError #

Why the classifier did not open, or why the last call returned Nil.

method : public : GetError() ~ String

Return

TypeDescription
Stringthe reason, or Nil

GetFunction #

How logits become scores.

method : public : GetFunction() ~ TextClassifier->Function

Return

TypeDescription
TextClassifier->FunctionSOFTMAX, SIGMOID or NONE

GetLabels #

The labels, in the order of the model's logits.

method : public : GetLabels() ~ String[]

Return

TypeDescription
Stringlabels

GetMaxLength #

The length, in tokens, a text is truncated to, [CLS] and [SEP] included.

method : public : GetMaxLength() ~ Int

Return

TypeDescription
Intmaximum length

GetModelPath #

The ONNX file that was opened.

method : public : GetModelPath() ~ String

Return

TypeDescription
Stringpath

GetSession #

The ONNX session, for what this class does not cover.

method : public : GetSession() ~ Session

Return

TypeDescription
Sessionthe session

GetTokenizer #

The tokenizer, e.g. to inspect what a text becomes.

method : public : GetTokenizer() ~ WordPieceTokenizer

Return

TypeDescription
WordPieceTokenizerthe tokenizer

IsCpuFallback #

Whether the build's default provider failed to open or to run the model and the CPU provider was used instead. Only when no session options were given.

method : public : IsCpuFallback() ~ Bool

Return

TypeDescription
Booltrue when running on the CPU as a fallback

IsOpen #

Whether the model, tokenizer and labels loaded.

method : public : IsOpen() ~ Bool

Return

TypeDescription
Booltrue when Classify can run

Logits #

The raw logits for each text.

method : public : Logits(texts:String[]) ~ Float[,]

Parameters

NameTypeDescription
textsStringthe texts

Return

TypeDescription
Float[texts, labels]; Nil on failure (see GetError)

New # constructor

Opens a model folder.

New(folder:String)

Parameters

NameTypeDescription
folderStringthe folder holding the model, tokenizer and config.json

New # constructor

Opens a model folder with options: "model" for the ONNX file inside the folder (e.g. onnx/model_quantized.onnx); anything else is a session option, e.g. "ep" to "cpu".

New(folder:String, options:Map<String,String>)

Parameters

NameTypeDescription
folderStringthe folder holding the model, tokenizer and config.json
optionsMap<String,String>options; may be Nil

Scores #

Every label's score for each text, with the configured function.

method : public : Scores(texts:String[]) ~ Float[,]

Parameters

NameTypeDescription
textsStringthe texts

Return

TypeDescription
Float[texts, labels]; Nil on failure (see GetError)

SetBatchSize #

Sets how many texts run through the model at once. A model with a fixed batch axis caps it. With a dynamically quantized model, 1 reproduces one-at-a-time results.

method : public : SetBatchSize(batch_size:Int) ~ Nil

Parameters

NameTypeDescription
batch_sizeInttexts per run, at least 1

SetFunction #

Sets how logits become scores, overriding what config.json implies.

method : public : SetFunction(scoring:TextClassifier->Function) ~ Nil

Parameters

NameTypeDescription
scoringTextClassifier->FunctionSOFTMAX, SIGMOID or NONE

SetMaxLength #

Sets the length, in tokens, a text is truncated to, e.g. the MAX_SEQ_LENGTH a model was fine-tuned with. A model with a fixed sequence axis keeps its own.

method : public : SetMaxLength(max_len:Int) ~ Bool

Parameters

NameTypeDescription
max_lenInttokens, at least 2

Return

TypeDescription
Booltrue when set