Sentence embeddings from a pretrained BERT-family encoder exported to ONNX: the text is tokenized with WordPieceTokenizer, run through API.Onnx, and pooled into one vector per text.
A library of its own because it needs both System.NLP and API.Onnx, and the compiler resolves every class a linked library names: an embedder inside onnx.obl would make every ONNX program link nlp too, and one inside nlp.obl would make every NLP program need ONNX Runtime (docs/ML_BERT_DESIGN.md). Compile with -lib models,opencv,cipher,json,onnx,nlp,nlp_onnx.
BertEmbedder->Layers
Which hidden layers the token vectors are taken from. LAST is the final layer, as every earlier release did. The other two need a model exported with its intermediate layers as outputs (see HasHiddenLayers) and take the last four of them: the common recipe of Devlin et al. (2019) for feature-based use of BERT.
Items
- LAST — The final layer, last_hidden_state.
- SUM_LAST_4 — The element-wise sum of the last four hidden layers, then pooled; the embedding is hidden-size long.
- CONCAT_LAST_4 — The last four hidden layers side by side, each pooled; the embedding is four times the hidden size long, the oldest of the four first.