v2026.11.2
All Bundles
Bundle

Core machine learning types: the seedable Random generator, Matrix2D linear algebra and activations, matrix reference holders, the NeuralNetwork and CSV-backed MatrixReader. Compile with -lib ml.

EmbeddingClassifier

A binary classifier trained on fixed-size feature vectors -- sentence embeddings from a frozen encoder above all: logistic regression (no hidden layer) or a small multi-layer network with ReLU hidden layers, a sigmoid output and binary cross-entropy loss, trained by minibatch Adam or SGD at a configurable learning rate.

For class imbalance, SetClassWeights (or SetBalancedWeights) weights each row's loss by its class. GetLossHistory gives the training loss after every epoch, for a loss curve.

Inputs are standardised with the training set's mean and deviation, which are kept with the model: embedding dimensions differ widely in scale, and an unscaled head trains far slower. Weights start from System.ML.Random with the given seed, and minibatches are shuffled by the same generator: the same seed and data give the same model, bit for bit.

This is the head of a frozen-encoder classifier (docs/ML_BERT_DESIGN.md); nothing here trains the encoder.

Implements: ScoreModel

Example

head := EmbeddingClassifier->New([64], 0.001, 20, 32, 42);
head->SetBalancedWeights();
head->Fit(X_train, y_train);
loss := head->GetLossHistory();             # one value per epoch
p := head->PredictProba(X_test);            # P(positive) per row
yhat := head->PredictClass(X_test, 0.5);

Operations

Fit #

Trains the head.

method : public : Fit(X:Float[,], y:Bool[]) ~ Bool

Parameters

NameTypeDescription
XFloatrows x features
yBoolone label per row, true for the positive class

Return

TypeDescription
Booltrue when trained; false with GetError set for empty, mismatched or NaN input, or a configuration that cannot train

GetBias #

A layer's biases, for inspection. A copy.

method : public : GetBias(layer:Int) ~ Float[]

Parameters

NameTypeDescription
layerIntlayer index

Return

TypeDescription
Floatbiases, or Nil before Fit or for a layer out of range

GetClassWeights #

The class weights in use: those set, or after a balanced Fit, those computed.

method : public : GetClassWeights() ~ Float[]

Return

TypeDescription
Float[negative, positive]

GetError #

Why the last Fit returned false or the last prediction Nil.

method : public : GetError() ~ String

Return

TypeDescription
Stringthe reason, or Nil

GetLossHistory #

The training loss after each epoch: the class-weighted binary cross-entropy over the whole training set, averaged over its rows, with L2 left out.

method : public : GetLossHistory() ~ Float[]

Return

TypeDescription
Floatone value per epoch, or Nil before Fit

GetParameterCount #

Number of trainable parameters, weights and biases.

method : public : GetParameterCount() ~ Int

Return

TypeDescription
Intparameter count, 0 before Fit

GetWeights #

A layer's weights, inputs by outputs, for inspection: layer 0 takes the (standardised) input, the last gives the logit. A copy.

method : public : GetWeights(layer:Int) ~ Float[,]

Parameters

NameTypeDescription
layerIntlayer index

Return

TypeDescription
Floatweights, or Nil before Fit or for a layer out of range

IsFitted #

Whether Fit has succeeded.

method : public : IsFitted() ~ Bool

Return

TypeDescription
Booltrue when the model can predict

Loss #

The class-weighted binary cross-entropy of the model on labelled rows, averaged over the rows: the measure GetLossHistory records, for a validation curve.

method : public : Loss(X:Float[,], y:Bool[]) ~ Float

Parameters

NameTypeDescription
XFloatrows x features
yBoolone label per row

Return

TypeDescription
Floatthe loss, or -1.0 when X or y cannot be scored (see GetError)

New # constructor

A head with one hidden layer of 64, learning rate 0.001, 20 epochs, batch 32, seed 42.

New()

New # constructor

A head.

New(hidden:Int[], learning_rate:Float, epochs:Int, batch_size:Int, seed:Int)

Parameters

NameTypeDescription
hiddenInthidden-layer sizes; an empty array is logistic regression
learning_rateFloatstep size
epochsIntpasses over the training data
batch_sizeIntrows per update
seedIntseed for the initial weights and the shuffles

PredictClass #

Predicts each row's class: positive when its probability is at least threshold.

method : public : PredictClass(X:Float[,], threshold:Float) ~ Bool[]

Parameters

NameTypeDescription
XFloatrows x features
thresholdFloatthe cut, e.g. 0.5

Return

TypeDescription
Boolone label per row, or Nil (see GetError)

PredictClass #

Predicts each row's class at a threshold of 0.5.

method : public : PredictClass(X:Float[,]) ~ Bool[]

Parameters

NameTypeDescription
XFloatrows x features

Return

TypeDescription
Boolone label per row, or Nil (see GetError)

PredictProba #

The probability of the positive class for each row.

method : public : PredictProba(X:Float[,]) ~ Float[]

Parameters

NameTypeDescription
XFloatrows x features, as many as Fit saw

Return

TypeDescription
Floatone probability per row, or Nil (see GetError)

PredictScore #

Scores each row, higher meaning more positive: the positive-class probability, for CrossValidation and the threshold tools in Metrics.

method : public : PredictScore(X:Float[,]) ~ Float[]

Parameters

NameTypeDescription
XFloatrows x features

Return

TypeDescription
Floatone score per row, or Nil (see GetError)

SetBalancedWeights #

Weights each class by n / (2 x its count) at Fit, so both classes carry equal total weight -- scikit-learn's class_weight="balanced".

method : public : SetBalancedWeights() ~ Nil

SetClassWeights #

Weights each training row's loss by its class, for imbalanced data: a positive weight of 99 makes one missed positive cost as much as 99 false alarms. Both 1.0 by default.

method : public : SetClassWeights(negative:Float, positive:Float) ~ Nil

Parameters

NameTypeDescription
negativeFloatweight of a negative row
positiveFloatweight of a positive row

SetL2 #

Sets L2 regularisation of the weights (not the biases); 0.0 by default.

method : public : SetL2(l2:Float) ~ Nil

Parameters

NameTypeDescription
l2Floatstrength

SetOptimizer #

Sets the optimizer; Adam by default.

method : public : SetOptimizer(optimizer:EmbeddingClassifier->Optimizer) ~ Nil

Parameters

NameTypeDescription
optimizerEmbeddingClassifier->OptimizerADAM or SGD

SetStandardize #

Whether inputs are standardised with the training mean and deviation; true by default.

method : public : SetStandardize(standardize:Bool) ~ Nil

Parameters

NameTypeDescription
standardizeBoolfalse to train on the inputs as given