Evaluate # function

Runs stratified k-fold cross-validation and reports per-fold scores. The model is refitted on each fold's training split, so it is left holding the last fold's fit. Standardization, when asked for, fits the scaler on the TRAINING split only and reapplies it to the test split -- scaling test data by its own statistics leaks the test distribution and flatters any model measured at a fixed false-alarm rate.

function : Evaluate(model:ScoreModel, X:Float[,], y:Bool[], k:Int, seed:Int, max_fpr:Float, standardize:Bool) ~ CrossValResult

Parameters

Return

Evaluate # function

Cross-validation whose folds are stratified by a key of your choosing. The overload above stratifies by the binary label, which keeps the positive rate steady across folds but says nothing about what the positives are made of. When the positive class is a union of rare sub-populations -- attack families, fault types, device models -- a fold can be balanced on the label and still hold none of a sub-population that only has a few dozen members. Pass those category codes as strata and each fold keeps their proportions too. strata is a fold-assignment key only. The model still fits and scores against y, so the metrics are unchanged.

function : Evaluate(model:ScoreModel, X:Float[,], y:Bool[], k:Int, seed:Int, max_fpr:Float, standardize:Bool, strata:Float[]) ~ CrossValResult

Parameters

Return

Evaluate # function

Cross-validation with standardization on, which is the usual case.

function : Evaluate(model:ScoreModel, X:Float[,], y:Bool[], k:Int, seed:Int, max_fpr:Float) ~ CrossValResult

Parameters

Return

KFold # native

Splits data into K folds for cross-validation

function : native : KFold(data:Float[,], k:Int) ~ Vector[]<IntRef>

Parameters

Return

Split # function

Builds the train/test matrices for one fold. StratifiedKFold returns row indexes; turning those into matrices is about twenty lines of copying that every experiment otherwise rewrites, and getting it subtly wrong shifts labels against features without any error. Takes ONE fold's row indexes rather than the whole array plus an index: a `Vector[]` parameter compiles inside a library but does not resolve across a library boundary, so the caller passes `folds[i]`.

function : Split(X:Float[,], y:Bool[], test_rows:Vector<IntRef>) ~ FoldSplit

Parameters

Return

StratifiedKFold # function

Splits row indices into K folds that each keep the class ratio of the whole, shuffled from a caller-supplied seed so the same seed gives the same folds. KFold above shuffles with the global generator and ignores labels. Both hurt on imbalanced data: an unseeded split cannot be reproduced, and an unstratified split lets the rare class land unevenly, so a fold may hold too few positives to measure anything at a low false-positive rate. Intrusion-detection sets are exactly that shape. Classes are taken from the label values as they first appear. Each class is shuffled independently and dealt round-robin across the folds, continuing from where the previous class stopped, so a remainder cannot pile into fold 0.

function : StratifiedKFold(labels:Float[], k:Int, seed:Int) ~ Vector[]<IntRef>

Parameters

Return

Example

folds := CrossValidation->StratifiedKFold(y, 5, 42);
each(f : folds) {
  test_rows := folds[f];
};

ToBool # function

Converts scored labels to the Bool[] the classifiers and Metrics take. TableEncoder hands back Float[] and StratifiedKFold wants that form, but every classifier Fit and every Metrics function takes Bool[]. Without this, each caller keeps both representations alive and writes the same conversion loop.

function : ToBool(labels:Float[]) ~ Bool[]

Parameters

Return

ToFloat # function

Converts Bool labels to the Float[] StratifiedKFold takes.

function : ToFloat(labels:Bool[]) ~ Float[]

Parameters

Return

TrainTestSplit # native

Splits data into training and test sets

function : native : TrainTestSplit(data:Float[,], test_ratio:Float) ~ FloatMatrixRef[]

Parameters

Return