v2026.11.2
All Bundles
Bundle

Experiment support for a parameter sweep: a deterministic stratified hold-out split, class-ratio resampling of the training half against a fixed test set, a wall-clock stopwatch and label helpers. Compile with -lib ml,ml_lab,csv,json, or with the ai alias group. Design: docs/ML_LAB_SUPPORT_DESIGN.md.

Experiment

A sweep over one named parameter: for every value (and every seed, when repeats are asked for) it runs the Trial, scores the predictions, and collects a ResultTable with train_accuracy, test_accuracy, precision, recall, f1, fpr, false_negatives and fit_ms per value -- mean and sample standard deviation over the seeds when there is more than one.

Test-set metrics come from the test predictions; train_accuracy is the only training-set number, there for the overfitting curve. The confusion counts and rates come from System.ML's ConfusionMatrix over the trial's Bool predictions, every ratio guarded (a value that predicts nothing positive has no precision and reports 0.0).

Example

exp := Experiment->New("Random forest: trees", "n_estimators");
exp->SetValues([10.0, 50.0, 100.0, 200.0, 500.0]);
exp->SetSeeds([1, 2, 3]);
exp->AddSetting("max_depth", "8");
table := exp->Run(ForestTrial->New(split));
table->ToMarkdown()->PrintLine();

Operations

AddSetting #

Records a fixed setting for the footnote, e.g. ("max_depth", "8").

method : public : AddSetting(name:String, value:String) ~ Nil

Parameters

NameTypeDescription
nameStringthe setting
valueStringits value

New # constructor

Constructor

New(title:String, parameter:String)

Parameters

NameTypeDescription
titleStringthe table's title
parameterStringthe swept parameter's name

Run #

Runs the sweep.

method : public : Run(trial:Trial) ~ ResultTable

Parameters

NameTypeDescription
trialTrialbuilds, fits and predicts for one value

Return

TypeDescription
ResultTablethe table, one row per value; raises LabError when no values are set, the trial returns Nil, or its arrays disagree in length

SetSeeds #

Seeds for repeated runs; each value is run once per seed and the table reports the mean and sample standard deviation. The default is one run with seed 0.

method : public : SetSeeds(seeds:Int[]) ~ Nil

Parameters

NameTypeDescription
seedsIntone per repetition

SetValues #

The values to sweep, named by their number.

method : public : SetValues(values:Float[]) ~ Nil

Parameters

NameTypeDescription
valuesFloatone per trial

SetValues #

The values to sweep with display names, for a categorical parameter such as a kernel (values 0, 1, 2; names "linear", "rbf", "poly").

method : public : SetValues(values:Float[], names:String[]) ~ Nil

Parameters

NameTypeDescription
valuesFloatone per trial
namesStringone per value, or Nil to name each by its number