Objeck Documentation

Bundle

Core machine learning types: the seedable Random generator, Matrix2D linear algebra and activations, matrix reference holders, the NeuralNetwork and CSV-backed MatrixReader. Compile with -lib ml.

DecisionTreeClassifier

A classification tree that splits continuous features on a threshold, chosen by Gini impurity. DecisionTree above takes Bool[,]: every feature must already be binary, so continuous data has to be quantile-binned before it can be used, and the binning throws away exactly the thresholds the tree exists to find. This takes Float[,] and finds the threshold itself, the same way RegressionTree does for a continuous target -- sorted values, midpoints between consecutive distinct values as candidates -- with Gini in place of squared error and a class proportion at the leaf. PredictScore returns that proportion rather than a label, because a fixed false-positive budget needs a score to threshold. A Bool prediction fixes one operating point and cannot answer what the recall would be at another.

Implements: ScoreModel

Example

tree := DecisionTreeClassifier->New(8, 5);
tree->Fit(x_train, y_train);
scores := tree->PredictScore(x_test);
recall := Metrics->RecallAtFpr(scores, y_test, 0.001);

Fit #

Builds the tree.

method : public : Fit(X:Float[,], y:Bool[]) ~ Bool

Parameters

Name	Type	Description
X	Float	rows x features, continuous
y	Bool	one label per row, true for the positive class

Return

Type	Description
Bool	true when the tree was built, false when the inputs do not line up

FitRows #

Builds the tree over a chosen list of rows, which may repeat. A forest fits each tree to a bootstrap sample, and a sample drawn with replacement is a list of row indexes rather than a matrix, so passing the list avoids copying the data once per tree.

method : public : FitRows(X:Float[,], y:Bool[], rows_idx:Int[]) ~ Bool

Parameters

Name	Type	Description
X	Float	rows x features, continuous
y	Bool	one label per row of X
rows_idx	Int	row indexes to build over; repeats are allowed and meaningful

Return

Type	Description
Bool	true when the tree was built

IsFitted #

Whether Fit has been called successfully.

method : public : IsFitted() ~ Bool

Return

Type	Description
Bool	true when this tree has been built

New # constructor

Constructor. Every feature is considered at every node.

New(max_depth:Int, min_samples:Int)

Parameters

Name	Type	Description
max_depth	Int	deepest the tree may grow
min_samples	Int	a node with this many rows or fewer becomes a leaf

New # constructor

Constructor for a tree that considers only a sample of the features at each node, which is how a forest decorrelates its trees. It is also the cheapest lever on build time: the split search dominates, and it is linear in the number of features tried, so trying sqrt(f) of f is roughly that speedup.

New(max_depth:Int, min_samples:Int, max_features:Int, seed:Int)

Parameters

Name	Type	Description
max_depth	Int	deepest the tree may grow
min_samples	Int	a node with this many rows or fewer becomes a leaf
max_features	Int	features to sample per node; 0 or more than the width means all
seed	Int	seeds the feature sampling, so the same seed rebuilds the same tree

Predict #

Labels at the usual half-way threshold. Prefer PredictScore with an explicit threshold when the false-alarm rate matters.

method : public : Predict(X:Float[,]) ~ Bool[]

Parameters

Name	Type	Description
X	Float	rows x features

Return

Type	Description
Bool	one label per row, or Nil when unfitted

PredictScore #

The fraction of training rows at each row's leaf that were positive. This is the score to threshold, and what the fixed-false-positive-rate metrics consume.

method : public : PredictScore(X:Float[,]) ~ Float[]

Parameters

Name	Type	Description
X	Float	rows x features, with the same feature count used at Fit

Return

Type	Description
Float	one score per row, or Nil when unfitted or the shape does not match