v2026.11.2
All Bundles
Bundle

Core machine learning types: the seedable Random generator, Matrix2D linear algebra and activations, matrix reference holders, the NeuralNetwork and CSV-backed MatrixReader. Compile with -lib ml.

KernelSVM

A kernel support vector machine for binary classification (C-SVC), with scikit-learn's SVC parameters: kernel, C, gamma ('scale', 'auto' or a value), degree, coef0, tol, shrinking, cache size, max_iter and class weights.

It is trained by LIBSVM's SMO solver -- second-order working-set selection, shrinking, and an LRU cache of kernel rows sized in megabytes -- so it finds the exact dual optimum to the tolerance, deterministically: there is no randomness in fitting and no seed. The linear SVM class is a different model (a primal subgradient method parameterised by lambda); use this one for kernels or for scikit-learn's C.

Cost grows between n^2 and n^3 in the training rows, as for LIBSVM: tens of thousands of rows is practical, the full 126k-row NSL-KDD training set is not. Subsample for parameter sweeps. Scale the features first (FeatureScaler): RBF, polynomial and sigmoid kernels are distances and dot products, and an unscaled column dominates them.

DecisionFunction returns a signed margin, higher meaning more positive, which is the score Metrics->AucRoc, RecallAtFpr and ThresholdAtFpr take.

Implements: ScoreModel

Example

svm := KernelSVM->New(SvmKernel->RBF, 10.0, 0.1);
if(svm->Fit(x_train, y_train)) {
  scores := svm->DecisionFunction(x_test);
  auc := Metrics->AucRoc(scores, y_test);
  labels := svm->PredictClass(x_test);
};

Operations

DecisionFunction #

The signed distance-like margin of each row: sum of alpha_i y_i K(x_i, x) over the support vectors, plus the intercept. Positive means the true class; the magnitude grows with confidence but is not a probability. This is the score for Metrics->AucRoc, RecallAtFpr and ThresholdAtFpr.

method : public : DecisionFunction(X:Float[,]) ~ Float[]

Parameters

NameTypeDescription
XFloatrows x features, with the column count used at Fit

Return

TypeDescription
Floatone value per row, or Nil when unfitted or the shape differs

Fit #

Fits the model.

method : public : Fit(X:Float[,], y:Bool[]) ~ Bool

Parameters

NameTypeDescription
XFloatrows x features
yBoolone label per row, true for the positive class

Return

TypeDescription
Booltrue when fitted; false on bad input or parameters, with GetLastError saying why

GetBoundedSupportVectorCount #

Number of support vectors at their upper bound C (margin violators).

method : public : GetBoundedSupportVectorCount() ~ Int

Return

TypeDescription
Intcount, or 0 when unfitted

GetC #

C.

method : public : GetC() ~ Float

Return

TypeDescription
FloatC

GetCacheSize #

The kernel-row cache size.

method : public : GetCacheSize() ~ Int

Return

TypeDescription
Intmegabytes

GetCoef0 #

The POLY/SIGMOID independent term.

method : public : GetCoef0() ~ Float

Return

TypeDescription
Floatcoef0

GetDegree #

The POLY degree.

method : public : GetDegree() ~ Int

Return

TypeDescription
Intdegree

GetDualCoefficients #

alpha_i * y_i per support vector (y as +1/-1), scikit-learn's dual_coef_.

method : public : GetDualCoefficients() ~ Float[]

Return

TypeDescription
Floatcoefficients, or Nil when unfitted

GetFitSeconds #

Seconds the last Fit took.

method : public : GetFitSeconds() ~ Float

Return

TypeDescription
Floatelapsed seconds

GetGamma #

The gamma in use: after Fit the value the kernel used, including the one 'scale' or 'auto' resolved to; before Fit the explicit value, or -1.0 for 'scale' and -2.0 for 'auto'.

method : public : GetGamma() ~ Float

Return

TypeDescription
Floatgamma

GetIntercept #

The intercept b of the decision function (-rho in LIBSVM's terms).

method : public : GetIntercept() ~ Float

Return

TypeDescription
Floatintercept

GetIterations #

Solver iterations used by the last Fit.

method : public : GetIterations() ~ Int

Return

TypeDescription
Intiterations

GetKernel #

The kernel.

method : public : GetKernel() ~ SvmKernel

Return

TypeDescription
SvmKernelkernel

GetLastError #

Why the last Fit returned false.

method : public : GetLastError() ~ String

Return

TypeDescription
Stringa reason, or an empty string after a successful Fit

GetMaxIterations #

The iteration cap as set; 0 or less means LIBSVM's own.

method : public : GetMaxIterations() ~ Int

Return

TypeDescription
Intcap

GetObjective #

The dual objective 1/2 a'Qa - sum(a) at the solution.

method : public : GetObjective() ~ Float

Return

TypeDescription
Floatobjective

GetSupportIndices #

Training-row indexes of the support vectors, ascending (scikit-learn's support_, which groups them by class instead).

method : public : GetSupportIndices() ~ Int[]

Return

TypeDescription
Intindexes, or Nil when unfitted

GetSupportVectorCount #

Number of support vectors (training rows with a non-zero dual coefficient).

method : public : GetSupportVectorCount() ~ Int

Return

TypeDescription
Intcount, or 0 when unfitted

GetSupportVectors #

The support vectors, one row each, in GetSupportIndices order.

method : public : GetSupportVectors() ~ Float[,]

Return

TypeDescription
Floatsupport vectors x features, or Nil when unfitted

GetTolerance #

The stopping tolerance.

method : public : GetTolerance() ~ Float

Return

TypeDescription
Floattol

IsConverged #

Whether the last Fit met the tolerance; false when the iteration cap stopped it first.

method : public : IsConverged() ~ Bool

Return

TypeDescription
Booltrue when converged

IsFitted #

Whether Fit has succeeded.

method : public : IsFitted() ~ Bool

Return

TypeDescription
Booltrue when fitted

IsShrinking #

Whether shrinking is on.

method : public : IsShrinking() ~ Bool

Return

TypeDescription
Booltrue when shrinking

New # constructor

Constructor with scikit-learn's defaults: C = 1.0, gamma = 'scale', degree = 3, coef0 = 0.0, tol = 0.001, shrinking on, a 200 MB cache and no iteration cap beyond LIBSVM's own.

New(kernel:SvmKernel)

Parameters

NameTypeDescription
kernelSvmKernelkernel function

New # constructor

Constructor with a regularisation strength; gamma is 'scale'.

New(kernel:SvmKernel, c:Float)

Parameters

NameTypeDescription
kernelSvmKernelkernel function
cFloatC, the penalty on margin violations; must be > 0

New # constructor

Constructor with C and an explicit gamma.

New(kernel:SvmKernel, c:Float, gamma:Float)

Parameters

NameTypeDescription
kernelSvmKernelkernel function
cFloatC, the penalty on margin violations; must be > 0
gammaFloatkernel coefficient for RBF, POLY and SIGMOID; must be >= 0

New # constructor

Constructor with every kernel parameter.

New(kernel:SvmKernel, c:Float, gamma:Float, degree:Int, coef0:Float)

Parameters

NameTypeDescription
kernelSvmKernelkernel function
cFloatC, the penalty on margin violations; must be > 0
gammaFloatkernel coefficient for RBF, POLY and SIGMOID; must be >= 0
degreeIntPOLY degree; must be >= 0
coef0Floatindependent term of POLY and SIGMOID

PredictClass #

Class labels: true where the decision value is above zero. Prefer DecisionFunction with Metrics->ThresholdAtFpr when false alarms are budgeted.

method : public : PredictClass(X:Float[,]) ~ Bool[]

Parameters

NameTypeDescription
XFloatrows x features

Return

TypeDescription
Boolone label per row, or Nil when unfitted

PredictScore #

The ScoreModel score: the same values as DecisionFunction.

method : public : PredictScore(X:Float[,]) ~ Float[]

Parameters

NameTypeDescription
XFloatrows x features

Return

TypeDescription
Floatone score per row, or Nil when unfitted

Score #

Accuracy of PredictClass against labels.

method : public : Score(X:Float[,], y:Bool[]) ~ Float

Parameters

NameTypeDescription
XFloatrows x features
yBoolone label per row

Return

TypeDescription
Floatfraction correct, or 0.0 when unfitted or the sizes differ

SetC #

Sets C, the penalty on margin violations.

method : public : SetC(c:Float) ~ Nil

Parameters

NameTypeDescription
cFloatC; must be > 0

SetCacheSize #

Sets the kernel-row cache size. Larger caches recompute fewer kernel rows; the result does not depend on it.

method : public : SetCacheSize(megabytes:Int) ~ Nil

Parameters

NameTypeDescription
megabytesIntcache size in MB; must be >= 1

SetClassWeight #

Weights C per class: a row's bound is C times its class's weight, so a weight above 1 makes errors on that class costlier.

method : public : SetClassWeight(negative:Float, positive:Float) ~ Nil

Parameters

NameTypeDescription
negativeFloatweight for false labels; must be > 0
positiveFloatweight for true labels; must be > 0

SetClassWeightBalanced #

Weights C per class by n / (2 * rows in the class) at Fit, scikit-learn's class_weight='balanced', so a rare class is not traded away.

method : public : SetClassWeightBalanced() ~ Nil

SetCoef0 #

Sets the independent term of the POLY and SIGMOID kernels.

method : public : SetCoef0(coef0:Float) ~ Nil

Parameters

NameTypeDescription
coef0Floatoffset

SetDegree #

Sets the POLY degree.

method : public : SetDegree(degree:Int) ~ Nil

Parameters

NameTypeDescription
degreeIntdegree; must be >= 0

SetGamma #

Sets an explicit gamma.

method : public : SetGamma(gamma:Float) ~ Nil

Parameters

NameTypeDescription
gammaFloatkernel coefficient; must be >= 0

SetGammaAuto #

Uses gamma = 1 / features at Fit, scikit-learn's 'auto'.

method : public : SetGammaAuto() ~ Nil

SetGammaScale #

Uses gamma = 1 / (features * X.var()) at Fit, scikit-learn's 'scale' and the default. X.var() is the population variance of every entry of X; when it is zero gamma is 1.0.

method : public : SetGammaScale() ~ Nil

SetMaxIterations #

Caps the solver's iterations. When the cap is reached Fit still returns true and the model is usable, but IsConverged is false.

method : public : SetMaxIterations(max_iter:Int) ~ Nil

Parameters

NameTypeDescription
max_iterIntiteration cap; 0 or less means LIBSVM's own cap, max(10,000,000, 100 * rows)

SetShrinking #

Turns LIBSVM's shrinking heuristic on or off. It changes speed, and the path to the optimum, not the optimum.

method : public : SetShrinking(on:Bool) ~ Nil

Parameters

NameTypeDescription
onBooltrue to shrink

SetTolerance #

Sets the stopping tolerance: training stops when the largest violation of the optimality conditions is below it (LIBSVM's eps, scikit-learn's tol).

method : public : SetTolerance(tol:Float) ~ Nil

Parameters

NameTypeDescription
tolFloattolerance; must be > 0