Core machine learning types: the seedable Random generator, Matrix2D linear algebra and activations, matrix reference holders, the NeuralNetwork and CSV-backed MatrixReader. Compile with -lib ml.
OneClassSVM
One-class SVM for outlier and novelty detection, with scikit-learn's OneClassSVM parameters (kernel, nu, gamma, degree, coef0, tol, shrinking, cache size, max_iter) and LIBSVM's formulation, solved by the same SMO solver as KernelSVM: minimise 1/2 a'Ka subject to 0 <= a_i <= 1 and sum(a) = nu * n.
nu is an upper bound on the fraction of training rows STRICTLY outside the boundary and a lower bound on the fraction that are support vectors.
The flagged training fraction can exceed nu, as with LIBSVM and scikit-learn. At the optimum the free support vectors lie exactly on the boundary, decision 0, and those that round just below 0 are flagged by IsOutlier and Predict (scikit-learn's predict is decision >= 0 too). At a finite tolerance the optimality conditions place rows only within tol of the boundary, so at the default 1e-3 rows down to about -tol are flagged: 230 rows at nu = 0.01 (nu * n = 2.3) flag 20, all within 4e-4 of 0 and none at its bound. Rows below -tol never exceed nu * n. When the flagged fraction should track nu -- a nu sweep, say -- tighten the tolerance (SetTolerance(1e-6) costs little) or count rows whose decision is below -GetTolerance().
Signs follow scikit-learn: DecisionFunction is positive inside the boundary (an inlier) and negative outside, and Predict returns +1 / -1. Because the fixed-false-positive-rate metrics treat higher scores as more positive and the positive class in intrusion detection is the attack, PredictScore is the NEGATED decision value -- higher means more anomalous -- and IsOutlier is true for an outlier. Train on normal rows only for novelty detection; Fit(X, y), the ScoreModel form, ignores y as scikit-learn does.
Example
detector := OneClassSVM->New(SvmKernel->RBF, 0.05);
if(detector->Fit(x_normal)) {
anomaly := detector->PredictScore(x_test);
auc := Metrics->AucRoc(anomaly, is_attack);
flagged := detector->IsOutlier(x_test);
};Operations
- New
- DecisionFunction
- Fit
- GetBoundedSupportVectorCount
- GetCacheSize
- GetCoef0
- GetDegree
- GetDualCoefficients
- GetFitSeconds
- GetGamma
- GetIntercept
- GetIterations
- GetKernel
- GetLastError
- GetMaxIterations
- GetNu
- GetObjective
- GetOffset
- GetSupportIndices
- GetSupportVectorCount
- GetSupportVectors
- GetTolerance
- IsConverged
- IsFitted
- IsOutlier
- IsShrinking
- Predict
- PredictScore
- ScoreSamples
- SetCacheSize
- SetCoef0
- SetDegree
- SetGamma
- SetGammaAuto
- SetGammaScale
- SetMaxIterations
- SetNu
- SetShrinking
- SetTolerance
DecisionFunction #
The signed decision value sum(alpha_i K(x_i, x)) - rho: positive inside the boundary (an inlier), negative outside (an outlier), scikit-learn's decision_function.
method : public : DecisionFunction(X:Float[,]) ~ Float[]Parameters
| Name | Type | Description |
|---|---|---|
| X | Float | rows x features, with the column count used at Fit |
Return
| Type | Description |
|---|---|
| Float | one value per row, or Nil when unfitted or the shape differs |
Fit #
Fits the boundary around the rows of X.
method : public : Fit(X:Float[,]) ~ BoolParameters
| Name | Type | Description |
|---|---|---|
| X | Float | rows x features; for novelty detection, normal rows only |
Return
| Type | Description |
|---|---|
| Bool | true when fitted; false on bad input or parameters, with GetLastError saying why |
Fit #
The ScoreModel form of Fit. y is ignored, as in scikit-learn; pass only the normal rows to Fit(X) for novelty detection.
method : public : Fit(X:Float[,], y:Bool[]) ~ BoolParameters
| Name | Type | Description |
|---|---|---|
| X | Float | rows x features |
| y | Bool | ignored |
Return
| Type | Description |
|---|---|
| Bool | true when fitted |
GetBoundedSupportVectorCount #
Number of support vectors at the bound alpha = 1; every training outlier is one.
method : public : GetBoundedSupportVectorCount() ~ IntReturn
| Type | Description |
|---|---|
| Int | count, or 0 when unfitted |
GetCacheSize #
The kernel-row cache size.
method : public : GetCacheSize() ~ IntReturn
| Type | Description |
|---|---|
| Int | megabytes |
GetCoef0 #
The POLY/SIGMOID independent term.
method : public : GetCoef0() ~ FloatReturn
| Type | Description |
|---|---|
| Float | coef0 |
GetDualCoefficients #
alpha per support vector, scikit-learn's dual_coef_; they sum to nu * n.
method : public : GetDualCoefficients() ~ Float[]Return
| Type | Description |
|---|---|
| Float | coefficients, or Nil when unfitted |
GetFitSeconds #
Seconds the last Fit took.
method : public : GetFitSeconds() ~ FloatReturn
| Type | Description |
|---|---|
| Float | elapsed seconds |
GetGamma #
The gamma in use: after Fit the value the kernel used; before Fit the explicit value, or -1.0 for 'scale' and -2.0 for 'auto'.
method : public : GetGamma() ~ FloatReturn
| Type | Description |
|---|---|
| Float | gamma |
GetIntercept #
The intercept of the decision function (-rho, scikit-learn's intercept_).
method : public : GetIntercept() ~ FloatReturn
| Type | Description |
|---|---|
| Float | intercept |
GetIterations #
Solver iterations used by the last Fit.
method : public : GetIterations() ~ IntReturn
| Type | Description |
|---|---|
| Int | iterations |
GetLastError #
Why the last Fit returned false.
method : public : GetLastError() ~ StringReturn
| Type | Description |
|---|---|
| String | a reason, or an empty string after a successful Fit |
GetMaxIterations #
The iteration cap as set; 0 or less means LIBSVM's own.
method : public : GetMaxIterations() ~ IntReturn
| Type | Description |
|---|---|
| Int | cap |
GetObjective #
The dual objective 1/2 a'Ka at the solution.
method : public : GetObjective() ~ FloatReturn
| Type | Description |
|---|---|
| Float | objective |
GetOffset #
rho, scikit-learn's offset_: ScoreSamples minus DecisionFunction.
method : public : GetOffset() ~ FloatReturn
| Type | Description |
|---|---|
| Float | offset |
GetSupportIndices #
Training-row indexes of the support vectors, ascending.
method : public : GetSupportIndices() ~ Int[]Return
| Type | Description |
|---|---|
| Int | indexes, or Nil when unfitted |
GetSupportVectorCount #
Number of support vectors (training rows with alpha > 0).
method : public : GetSupportVectorCount() ~ IntReturn
| Type | Description |
|---|---|
| Int | count, or 0 when unfitted |
GetSupportVectors #
The support vectors, one row each, in GetSupportIndices order.
method : public : GetSupportVectors() ~ Float[,]Return
| Type | Description |
|---|---|
| Float | support vectors x features, or Nil when unfitted |
GetTolerance #
The stopping tolerance.
method : public : GetTolerance() ~ FloatReturn
| Type | Description |
|---|---|
| Float | tol |
IsConverged #
Whether the last Fit met the tolerance before the iteration cap.
method : public : IsConverged() ~ BoolReturn
| Type | Description |
|---|---|
| Bool | true when converged |
IsFitted #
Whether Fit has succeeded.
method : public : IsFitted() ~ BoolReturn
| Type | Description |
|---|---|
| Bool | true when fitted |
IsOutlier #
True for an outlier (decision value below 0), the label Metrics takes when the attack is the positive class. Training rows on the boundary sit within the tolerance of 0 and can land on either side; see the class notes on nu.
method : public : IsOutlier(X:Float[,]) ~ Bool[]Parameters
| Name | Type | Description |
|---|---|---|
| X | Float | rows x features |
Return
| Type | Description |
|---|---|
| Bool | one flag per row, or Nil when unfitted |
IsShrinking #
Whether shrinking is on.
method : public : IsShrinking() ~ BoolReturn
| Type | Description |
|---|---|
| Bool | true when shrinking |
New # constructor
Constructor with scikit-learn's defaults: nu = 0.5, gamma = 'scale', degree = 3, coef0 = 0.0, tol = 0.001, shrinking on, a 200 MB cache.
New(kernel:SvmKernel)Parameters
| Name | Type | Description |
|---|---|---|
| kernel | SvmKernel | kernel function |
New # constructor
Constructor with nu; gamma is 'scale'.
New(kernel:SvmKernel, nu:Float)Parameters
| Name | Type | Description |
|---|---|---|
| kernel | SvmKernel | kernel function |
| nu | Float | bound on the training outlier fraction, in (0, 1] |
New # constructor
Constructor with nu and an explicit gamma.
New(kernel:SvmKernel, nu:Float, gamma:Float)Parameters
| Name | Type | Description |
|---|---|---|
| kernel | SvmKernel | kernel function |
| nu | Float | bound on the training outlier fraction, in (0, 1] |
| gamma | Float | kernel coefficient for RBF, POLY and SIGMOID; must be >= 0 |
New # constructor
Constructor with every kernel parameter.
New(kernel:SvmKernel, nu:Float, gamma:Float, degree:Int, coef0:Float)Parameters
| Name | Type | Description |
|---|---|---|
| kernel | SvmKernel | kernel function |
| nu | Float | bound on the training outlier fraction, in (0, 1] |
| gamma | Float | kernel coefficient for RBF, POLY and SIGMOID; must be >= 0 |
| degree | Int | POLY degree; must be >= 0 |
| coef0 | Float | independent term of POLY and SIGMOID |
Predict #
scikit-learn's predict: +1 for an inlier (decision >= 0), -1 for an outlier.
method : public : Predict(X:Float[,]) ~ Int[]Parameters
| Name | Type | Description |
|---|---|---|
| X | Float | rows x features |
Return
| Type | Description |
|---|---|
| Int | one label per row, or Nil when unfitted |
PredictScore #
The anomaly score for the metric functions: the negated decision value, so higher means more anomalous and Metrics->AucRoc / RecallAtFpr can take the attack as the positive class. Satisfies ScoreModel.
method : public : PredictScore(X:Float[,]) ~ Float[]Parameters
| Name | Type | Description |
|---|---|---|
| X | Float | rows x features |
Return
| Type | Description |
|---|---|
| Float | one score per row, or Nil when unfitted |
ScoreSamples #
The unshifted score sum(alpha_i K(x_i, x)), scikit-learn's score_samples: DecisionFunction plus GetOffset. Higher means more normal.
method : public : ScoreSamples(X:Float[,]) ~ Float[]Parameters
| Name | Type | Description |
|---|---|---|
| X | Float | rows x features |
Return
| Type | Description |
|---|---|
| Float | one value per row, or Nil when unfitted or the shape differs |
SetCacheSize #
Sets the kernel-row cache size; the result does not depend on it.
method : public : SetCacheSize(megabytes:Int) ~ NilParameters
| Name | Type | Description |
|---|---|---|
| megabytes | Int | cache size in MB; must be >= 1 |
SetCoef0 #
Sets the independent term of the POLY and SIGMOID kernels.
method : public : SetCoef0(coef0:Float) ~ NilParameters
| Name | Type | Description |
|---|---|---|
| coef0 | Float | offset |
SetDegree #
Sets the POLY degree.
method : public : SetDegree(degree:Int) ~ NilParameters
| Name | Type | Description |
|---|---|---|
| degree | Int | degree; must be >= 0 |
SetGamma #
Sets an explicit gamma.
method : public : SetGamma(gamma:Float) ~ NilParameters
| Name | Type | Description |
|---|---|---|
| gamma | Float | kernel coefficient; must be >= 0 |
SetGammaAuto #
Uses gamma = 1 / features at Fit, scikit-learn's 'auto'.
method : public : SetGammaAuto() ~ NilSetGammaScale #
Uses gamma = 1 / (features * X.var()) at Fit, scikit-learn's 'scale' and the default; 1.0 when the variance is zero.
method : public : SetGammaScale() ~ NilSetMaxIterations #
Caps the solver's iterations; when reached, Fit still returns true and IsConverged is false.
method : public : SetMaxIterations(max_iter:Int) ~ NilParameters
| Name | Type | Description |
|---|---|---|
| max_iter | Int | iteration cap; 0 or less means LIBSVM's own, max(10,000,000, 100 * rows) |
SetNu #
Sets nu: an upper bound on the fraction of training rows outside the boundary and a lower bound on the fraction of support vectors.
method : public : SetNu(nu:Float) ~ NilParameters
| Name | Type | Description |
|---|---|---|
| nu | Float | value in (0, 1] |