Core machine learning types: the seedable Random generator, Matrix2D linear algebra and activations, matrix reference holders, the NeuralNetwork and CSV-backed MatrixReader. Compile with -lib ml.
DenseNetwork
A multi-layer dense (fully connected) network trained by minibatch Adam on mean squared error.
NeuralNetwork above has one hidden layer of sigmoid units and answers Bool; this is the general case: any number of layers of any size, a hidden activation and an output activation (LINEAR, RELU, SIGMOID, TANH), real-valued outputs. Each layer computes act(A W + b). Weights start Glorot-uniform from the seed, biases at 0. Every epoch visits the rows in a seeded shuffle, in minibatches (default 32; the last may be shorter). The loss is the mean over rows and outputs of the squared error, as Keras's and PyTorch's MSE. Adam follows PyTorch: lr 0.001, beta1 0.9, beta2 0.999, eps 1e-8, w -= (lr / (1 - beta1^t)) m / (sqrt(v) / sqrt(1 - beta2^t) + eps).
Deterministic: the same seed and data train the same weights.
Example
net := DenseNetwork->New([4, 8, 4], DenseNetwork->Activation->RELU, DenseNetwork->Activation->LINEAR, 42);
loss := net->Fit(X, Y, 100);
Y_hat := net->Predict(X);Operations
- New
- Fit
- GetBiases
- GetError
- GetLayerSizes
- GetWeights
- Loss
- Predict
- SetAdam
- SetBatchSize
- SetBiases
- SetLearningRate
- SetWeights
- TrainBatch
Fit #
Trains on X -> Y.
method : public : Fit(X:Float[,], Y:Float[,], epochs:Int) ~ FloatParameters
| Name | Type | Description |
|---|---|---|
| X | Float | rows x input units |
| Y | Float | rows x output units |
| epochs | Int | passes over the data |
Return
| Type | Description |
|---|---|
| Float | the last epoch's mean squared error, or -1.0 on bad input (GetError says why) |
GetBiases #
A layer's biases (1 x outputs); the live matrix, not a copy.
method : public : GetBiases(layer:Int) ~ Float[,]Parameters
| Name | Type | Description |
|---|---|---|
| layer | Int | 0 for the first weight layer |
Return
| Type | Description |
|---|---|
| Float | biases, or Nil for a bad layer |
GetError #
Why the last call failed.
method : public : GetError() ~ StringReturn
| Type | Description |
|---|---|
| String | the reason, or an empty string |
GetLayerSizes #
Units per layer, input first.
method : public : GetLayerSizes() ~ Int[]Return
| Type | Description |
|---|---|
| Int | sizes, or Nil when construction failed |
GetWeights #
A layer's weights (inputs x outputs); the live matrix, not a copy.
method : public : GetWeights(layer:Int) ~ Float[,]Parameters
| Name | Type | Description |
|---|---|---|
| layer | Int | 0 for the first weight layer |
Return
| Type | Description |
|---|---|
| Float | weights, or Nil for a bad layer |
Loss #
Mean squared error over rows and outputs.
method : public : Loss(X:Float[,], Y:Float[,]) ~ FloatParameters
| Name | Type | Description |
|---|---|---|
| X | Float | rows x input units |
| Y | Float | rows x output units |
Return
| Type | Description |
|---|---|
| Float | the error, or -1.0 on bad input |
New # constructor
Constructor.
New(sizes:Int[], hidden:DenseNetwork->Activation, output:DenseNetwork->Activation, seed:Int)Parameters
| Name | Type | Description |
|---|---|---|
| sizes | Int | units per layer, input first and output last (at least two entries, each >= 1) |
| hidden | DenseNetwork->Activation | activation of every hidden layer |
| output | DenseNetwork->Activation | activation of the output layer |
| seed | Int | seeds the weights and the minibatch shuffles |
Predict #
Network outputs for each row.
method : public : Predict(X:Float[,]) ~ Float[,]Parameters
| Name | Type | Description |
|---|---|---|
| X | Float | rows x input units |
Return
| Type | Description |
|---|---|
| Float | rows x output units, or Nil on bad input |
SetAdam #
Sets Adam's moment decay rates and epsilon (defaults 0.9, 0.999, 1e-8).
method : public : SetAdam(beta1:Float, beta2:Float, epsilon:Float) ~ BoolParameters
| Name | Type | Description |
|---|---|---|
| beta1 | Float | first-moment decay in [0, 1) |
| beta2 | Float | second-moment decay in [0, 1) |
| epsilon | Float | denominator guard (> 0) |
Return
| Type | Description |
|---|---|
| Bool | false when out of range |
SetBatchSize #
Sets the minibatch size (default 32).
method : public : SetBatchSize(size:Int) ~ BoolParameters
| Name | Type | Description |
|---|---|---|
| size | Int | rows per batch (>= 1) |
Return
| Type | Description |
|---|---|
| Bool | false when out of range |
SetBiases #
Replaces a layer's biases (copied).
method : public : SetBiases(layer:Int, biases:Float[,]) ~ BoolParameters
| Name | Type | Description |
|---|---|---|
| layer | Int | 0 for the first weight layer |
| biases | Float | 1 x outputs |
Return
| Type | Description |
|---|---|
| Bool | false when the layer or the shape is wrong |
SetLearningRate #
Sets the Adam learning rate (default 0.001).
method : public : SetLearningRate(rate:Float) ~ BoolParameters
| Name | Type | Description |
|---|---|---|
| rate | Float | learning rate (> 0) |
Return
| Type | Description |
|---|---|
| Bool | false when out of range |
SetWeights #
Replaces a layer's weights (copied), e.g. to start from known values.
method : public : SetWeights(layer:Int, weights:Float[,]) ~ BoolParameters
| Name | Type | Description |
|---|---|---|
| layer | Int | 0 for the first weight layer |
| weights | Float | inputs x outputs |
Return
| Type | Description |
|---|---|
| Bool | false when the layer or the shape is wrong |