Convolutional neural networks on the CPU: layers with forward and backward passes, the Adam and SGD optimizers, an image dataset with train, validation and test splits and on-the-fly augmentation, and a training loop that records the metrics of every epoch. The layers are native (libobjk_ml, Eigen plus a small thread pool); this library holds the model API, the dataset and the trainer. Compile with -lib ml_cnn,ml,ml_lab,opencv,csv,json,cipher,gen_collect. Design, validation and measurements: docs/AI.md, "Convolutional networks (System.ML.Cnn)".
ImageDataset
A set of labelled images for CnnTrainer: file paths (decoded and resized when a batch needs them) or in-memory pixels, each with a class label of 0 or 1. Every image is resized to size x size. A bad file is reported by Load and counted, never fatal.
Split(train, validation, seed) deals the images into three ImageSubsets by a seeded shuffle, so the same seed gives the same split on every platform, and the remainder is the test set. Only the training subset is augmented. When the data is already split into folders, make one dataset per folder and use All.
Example
data := ImageDataset->New(64, 3);
data->AddDirectory("real", 0);
data->AddDirectory("fake", 1);
splits := data->Split(0.7, 0.15, 42); # train, validation, test
train := splits[0];
train->SetAugmentation(aug);Operations
- New
- AddDirectory
- AddFile
- AddPixels
- All
- Begin
- Check
- Count
- GetError
- GetLabel
- GetPath
- GetSize
- GetThreads
- SetCache
- SetThreads
- Split
- Synthetic
AddDirectory #
Adds every .jpg, .jpeg, .png and .bmp file of a folder (not its subfolders), in name order so the same folder always gives the same indexes.
method : public : AddDirectory(directory:String, cls:Int) ~ IntParameters
| Name | Type | Description |
|---|---|---|
| directory | String | folder |
| cls | Int | the class label, 0 or 1 for all of them |
Return
| Type | Description |
|---|---|
| Int | how many were added, 0 if the folder is missing |
AddFile #
Adds an image file. It is decoded on use, so a missing or bad file shows up as a refused batch, or earlier with Check.
method : public : AddFile(path:String, cls:Int) ~ BoolParameters
| Name | Type | Description |
|---|---|---|
| path | String | image file |
| cls | Int | the class label, 0 or 1 |
Return
| Type | Description |
|---|---|
| Bool | true if added |
AddPixels #
Adds an image held in memory.
method : public : AddPixels(pixels:Byte[], cls:Int) ~ BoolParameters
| Name | Type | Description |
|---|---|---|
| pixels | Byte | size x size x channels bytes, interleaved BGR (or one gray channel; 1, 3 or 4 channels) |
| cls | Int | the class label, 0 or 1 |
Return
| Type | Description |
|---|---|
| Bool | true if added, false if the length does not fit size x size x (1, 3 or 4) |
All #
All the images as one subset.
method : public : All(augment:Bool) ~ ImageSubsetParameters
| Name | Type | Description |
|---|---|---|
| augment | Bool | augment them as they are loaded (for a training set) |
Return
| Type | Description |
|---|---|
| ImageSubset | the subset |
Begin #
Starts decoding the images with these dataset indexes on the decoding threads and returns at once. Images held in memory (added as pixels, or already cached) need no decoding. Read the images with PendingImages->Get, which waits for them.
method : public : Begin(indexes:Int[]) ~ PendingImagesParameters
| Name | Type | Description |
|---|---|---|
| indexes | Int | dataset indexes, in the order Get will be asked for them |
Return
| Type | Description |
|---|---|
| PendingImages | the pending images, or Nil for an index out of range |
Check #
Decodes every image once and reports how many failed.
method : public : Check() ~ IntReturn
| Type | Description |
|---|---|
| Int | the number of images that cannot be decoded |
GetError #
Why the last call refused.
method : public : GetError() ~ StringReturn
| Type | Description |
|---|---|
| String | the reason |
GetLabel #
The class label of an image.
method : public : GetLabel(index:Int) ~ IntParameters
| Name | Type | Description |
|---|---|---|
| index | Int | image index |
Return
| Type | Description |
|---|---|
| Int | 0 or 1, or -1 for no such image |
GetPath #
The file path an image was added from.
method : public : GetPath(index:Int) ~ StringParameters
| Name | Type | Description |
|---|---|---|
| index | Int | image index |
Return
| Type | Description |
|---|---|
| String | the path, or an empty string for an image added as pixels or an index out of range |
GetThreads #
The decoding thread count.
method : public : GetThreads() ~ IntReturn
| Type | Description |
|---|---|
| Int | threads |
New # constructor
Creates an empty dataset.
New(size:Int, channels:Int)Parameters
| Name | Type | Description |
|---|---|---|
| size | Int | images are resized to size x size |
| channels | Int | channels the model takes, 1 or 3 (informational: the model converts) |
SetCache #
Keep decoded images in memory after the first load (fast epochs, but size x size x 3 bytes per image). Off by default.
method : public : SetCache(cache:Bool) ~ NilParameters
| Name | Type | Description |
|---|---|---|
| cache | Bool | true to cache |
SetThreads #
How many threads decode and resize image files for a batch (default 4). Several images of a batch are decoded at once; the pixels, the training order and the results do not depend on the count, only the time does. 1 decodes one image at a time, as before. This is separate from CnnRuntime->SetThreads, which sets the threads of the layers.
method : public : SetThreads(threads:Int) ~ NilParameters
| Name | Type | Description |
|---|---|---|
| threads | Int | 1 to 64 |
Split #
Deals the images into training, validation and test subsets by a seeded shuffle. The training subset is the one that is augmented, once SetAugmentation is called on it.
method : public : Split(train_fraction:Float, validation_fraction:Float, seed:Int) ~ ImageSubset[]Parameters
| Name | Type | Description |
|---|---|---|
| train_fraction | Float | share for training, above 0 |
| validation_fraction | Float | share for validation, 0 or more; what is left over is the test set |
| seed | Int | shuffle seed |
Return
| Type | Description |
|---|---|
| ImageSubset | three subsets: training, validation, test (Nil if the fractions are not valid or there are no images) |
Synthetic # function
A procedurally generated two-class set for demos and tests, with no files. Every image is a random-orientation, random-phase sine texture plus noise on a coloured background: class 0 has a low spatial frequency (1 to 3 cycles across the image), class 1 a high one (6 to 10 cycles), so a convolutional network learns to tell them apart in seconds. The images alternate class 0, 1, 0, 1, ..., and the same seed gives the same images on every platform (it uses an integer generator, not the VM's random numbers).
function : Synthetic(size:Int, per_class:Int, seed:Int) ~ ImageDatasetParameters
| Name | Type | Description |
|---|---|---|
| size | Int | image side in pixels, 8 or more |
| per_class | Int | images of each class, 1 or more |
| seed | Int | generator seed |
Return
| Type | Description |
|---|---|
| ImageDataset | the dataset (3 channels, in memory), or Nil for arguments out of range |