Image embeddings from an exported CLIP image encoder, and the plumbing to score face-verification pairs with them: the preprocessing CLIP was trained with, an encoder that runs an ONNX image tower, and a reader for LFW-style pair lists.
ClipPreprocessor does what Hugging Face's CLIPImageProcessor does: resize the shortest side to 224 with a bicubic filter, centre-crop 224 x 224, read the pixels as RGB, scale to 0..1 and normalise with the CLIP mean (0.48145466, 0.4578275, 0.40821073) and standard deviation (0.26862954, 0.26130258, 0.27577711), as a float tensor in CHW order. The default resize reproduces Pillow's bicubic filter bit for bit (its antialiasing support scaling and its 8-bit fixed-point arithmetic, written in Objeck, so no native code is added). OpenCV's INTER_CUBIC does not antialias when it shrinks an image and uses a different kernel, so it gives pixels that differ by up to a few grey levels; it is available as a mode and measured against the default in docs/AI.md.
ClipEncoder opens a CLIP image ONNX model (input pixel_values, output the embedding), embeds one image or a batch, and optionally L2-normalises. It scores pairs by cosine similarity, embedding every distinct image once. FacePairs reads the standard LFW pair lists (pairs.txt and friends) or a plain CSV.
Models are exported from the Hugging Face checkpoints, for example with torch.onnx.export of CLIPModel.get_image_features; the text tower is not needed. Nothing here ends the program on bad input: a call that cannot do what it was asked returns Nil or false and the object keeps the reason, read with GetError(), which is "" when there is nothing to report. A model that fails to open is not an error at construction (see IsOpen). Not thread-safe: use one object per thread. Compile with -lib ml_clip,ml_verify,opencv,onnx,models,cipher,json,gen_collect.
ClipPreprocessor
Image preprocessing for a CLIP image encoder. Immutable after construction, so one instance can be shared.
Operations
GetLength #
The number of floats Process returns: 3 x size x size.
method : public : GetLength() ~ IntReturn
| Type | Description |
|---|---|
| Int | the length |
GetMode #
The resize method.
method : public : GetMode() ~ ClipPreprocessor->ResizeReturn
| Type | Description |
|---|---|
| ClipPreprocessor->Resize | the method |
GetSize #
The side of the network input.
method : public : GetSize() ~ IntReturn
| Type | Description |
|---|---|
| Int | pixels |
New # constructor
The CLIP preprocessing for ViT-B/32, ViT-B/16 and ViT-L/14: 224 pixels, Pillow bicubic.
New()New # constructor
A preprocessor with a chosen size and resize method.
New(size:Int, mode:ClipPreprocessor->Resize)Parameters
| Name | Type | Description |
|---|---|---|
| size | Int | side of the square network input, 1 or more (224 for the three models above; 336 for ViT-L/14-336) |
| mode | ClipPreprocessor->Resize | the resize method |
Process #
Preprocesses a decoded image: resize the shortest side to the size, centre-crop, RGB, scale to 0..1, normalise.
method : public : Process(image:API.OpenCV.Image) ~ Float[]Parameters
| Name | Type | Description |
|---|---|---|
| image | Image | the decoded image (1, 3 or 4 channels; OpenCV's BGR order, an alpha channel is dropped) |
Return
| Type | Description |
|---|---|
| Float | size x size x 3 floats in CHW order; Nil when the image is Nil or empty |
ProcessFile #
Loads an image file and preprocesses it. OpenCV decodes it, so it reads what OpenCV reads (JPEG, PNG, BMP, TIFF, WebP) and applies an EXIF orientation tag, which Pillow-based code does not.
method : public : ProcessFile(path:String) ~ Float[]Parameters
| Name | Type | Description |
|---|---|---|
| path | String | image file |
Return
| Type | Description |
|---|---|
| Float | size x size x 3 floats in CHW order, RGB, normalised; Nil when the file is missing or cannot be decoded |