Image embeddings from an exported CLIP image encoder, and the plumbing to score face-verification pairs with them: the preprocessing CLIP was trained with, an encoder that runs an ONNX image tower, and a reader for LFW-style pair lists.
ClipPreprocessor does what Hugging Face's CLIPImageProcessor does: resize the shortest side to 224 with a bicubic filter, centre-crop 224 x 224, read the pixels as RGB, scale to 0..1 and normalise with the CLIP mean (0.48145466, 0.4578275, 0.40821073) and standard deviation (0.26862954, 0.26130258, 0.27577711), as a float tensor in CHW order. The default resize reproduces Pillow's bicubic filter bit for bit (its antialiasing support scaling and its 8-bit fixed-point arithmetic, written in Objeck, so no native code is added). OpenCV's INTER_CUBIC does not antialias when it shrinks an image and uses a different kernel, so it gives pixels that differ by up to a few grey levels; it is available as a mode and measured against the default in docs/AI.md.
ClipEncoder opens a CLIP image ONNX model (input pixel_values, output the embedding), embeds one image or a batch, and optionally L2-normalises. It scores pairs by cosine similarity, embedding every distinct image once. FacePairs reads the standard LFW pair lists (pairs.txt and friends) or a plain CSV.
Models are exported from the Hugging Face checkpoints, for example with torch.onnx.export of CLIPModel.get_image_features; the text tower is not needed. Nothing here ends the program on bad input: a call that cannot do what it was asked returns Nil or false and the object keeps the reason, read with GetError(), which is "" when there is nothing to report. A model that fails to open is not an error at construction (see IsOpen). Not thread-safe: use one object per thread. Compile with -lib ml_clip,ml_verify,opencv,onnx,models,cipher,json,gen_collect.
ClipPreprocessor->Resize
How the shortest side is brought to the target size.
Items
- PIL_BICUBIC — Pillow's bicubic resize with antialiasing, in 8-bit fixed point as Pillow computes it: what CLIPImageProcessor does.
- OPENCV_CUBIC — OpenCV's INTER_CUBIC (Keys' kernel, a = -0.75, no antialiasing when shrinking). Differs from the reference by a few grey levels.