v2026.11.2
All Bundles
Bundle

In-memory query engine over typed columns. Tables are filled row by row or loaded from sources (CSV files, folders, the program's own objects) and queried with SQL-like statements; aggregates, Describe() and Finder (file search) are included. Compile with -lib query.

TextSource

A text file read with a regular expression: each line (or each multi-line record) is searched for the pattern, and its numbered capture groups become the named columns. Columns are typed by reading all their values, as CsvSource types CSV columns. Lines that do not match are counted, and can be kept in a second table '_unmatched' with the line number and text. Training logs, server logs and instrument output are the usual inputs. A name ending in '.gz' is decompressed first.

Implements: Source

Example

# epoch 3 loss 0.4123 acc 0.871
source := TextSource->New("train.log", "epoch (\\d+) loss ([0-9.]+) acc ([0-9.]+)", ["epoch", "loss", "acc"]);
log := source->Read();
log->Query("select epoch, loss from train where acc > 0.85")->ToString()->PrintLine();

Operations

GetError #

Gets the reason the last read failed

method : public : GetError() ~ String

Return

TypeDescription
Stringerror, Nil if none

GetTables #

Gets the tables the file provides; column types are known only after reading it

method : public : GetTables() ~ TableSpec[]

Return

TypeDescription
TableSpecthe table, and the unmatched-lines table when kept

GetUnmatched #

Gets the lines that did not match, when they are kept

method : public : GetUnmatched() ~ Table

Return

TypeDescription
Tabletable of line numbers and text

GetUnmatchedCount #

Gets how many lines or records did not match in the last read

method : public : GetUnmatchedCount() ~ Int

Return

TypeDescription
Intcount

Load #

Loads the file into a catalog, with the unmatched-lines table when kept

method : public : Load(catalog:Catalog, hint:ScanHint) ~ Bool

Parameters

NameTypeDescription
catalogCatalogcatalog
hintScanHintcolumns and rows needed, Nil for everything

Return

TypeDescription
Booltrue if loaded

New # constructor

Constructor; the table is named after the file

New(path:String, pattern:String, names:String[])

Parameters

NameTypeDescription
pathStringtext file
patternStringregular expression with one capture group per column
namesStringcolumn names, one per capture group in order

New # constructor

Constructor

New(path:String, name:String, pattern:String, names:String[], record_start:String, keep_unmatched:Bool)

Parameters

NameTypeDescription
pathStringtext file
nameStringtable name
patternStringregular expression with one capture group per column
namesStringcolumn names, one per capture group in order
record_startStringa regular expression a line matches at its start when it begins a new record; lines up to the next such line are joined with '\n'. Empty for one record per line.
keep_unmatchedBooltrue to keep lines that do not match in a second table

Read #

Reads the file as a table

method : public : Read() ~ Table

Return

TypeDescription
Tabletable, Nil on error (see GetError)

Read #

Reads the file as a table

method : public : Read(hint:ScanHint) ~ Table

Parameters

NameTypeDescription
hintScanHintcolumns and rows needed, Nil for everything

Return

TypeDescription
Tabletable, Nil on error (see GetError)