In-memory query engine over typed columns. Tables are filled row by row or loaded from sources (CSV files, folders, the program's own objects) and queried with SQL-like statements; aggregates, Describe() and Finder (file search) are included. Compile with -lib query.
TextSource
A text file read with a regular expression: each line (or each multi-line record) is searched for the pattern, and its numbered capture groups become the named columns. Columns are typed by reading all their values, as CsvSource types CSV columns. Lines that do not match are counted, and can be kept in a second table '
Example
# epoch 3 loss 0.4123 acc 0.871
source := TextSource->New("train.log", "epoch (\\d+) loss ([0-9.]+) acc ([0-9.]+)", ["epoch", "loss", "acc"]);
log := source->Read();
log->Query("select epoch, loss from train where acc > 0.85")->ToString()->PrintLine();Operations
GetError #
Gets the reason the last read failed
method : public : GetError() ~ StringReturn
| Type | Description |
|---|---|
| String | error, Nil if none |
GetTables #
Gets the tables the file provides; column types are known only after reading it
method : public : GetTables() ~ TableSpec[]Return
| Type | Description |
|---|---|
| TableSpec | the table, and the unmatched-lines table when kept |
GetUnmatched #
Gets the lines that did not match, when they are kept
method : public : GetUnmatched() ~ TableReturn
| Type | Description |
|---|---|
| Table | table of line numbers and text |
GetUnmatchedCount #
Gets how many lines or records did not match in the last read
method : public : GetUnmatchedCount() ~ IntReturn
| Type | Description |
|---|---|
| Int | count |
Load #
Loads the file into a catalog, with the unmatched-lines table when kept
method : public : Load(catalog:Catalog, hint:ScanHint) ~ BoolParameters
| Name | Type | Description |
|---|---|---|
| catalog | Catalog | catalog |
| hint | ScanHint | columns and rows needed, Nil for everything |
Return
| Type | Description |
|---|---|
| Bool | true if loaded |
New # constructor
Constructor; the table is named after the file
New(path:String, pattern:String, names:String[])Parameters
| Name | Type | Description |
|---|---|---|
| path | String | text file |
| pattern | String | regular expression with one capture group per column |
| names | String | column names, one per capture group in order |
New # constructor
Constructor
New(path:String, name:String, pattern:String, names:String[], record_start:String, keep_unmatched:Bool)Parameters
| Name | Type | Description |
|---|---|---|
| path | String | text file |
| name | String | table name |
| pattern | String | regular expression with one capture group per column |
| names | String | column names, one per capture group in order |
| record_start | String | a regular expression a line matches at its start when it begins a new record; lines up to the next such line are joined with '\n'. Empty for one record per line. |
| keep_unmatched | Bool | true to keep lines that do not match in a second table |