In-memory query engine over typed columns. Tables are filled row by row or loaded from sources (CSV files, folders, the program's own objects) and queried with SQL-like statements; aggregates, Describe() and Finder (file search) are included. Compile with -lib query.
Column
One column of a table, held as a typed array rather than one object per cell. A column is either declared, in which case storing a value of the wrong type is refused, or inferred from its first value, in which case an Int widens into a Float column, a Float promotes an Int column to Float, and any other mix promotes the column to OBJECT so every value is kept as it was given. Empty cells are tracked in a bitmap allocated with the first one. Text columns store each distinct string once while they have few of them. Numeric and date columns keep the minimum and maximum of every 65,536-row group, so a filter can skip groups.
Operations
- New
- Append
- Compact
- CopyFrom
- DictionaryLimit
- FillInts
- FillNulls
- FillNumbers
- FindCode
- Gather
- Get
- GetCode
- GetColumnIndex
- GetDictionary
- GetDictionaryRanks
- GetDictionarySize
- GetError
- GetFloat
- GetFloatArray
- GetGmtBits
- GetGroupCount
- GetGroupMax
- GetGroupMin
- GetIndexKind
- GetIndexName
- GetInt
- GetIntArray
- GetName
- GetNullBits
- GetNullCount
- GetObjectArray
- GetPromotions
- GetString
- GetStringArray
- GetType
- GmtAt
- GroupHasValues
- GroupRows
- HasZones
- IndexDeleted
- IsDeclared
- IsEncoded
- IsNull
- IsNumeric
- KindOf
- Restore
- Set
- SetColumnIndex
- Size
- TypeName
Compact #
Keeps only the given rows, in order, and trims the arrays to fit. Used by Table->Compact.
method : public : Compact(keep:Int[]) ~ NilParameters
| Name | Type | Description |
|---|---|---|
| keep | Int | rows to keep, ascending |
CopyFrom #
Copies a cell from another column, converting through Set when the types differ
method : public : CopyFrom(src:Column, src_row:Int, row:Int) ~ BoolParameters
| Name | Type | Description |
|---|---|---|
| src | Column | column to copy from |
| src_row | Int | row in that column |
| row | Int | row in this column |
Return
| Type | Description |
|---|---|
| Bool | true if stored |
DictionaryLimit # function
Distinct strings a text column keeps in its dictionary before it may store plain strings
function : DictionaryLimit() ~ IntReturn
| Type | Description |
|---|---|
| Int | dictionary size limit |
FillInts #
Fills an empty INT column with values, none empty. Used by Table->Gather.
method : public : FillInts(values:Int[]) ~ NilParameters
| Name | Type | Description |
|---|---|---|
| values | Int | values |
FillNulls #
Fills an empty column with n empty cells. Used by Table->Gather.
method : public : FillNulls(n:Int) ~ NilParameters
| Name | Type | Description |
|---|---|---|
| n | Int | number of rows |
FillNumbers #
Refills a column with numbers. Used for query aggregates.
method : public : FillNumbers(type:ColumnType, ints:Int[], floats:Float[], present:Bool[]) ~ NilParameters
| Name | Type | Description |
|---|---|---|
| type | ColumnType | INT or FLOAT |
| ints | Int | values for an INT column |
| floats | Float | values for a FLOAT column |
| present | Bool | false where a cell is empty |
FindCode #
Gets the code of a string in an encoded column
method : public : FindCode(value:String) ~ IntParameters
| Name | Type | Description |
|---|---|---|
| value | String | string |
Return
| Type | Description |
|---|---|
| Int | code, -1 if the column does not hold it or is not encoded |
Gather #
Fills an empty column with copies of another column's cells, in the given order. The column takes the other's type. Used to copy query results.
method : public : Gather(src:Column, rows:Int[]) ~ NilParameters
| Name | Type | Description |
|---|---|---|
| src | Column | column to copy from |
| rows | Int | its row slots to copy; -1 gives an empty cell |
Get #
Gets a cell as an object: IntRef, FloatRef, String, Date, BoolRef or the stored object
method : public : Get(row:Int) ~ CompareParameters
| Name | Type | Description |
|---|---|---|
| row | Int | row number |
Return
| Type | Description |
|---|---|
| Compare | value, Nil if empty |
GetCode #
Gets the dictionary code of a cell in an encoded text column
method : public : GetCode(row:Int) ~ IntParameters
| Name | Type | Description |
|---|---|---|
| row | Int | row number |
Return
| Type | Description |
|---|---|
| Int | code, -1 if empty or not encoded |
GetColumnIndex #
Gets the column's index. Used by Table and the query planner.
method : public : GetColumnIndex() ~ ColumnIndexReturn
| Type | Description |
|---|---|
| ColumnIndex | index, Nil if none |
GetDictionary #
Gets the strings of an encoded STRING column, indexed by code
method : public : GetDictionary() ~ String[]Return
| Type | Description |
|---|---|
| String | dictionary, Nil if not encoded |
GetDictionaryRanks #
Gets each dictionary code's position in sorted order, so encoded text sorts by number
method : public : GetDictionaryRanks() ~ Int[]Return
| Type | Description |
|---|---|
| Int | rank per code, Nil if not encoded |
GetDictionarySize #
Gets the number of distinct strings in a dictionary-encoded column
method : public : GetDictionarySize() ~ IntReturn
| Type | Description |
|---|---|
| Int | dictionary size, 0 if not encoded |
GetError #
Gets the reason the last value was refused
method : public : GetError() ~ StringReturn
| Type | Description |
|---|---|
| String | error, Nil if none |
GetFloat #
Gets a numeric or date cell as a Float
method : public : GetFloat(row:Int) ~ FloatParameters
| Name | Type | Description |
|---|---|---|
| row | Int | row number |
Return
| Type | Description |
|---|---|
| Float | value, 0.0 if empty or not numeric |
GetFloatArray #
Gets the values of a FLOAT column, for code that scans the column directly
method : public : GetFloatArray() ~ Float[]Return
| Type | Description |
|---|---|
| Float | values, Nil for other types |
GetGmtBits #
Gets the array of 64-row words marking DATE cells stored as GMT dates
method : public : GetGmtBits() ~ Int[]Return
| Type | Description |
|---|---|
| Int | GMT bitmap |
GetGroupCount #
Gets the number of zone-map groups
method : public : GetGroupCount() ~ IntReturn
| Type | Description |
|---|---|
| Int | number of groups |
GetGroupMax #
Gets the largest value in a group, ignoring empty cells
method : public : GetGroupMax(group:Int) ~ FloatParameters
| Name | Type | Description |
|---|---|---|
| group | Int | group number |
Return
| Type | Description |
|---|---|
| Float | maximum (see GetGroupMin) |
GetGroupMin #
Gets the smallest value in a group, ignoring empty cells
method : public : GetGroupMin(group:Int) ~ FloatParameters
| Name | Type | Description |
|---|---|---|
| group | Int | group number |
Return
| Type | Description |
|---|---|
| Float | minimum; for a group with no values the minimum is above the maximum (see GroupHasValues) |
GetIndexKind #
Gets the kind of index the column has (see Table->CreateIndex)
method : public : GetIndexKind() ~ IndexKindReturn
| Type | Description |
|---|---|
| IndexKind | HASH, SORTED, or NONE if it has none |
GetIndexName #
Gets the name of the column's index
method : public : GetIndexName() ~ StringReturn
| Type | Description |
|---|---|
| String | index name, Nil if it has none |
GetInt #
Gets a cell of an INT, BOOL (0 or 1) or DATE (epoch milliseconds) column
method : public : GetInt(row:Int) ~ IntParameters
| Name | Type | Description |
|---|---|---|
| row | Int | row number |
Return
| Type | Description |
|---|---|
| Int | value, 0 if empty |
GetIntArray #
Gets the values of an INT, DATE or BOOL column, or the codes of an encoded STRING column, for code that scans the column directly
method : public : GetIntArray() ~ Int[]Return
| Type | Description |
|---|---|
| Int | values, Nil for other types |
GetName #
Gets the column name
method : public : GetName() ~ StringReturn
| Type | Description |
|---|---|
| String | column name |
GetNullBits #
Gets the array of 64-row words marking empty cells, for code that scans the column directly. Valid until the column next grows or changes type.
method : public : GetNullBits() ~ Int[]Return
| Type | Description |
|---|---|
| Int | null bitmap |
GetNullCount #
Gets the number of empty cells
method : public : GetNullCount() ~ IntReturn
| Type | Description |
|---|---|
| Int | number of empty cells |
GetObjectArray #
Gets the values of an OBJECT column
method : public : GetObjectArray() ~ Compare[]Return
| Type | Description |
|---|---|
| Compare | values, Nil for other types |
GetPromotions #
Gets how many times the column changed type to keep a value
method : public : GetPromotions() ~ IntReturn
| Type | Description |
|---|---|
| Int | number of promotions |
GetString #
Gets a cell of a STRING column
method : public : GetString(row:Int) ~ StringParameters
| Name | Type | Description |
|---|---|---|
| row | Int | row number |
Return
| Type | Description |
|---|---|
| String | value, Nil if empty or not a text column |
GetStringArray #
Gets the values of a plain (not encoded) STRING column
method : public : GetStringArray() ~ String[]Return
| Type | Description |
|---|---|
| String | values, Nil for other columns |
GetType #
Gets the storage type
method : public : GetType() ~ ColumnTypeReturn
| Type | Description |
|---|---|
| ColumnType | storage type |
GmtAt #
Whether a DATE cell was stored as a GMT date
method : public : GmtAt(row:Int) ~ BoolParameters
| Name | Type | Description |
|---|---|---|
| row | Int | row number |
Return
| Type | Description |
|---|---|
| Bool | true if GMT |
GroupHasValues #
Whether a group holds any non-empty value
method : public : GroupHasValues(group:Int) ~ BoolParameters
| Name | Type | Description |
|---|---|---|
| group | Int | group number |
Return
| Type | Description |
|---|---|
| Bool | true if the group has values |
GroupRows # function
Rows in a zone-map group
function : GroupRows() ~ IntReturn
| Type | Description |
|---|---|
| Int | rows per group |
HasZones #
Whether the column keeps zone maps: INT, FLOAT and DATE columns do
method : public : HasZones() ~ BoolReturn
| Type | Description |
|---|---|
| Bool | true if zone maps are kept |
IndexDeleted #
Tells the column's index that a row was deleted from its table. Used by Table->Delete.
method : public : IndexDeleted(row:Int) ~ NilParameters
| Name | Type | Description |
|---|---|---|
| row | Int | row slot |
IsDeclared #
Whether the column's type was declared, so mismatched values are refused
method : public : IsDeclared() ~ BoolReturn
| Type | Description |
|---|---|
| Bool | true if declared |
IsEncoded #
Whether a text column stores its values as dictionary codes
method : public : IsEncoded() ~ BoolReturn
| Type | Description |
|---|---|
| Bool | true if dictionary encoded |
IsNull #
Whether a cell is empty
method : public : IsNull(row:Int) ~ BoolParameters
| Name | Type | Description |
|---|---|---|
| row | Int | row number |
Return
| Type | Description |
|---|---|
| Bool | true if empty or out of range |
IsNumeric #
Whether a cell holds a number an aggregate can use: an Int or a Float
method : public : IsNumeric(row:Int) ~ BoolParameters
| Name | Type | Description |
|---|---|---|
| row | Int | row number |
Return
| Type | Description |
|---|---|
| Bool | true if numeric |
KindOf # function
Gets the storage type a value calls for
function : KindOf(value:Compare) ~ ColumnTypeParameters
| Name | Type | Description |
|---|---|---|
| value | Compare | value |
Return
| Type | Description |
|---|---|
| ColumnType | storage type |
New # constructor
Constructor
New(name:String, type:ColumnType, declared:Bool)Parameters
| Name | Type | Description |
|---|---|---|
| name | String | column name |
| type | ColumnType | storage type; NONE takes the type of the first value stored |
| declared | Bool | true to refuse values of another type |
Restore #
Refills a column with stored arrays, as an .otab file holds them. Used by OtabFile.
method : public : Restore(type:ColumnType, declared:Bool, n:Int, ints:Int[], floats:Float[], strings:String[], objects:Compare[], nulls:Int[], null_count:Int, gmt:Int[], encoded:Bool, dict:String[]) ~ NilParameters
| Name | Type | Description |
|---|---|---|
| type | ColumnType | column type |
| declared | Bool | true to refuse values of another type afterwards |
| n | Int | rows |
| ints | Int | INT, BOOL and DATE values, or dictionary codes of an encoded STRING column |
| floats | Float | FLOAT values |
| strings | String | plain STRING values |
| objects | Compare | OBJECT values |
| nulls | Int | null bitmap |
| null_count | Int | empty cells |
| gmt | Int | DATE GMT bitmap |
| encoded | Bool | true if a STRING column is dictionary encoded |
| dict | String | dictionary of an encoded STRING column |
Set #
Stores a value. Nil empties the cell.
method : public : Set(row:Int, value:Compare) ~ BoolParameters
| Name | Type | Description |
|---|---|---|
| row | Int | row number |
| value | Compare | value |
Return
| Type | Description |
|---|---|
| Bool | true if stored, false if out of range or refused by a declared type (see GetError) |
SetColumnIndex #
Gives the column an index, or takes it away. Used by Table->CreateIndex and DropIndex.
method : public : SetColumnIndex(index:ColumnIndex) ~ NilParameters
| Name | Type | Description |
|---|---|---|
| index | ColumnIndex | index, Nil for none |
Size #
Gets the number of rows, empty ones included
method : public : Size() ~ IntReturn
| Type | Description |
|---|---|
| Int | number of rows |
TypeName # function
Gets the name of a type
function : TypeName(type:ColumnType) ~ StringParameters
| Name | Type | Description |
|---|---|---|
| type | ColumnType | storage type |
Return
| Type | Description |
|---|---|
| String | lowercase type name |