In-memory query engine over typed columns. Tables are filled row by row or loaded from sources (CSV files, folders, the program's own objects) and queried with SQL-like statements; aggregates, Describe() and Finder (file search) are included. Compile with -lib query.
HttpJsonSource
A JSON web service as tables: the array of objects at 'records_path' in each response becomes rows, mapped as JsonSource maps them (nested arrays become child tables). Paging follows a next-page link found in each response, or adds an offset to the URL, until a page is short, the link is missing or repeats, or the row limit is reached. Responses go through the same download cache as files named in SQL, so an unchanged page is not downloaded again.
Example
source := HttpJsonSource->New("https://api.example.org/runs", "runs", "data.items");
source->SetNextLink("links.next");
runs := source->Read();Operations
GetChildTables #
Gets the child tables the last Read made
method : public : GetChildTables() ~ Vector<Table>Return
| Type | Description |
|---|---|
| Vector<Table> | child tables, empty if there are none |
GetError #
Gets the reason the last read failed
method : public : GetError() ~ StringReturn
| Type | Description |
|---|---|
| String | error, Nil if none |
GetPageCount #
Gets how many pages the last Read fetched
method : public : GetPageCount() ~ IntReturn
| Type | Description |
|---|---|
| Int | pages |
GetTables #
Gets the tables the service provides, the main table first; they are known only after reading it
method : public : GetTables() ~ TableSpec[]Return
| Type | Description |
|---|---|
| TableSpec | table shapes |
Load #
Loads the service's tables into a catalog
method : public : Load(catalog:Catalog, hint:ScanHint) ~ BoolParameters
| Name | Type | Description |
|---|---|---|
| catalog | Catalog | catalog |
| hint | ScanHint | columns and rows of the main table needed, Nil for everything |
Return
| Type | Description |
|---|---|
| Bool | true if loaded |
New # constructor
Constructor
New(url:String, name:String, records_path:String)Parameters
| Name | Type | Description |
|---|---|---|
| url | String | first page's URL |
| name | String | table name |
| records_path | String | dotted path to the array of records in a response ('data.items'); an empty path means the response itself is the array |
Read #
Reads every page into the main table; GetChildTables then has the others
method : public : Read() ~ TableReturn
| Type | Description |
|---|---|
| Table | table, Nil on error (see GetError) |
Read #
Reads every page into the main table; GetChildTables then has the others
method : public : Read(hint:ScanHint) ~ TableParameters
| Name | Type | Description |
|---|---|---|
| hint | ScanHint | columns and rows of the main table needed, Nil for everything |
Return
| Type | Description |
|---|---|
| Table | table, Nil on error (see GetError) |
SetLimit #
Stops after this many rows
method : public : SetLimit(rows:Int) ~ NilParameters
| Name | Type | Description |
|---|---|---|
| rows | Int | row limit, -1 for none |
SetNextLink #
Follows a next-page URL found at a dotted path in each response; a missing or null value ends paging
method : public : SetNextLink(path:String) ~ NilParameters
| Name | Type | Description |
|---|---|---|
| path | String | dotted path to the next page's URL |
SetOffsetPaging #
Pages by adding an offset parameter to the URL (offset 0 first); a page with fewer than 'page_size' records is the last
method : public : SetOffsetPaging(param:String, page_size:Int) ~ NilParameters
| Name | Type | Description |
|---|---|---|
| param | String | query parameter name, such as 'offset' |
| page_size | Int | records per page |