Classical statistics for small analyses: descriptive statistics, the Student t, normal, chi-square and F distributions, t-tests with confidence intervals and effect sizes, Pearson correlation with a p-value, and an ordinary least squares fit with the full summary table (standard errors, t statistics, p values, confidence intervals, F test, R-squared, AIC and BIC). Compile with -lib stats,gen_collect; it needs nothing from System.ML.
No call here ends the program on bad input. A function that returns a Float returns NaN, and a function that returns a result object returns one whose IsValid() is false and whose GetError() says why. Design and verification notes: docs/AI.md, "Statistics".
Design
Builds design matrices for OLS: an intercept column, 0/1 dummy columns for a categorical variable, and column joining.
Operations
AddIntercept # function
Copies X with a leading column of ones.
function : AddIntercept(X:Float[,]) ~ Float[,]Parameters
| Name | Type | Description |
|---|---|---|
| X | Float | matrix, rows = samples |
Return
| Type | Description |
|---|---|
| Float | the new matrix, Nil for Nil input |
Column # function
A one-column matrix from an array.
function : Column(v:Float[]) ~ Float[,]Parameters
| Name | Type | Description |
|---|---|---|
| v | Float | values |
Return
| Type | Description |
|---|---|
| Float | matrix with one column, Nil for Nil input |
Dummies # function
0/1 columns for a categorical variable, one per level after the first (the first sorted level is the reference, so an intercept does not make the matrix singular).
function : Dummies(values:String[]) ~ Float[,]Parameters
| Name | Type | Description |
|---|---|---|
| values | String | the variable, one entry per row |
Return
| Type | Description |
|---|---|
| Float | matrix with rows = values and one column per non-reference level, Nil for Nil input; a variable with one level gives a matrix with no columns, which is Nil |
DummiesFor # function
0/1 columns for the levels after the first of an explicit level list (which names the reference level first).
function : DummiesFor(values:String[], levels:String[]) ~ Float[,]Parameters
| Name | Type | Description |
|---|---|---|
| values | String | the variable, one entry per row |
| levels | String | levels in column order; levels[0] is the reference and gets no column |
Return
| Type | Description |
|---|---|
| Float | matrix with rows = values and levels - 1 columns, Nil for Nil or fewer than 2 levels |