Data — API reference

10/27 types documented. Of 187 public methods, 151 carry their own description, 0 are covered by their type's, and 36 have neither. An entry with no prose below its signature is undocumented in the source, not undocumented here.

CLASS BoolColumn

IMPLEMENTS IColumn

Data/src/BoolColumn.ev:19

Constructors

INIT(String columnName, MOVE mda::Column[uint8] backing, MOVE mda::TextColumn src)

Adopt a 0/1 backing. src is the original CSV text for byte-exact round-trip, or EMPTY for a derived column.

Methods

METHOD length() RETURNS int64

METHOD name() RETURNS String

METHOD isNull(int64 i) RETURNS boolean

METHOD nullCount() RETURNS int64

METHOD isNullOrBlank(int64 i) RETURNS boolean

A boolean is never blank: null is the only absence.

METHOD withName(String newName) RETURNS IColumn

METHOD filterRows(boolean[] keep) RETURNS IColumn

METHOD gatherRows(int64[] rows) RETURNS IColumn

A fresh owned copy whose cell j is source cell rows[j]; a -1 row is a null cell. Repeats and reorders are allowed — the gather seam every join, sort and group-first rides.

METHOD get(int64 i) RETURNS boolean

The cell as a boolean. Null cells have no value — check isNull first.

METHOD hasSourceText() RETURNS boolean

METHOD sourceTextAt(int64 i) RETURNS String

The cell's original CSV text (True, FALSE, …). Contract: hasSourceText().

METHOD trueCount() RETURNS int64

How many present cells are TRUE.

METHOD falseCount() RETURNS int64

How many present cells are FALSE.

METHOD fillNull(boolean v) RETURNS BoolColumn

A copy with every null replaced by v. Derived: no source text, so the filled cells (and the rest) write canonically.

METHOD appendColumn(REFERENCE REFERENCE IColumn otherCol) RETURNS BoolColumn

This column's rows followed by other's — the DataFrame.concat seam.

METHOD compareRows(int64 a, int64 b) RETURNS int32

Order cell a against cell b, FALSE before TRUE. Contract: both present.

CLASS ColumnExpr

Data/src/ColumnExpr.ev:24

Constructors

INIT(MOVE TextColumn[] cols)

Adopt the bound/transformed column set (built by DataFrame.col/cols and the transform methods). Each column is already materialised.

Methods

METHOD resultCount() RETURNS int64

How many columns this expression carries.

METHOD resultName(int64 j) RETURNS String

The name of result column j.

METHOD resultAt(int64 j) RETURNS IColumn

A fresh owned copy of result column j, as the erased IColumn face.

METHOD lower() RETURNS ColumnExpr

ASCII-lowercase every bound column.

METHOD upper() RETURNS ColumnExpr

ASCII-uppercase every bound column.

METHOD trim() RETURNS ColumnExpr

Trim ASCII whitespace from both ends of every cell of every bound column.

METHOD title() RETURNS ColumnExpr

Title-case every cell of every bound column (word-aware, kernel semantics).

METHOD replace(Regex::Regex pattern, String repl) RETURNS ColumnExpr

Every match of pattern replaced by repl in every bound column (global, per cell). The Step-1 Bob (Bobby) -> Bob clean.

METHOD map(LAMBDA cell) RETURNS ColumnExpr

The per-cell escape hatch: run cell (a DynamicString -> DynamicString lambda) over every present cell of every bound column. The deliberate slow path — materialises a DynamicString per cell — for logic with no vectorised form (e.g. a block-form lambda calling the MODIFY string transforms).

METHOD as(String newName) RETURNS ColumnExpr

Alias the result column. Single-column expressions only — aliasing a multi-column expression to one name is programmer misuse (a DataError).

CLASS CsvSink

IMPLEMENTS DataSink

Data/src/CsvSink.ev:18

Constructors

INIT(String filePath)

Write to path with the default CSV dialect.

INIT(String filePath, CsvOptions options)

Write to path with an explicit dialect (must match a reader's for a faithful round-trip).

Methods

METHOD write(DataFrame frame) RETURNS STATUS

Serialise frame to the file. Returns FAILURE if the file cannot be opened or written.

CLASS CsvSource

IMPLEMENTS DataSource

Data/src/CsvSource.ev:17

Fields

Constructors

INIT(String filePath)

Read path with the default CSV dialect (comma / double-quote / header row).

INIT(String filePath, CsvOptions options)

Read path with an explicit dialect.

Methods

MODIFY METHOD column(String name, ColumnType type) RETURNS void

Force column name to a dtype, overriding inference. Call once per column before read(); AUTO_SELECT (the default for any undeclared column) leaves the column to inference. A NUMBER-forced column still picks int64 vs float64 backing by inspecting its cells.

MODIFY METHOD column(String name, ColumnType type, String pattern) RETURNS void

As column(name, type), with a DateTimeFactory pattern (YYYY, MM, DD, HH, mm, ss, SSS; any other character literal) for a DATE / TIME / TIMESTAMP column whose cells are not ISO-8601. A pattern has no zone token, so a patterned TIMESTAMP is always TIMESTAMP_NTZ.

MODIFY METHOD requireColumns(ColumnSpec[] schema) RETURNS void

Declare the STRICT schema (§7.4): the file's columns must be EXACTLY this set of names — a missing one or an unexpected extra one fails the read. Each spec's type also declares that column's dtype (as column(...) would; AUTO_SELECT leaves it to inference). Call before read().

METHOD read() RETURNS | DataFrame

Open, read, tokenize, and build the frame. Missing/unreadable file → FAILURE; on success the frame exists (never a zombie).

CLASS DataError

EXTENDS Error

Data/src/DataError.ev:9

Constructors

INIT(String message)

CLASS DataFrame

Data/src/DataFrame.ev:22

Constructors

INIT(MOVE IColumn[] cols)

Adopt an ordered column set (the builder / reader path). The frame owns the columns from here; it is immutable, so they are never mutated in place.

Methods

METHOD rowCount() RETURNS int64

The number of rows — every column's length() (a rectangular frame). An empty frame (no columns) has 0 rows.

METHOD columnCount() RETURNS int64

The number of columns.

METHOD columnNames() RETURNS String[]

The column names, in column order.

METHOD hasColumn(String name) RETURNS boolean

Whether a column named name exists — the guard a caller uses before a typed accessor to avoid the panic.

METHOD column(String name) RETURNS REFERENCE IColumn

The column named name, as the erased IColumn face.

METHOD textColumn(String name) RETURNS REFERENCE TextColumn

The column named name, as a TextColumn. PANICs if missing, or if the column exists but is not text.

METHOD numberColumn(String name) RETURNS REFERENCE NumberColumn

The column named name, as a NumberColumn. PANICs if missing, or if the column exists but is not numeric.

METHOD boolColumn(String name) RETURNS REFERENCE BoolColumn

The column named name, as a BoolColumn. PANICs if missing, or if the column exists but is not boolean.

METHOD decimalColumn(String name) RETURNS REFERENCE DecimalColumn

The column named name, as a DecimalColumn. PANICs if missing, or if the column exists but is not decimal.

METHOD temporalColumn(String name) RETURNS REFERENCE TemporalColumn

The column named name, as a TemporalColumn. PANICs if missing, or if the column exists but is not temporal.

METHOD normalizeHeaders() RETURNS DataFrame

A new frame with every column name trimmed, lowercased, and snake_cased. Column data is unchanged (each column carried across verbatim).

METHOD dropDuplicateColumns() RETURNS DataFrame

A new frame keeping only the FIRST column of each repeated name.

METHOD dropEmptyColumns() RETURNS DataFrame

A new frame dropping every column whose cells are ALL null or blank.

METHOD requireValues(String[] cols) RETURNS DataFrame

A new frame dropping every row where ANY column in cols is null or blank (whitespace-aware). A name in cols that is not a column is programmer misuse — it PANICs a DataError, like the typed accessors.

METHOD iterator() RETURNS RowIterator

A by-reference iterator over the frame's rows — powers FOR row IN frame, each row a Row view synthesized at its index and yielded by reference (an ObjectIterator OF Row, Bug #331). Read-only.

METHOD col(String name) RETURNS ColumnExpr

Bind one text column into a ColumnExpr for transformation. The named column must exist and be text (else a DataError).

METHOD cols(String[] names) RETURNS ColumnExpr

Bind several text columns into one ColumnExpr; a transform then applies to each. Every name must exist and be text.

METHOD with(ColumnExpr[] exprs) RETURNS DataFrame

Evaluate exprs against the frame and ADD/REPLACE the named result columns (replace preserves column position; a new name is appended), returning a new frame. Result names are assumed unique.

METHOD select(ColumnExpr[] exprs) RETURNS DataFrame

A new frame of ONLY the expression results, in expression order.

METHOD withColumn(String name, LAMBDA fn) RETURNS DataFrame

A new frame with a column named name computed row-by-row by fn (a Row -> DynamicString lambda — the row-oriented escape hatch). Adds the column, or replaces an existing same-named one in place. The result is fully present (the lambda returns a value for every row).

METHOD select(String[] names) RETURNS DataFrame

A new frame with exactly the columns in names, in that order (a structural pick). An unknown name is programmer misuse — a DataError.

METHOD where(String col, Compare op, number v) RETURNS DataFrame

Rows where col op v holds, for a NUMBER / DECIMAL / BOOL / temporal column. v is compared in the column's own domain: an INTEGER column against an integral v exactly, a FLOAT column as float64, DECIMAL as decimal128 (exact), BOOL as 0/1, a temporal column against its raw unit (epoch days / millis of day / epoch millis — see TemporalColumn.valueAt). A null cell never matches. A text column is a DataError — use whereText.

METHOD whereText(String col, Compare op, String v) RETURNS DataFrame

Rows where col op v holds with v given as text. A TEXT column compares code point by code point; a temporal column parses v once as ISO-8601 in its own kind (DATE → YYYY-MM-DD; timestamps → the column's zone class); BOOL accepts true/false in any case; NUMBER and DECIMAL parse v as a value. A v that does not parse is a DataError. A null cell never matches.

METHOD filter(LAMBDA fn) RETURNS DataFrame

Rows for which fn(row) is TRUE — the row-oriented escape hatch (a Row -> boolean lambda), mirroring withColumn.

METHOD concat(REFERENCE REFERENCE DataFrame other) RETURNS DataFrame

This frame's rows followed by other's. The frames must have the same column names in the same order and each pair the same kind (a temporal pair the same TemporalKind); INTEGER + FLOAT numbers promote to FLOAT. Anything else is a DataError.

METHOD sortBy(String[] keys, boolean ascending) RETURNS DataFrame

A new frame with the rows ordered by keys, earlier keys outranking later ones. The sort is STABLE: rows that tie on every key keep their source order, which is what makes a sorted result reproducible.

NULLS SORT LAST IN BOTH DIRECTIONS. ascending reverses the order of the values, not the meaning of absence — the same rule mda's sortedIndices follows, so a Data sort and an mda sort agree.

A bottom-up merge sort over an index array, then one gatherRows per column — so every column keeps its kind and its source text.

METHOD groupBy(String[] keys) RETURNS GroupBy

Group the rows by keys, numbering groups in FIRST-SEEN order. The result is a GroupBy view — nothing is copied until an aggregate is asked for.

A single INTEGER NumberColumn key takes an int-keyed hash table; every other shape builds one composite byte key per row and hashes that. Either way a NULL key cell is a group of its own (the "no key" group), not a row dropped on the floor — a row that exists must land somewhere.

Keys compare by VALUE, never by source text: a column has one uniform backing, so two cells that hold the same value render the same key.

METHOD join(REFERENCE REFERENCE DataFrame right, String[] on, JoinKind kind) RETURNS DataFrame

Join right onto this frame by equality on the on columns, which must name a column in BOTH frames. The result carries every column of this frame, then right's non-key columns; a right column whose name is already taken gains a _right suffix. Key columns come from the left.

ONE-TO-MANY IS REAL: a left row matching k right rows yields k rows, the matches in ascending right-row order. INNER drops a left row with no match; LEFT keeps it and fills the right columns with NULL — not with empty text, so "no match" and "matched an empty cell" stay distinct.

A NULL key cell never matches, not even another NULL (SQL's rule, and the one where already follows). Under LEFT such a row survives unmatched.

Right columns keep their kind: the match list is gathered, not stringified.

METHOD leftJoin(REFERENCE REFERENCE DataFrame right, String[] onKeys) RETURNS DataFrame

join(right, onKeys, JoinKind.LEFT).

CHANGED IN STEP-2: this used to render every right column as text, write "" where nothing matched, and keep only the last row of a duplicated right key. It now preserves kinds, writes NULL for no match, and fans out on duplicates. It had no callers and no tests when that changed.

METHOD coalesce(String targetCol, String fallbackCol) RETURNS DataFrame

A new frame where targetCol cells that are empty are filled from fallbackCol, and fallbackCol is dropped. Both must exist.

METHOD groupByUnique(String keyCol, String valueCol) RETURNS DataFrame

Group by keyCol; emit a two-column frame [keyCol, valueCol] holding the keys that map to exactly ONE distinct valueCol value (first-seen order).

METHOD slice(int64 start, int64 count) RETURNS DataFrame

A new frame with the rows in [start, start+count) — row-range selection. Reuses each column's filterRows seam, so it preserves per-column dtype.

METHOD head(int64 n) RETURNS DataFrame

The first n rows (all, if the frame has fewer).

METHOD tail(int64 n) RETURNS DataFrame

The last n rows (all, if the frame has fewer).

INTERFACE DataSink

Data/src/DataSink.ev:9

Methods

METHOD write(DataFrame f) RETURNS STATUS

INTERFACE DataSource

Data/src/DataSource.ev:12

Methods

METHOD read() RETURNS | DataFrame

CLASS DecimalColumn

IMPLEMENTS IColumn

Data/src/DecimalColumn.ev:24

Constructors

INIT(String columnName, MOVE mda::Column[decimal128] backing, MOVE mda::TextColumn src, int32 scale)

Adopt a backing. src is the original CSV text for byte-exact round-trip, or EMPTY for a derived column; scale is the fractional-digit count used for canonical writing.

Methods

METHOD length() RETURNS int64

METHOD name() RETURNS String

METHOD isNull(int64 i) RETURNS boolean

METHOD nullCount() RETURNS int64

METHOD isNullOrBlank(int64 i) RETURNS boolean

A decimal is never blank: null is the only absence.

METHOD withName(String newName) RETURNS IColumn

METHOD filterRows(boolean[] keep) RETURNS IColumn

METHOD gatherRows(int64[] rows) RETURNS IColumn

A fresh owned copy whose cell j is source cell rows[j]; a -1 row is a null cell. Repeats and reorders are allowed — the gather seam every join, sort and group-first rides.

METHOD get(int64 i) RETURNS decimal128

The cell as an exact decimal. Null cells have no value — check isNull first.

METHOD scale() RETURNS int32

The fractional-digit count canonical writing pads to.

METHOD hasSourceText() RETURNS boolean

METHOD sourceTextAt(int64 i) RETURNS String

The cell's original CSV text. Contract: hasSourceText().

METHOD canonicalTextAt(int64 i) RETURNS String

The cell written canonically at this column's scale: decimal128 INTO String, padded with trailing zeros (and a point) up to scale() fractional digits. A cell already wider than the scale is written as it is — never rounded here. Exponent-form text (very large or tiny magnitudes) is left as the kernel prints it.

METHOD fillNull(decimal128 v) RETURNS DecimalColumn

A copy with every null replaced by v. Derived: no source text, so every cell writes canonically at the same scale.

METHOD appendColumn(REFERENCE REFERENCE IColumn otherCol) RETURNS DecimalColumn

This column's rows followed by other's — the DataFrame.concat seam. The scale is the wider of the two.

METHOD compareRows(int64 a, int64 b) RETURNS int32

Order cell a against cell b exactly, through decimal128's own comparison — so 2.00 and 2 compare equal. Contract: both present.

CLASS FrameBuilder

Data/src/FrameBuilder.ev:15

Constructors

INIT()

Methods

MODIFY METHOD addColumn(MOVE IColumn col) RETURNS void

Append a column. The builder takes ownership (MOVE) — the caller's handle is spent, matching the column-major, build-once shape.

METHOD columnCount() RETURNS int64

How many columns have been added so far.

MODIFY METHOD build() RETURNS DataFrame

Freeze the collected columns into an immutable DataFrame. The builder's column set is MOVED into the frame (MODIFY: .cols is consumed), so the builder is spent after this call.

CLASS GroupBy

Data/src/GroupBy.ev:27

Constructors

INIT(IColumn[] columns, String[] keys, int64[] groupIds, int64[] firsts, int64 groupTotal)

Adopt a computed grouping. DataFrame.groupBy is the only caller.

Methods

METHOD groupCount() RETURNS int64

How many distinct key combinations the frame held.

METHOD count() RETURNS DataFrame

Key columns plus count — the number of ROWS in each group (a null cell is still a row). For present-cell counts, use Agg.COUNT on a column.

METHOD sum(String col) RETURNS DataFrame

Key columns plus the per-group sum of col, named after it.

METHOD mean(String col) RETURNS DataFrame

Key columns plus the per-group mean of col (always FLOAT), named after it.

METHOD minimum(String col) RETURNS DataFrame

Key columns plus the per-group minimum of col, named after it.

METHOD maximum(String col) RETURNS DataFrame

Key columns plus the per-group maximum of col, named after it.

METHOD first() RETURNS DataFrame

EVERY column of the source frame, one row per group: the first row of each. This is the correct row-key dedup (DESIGN.md §9) — every kind survives.

METHOD agg(AggSpec[] specs) RETURNS DataFrame

Key columns plus one column per spec, in spec order.

CLASS Iso8601

Data/src/Iso8601.ev:22

Methods

METHOD zoneClassOf(REFERENCE REFERENCE String text) RETURNS int64

Zone class of a timestamp cell: 0 = not a timestamp shape, 2 = no zone (NTZ), 3 = Z (LTZ), 4 = ±HH:MM offset (TZ). The codes match TemporalKind's order so callers can MATCH on the kind they build.

METHOD isDateShape(REFERENCE REFERENCE String text) RETURNS boolean

Whether the cell is exactly YYYY-MM-DD.

METHOD parseDate(REFERENCE REFERENCE String text) RETURNS | int64

YYYY-MM-DD → days since 1970-01-01 (FAILURE on a bad calendar date).

METHOD parseWallMs(REFERENCE REFERENCE String text) RETURNS | int64

The wall clock of a timestamp cell (any zone class), read as if UTC, in epoch millis. The zone suffix is ignored here — see offsetMinutesOf.

METHOD offsetMinutesOf(REFERENCE REFERENCE String text) RETURNS int32

The ±HH:MM suffix of a zone-class-4 cell in minutes (0 for any other).

METHOD formatDate(int64 days) RETURNS String

Days since epoch → YYYY-MM-DD.

METHOD formatTime(int64 millisOfDay) RETURNS String

Millis of day → HH:MM:SS, with .fff only when the millis are non-zero.

METHOD formatWall(int64 ms) RETURNS String

A wall clock (epoch millis read as UTC) → YYYY-MM-DDTHH:MM:SS[.fff].

METHOD formatOffset(int32 minutes) RETURNS String

Offset minutes → ±HH:MM.

METHOD floorDiv(int64 a, int64 b) RETURNS int64

Division rounding toward negative infinity (a pre-1970 instant divides into a negative day count without drifting by one).

METHOD floorMod(int64 a, int64 b) RETURNS int64

The non-negative remainder that pairs with floorDiv.

CLASS NumberColumn

IMPLEMENTS IColumn

Data/src/NumberColumn.ev:29

Constructors

INIT(String columnName, MOVE mda::Column[int64] backing, MOVE mda::TextColumn src)

Adopt an integer backing. src is the original CSV text for byte-exact round-trip, or EMPTY for a derived column.

INIT(String columnName, MOVE mda::Column[float64] backing, MOVE mda::TextColumn src)

Adopt a float backing (same source-text contract).

Methods

METHOD length() RETURNS int64

METHOD name() RETURNS String

METHOD isNull(int64 i) RETURNS boolean

METHOD nullCount() RETURNS int64

METHOD kind() RETURNS NumberKind

Which backing this column holds — the kernel's own NumberKind, read off which backing is present.

METHOD get(int64 i) RETURNS number

Cell i as a number — the backing widens in implicitly (int64/float64 INTO number is total). The caller checks isNull first; a null cell reads as the backing's zero.

METHOD int64At(int64 i) RETURNS int64

Whether this column carries its original CSV text (and so round-trips byte-identically). FALSE for any derived column. The raw int64 of an INTEGER column's cell — no number is built. Contract: kind() == NumberKind.INTEGER and the cell is present.

METHOD float64At(int64 i) RETURNS float64

The cell as float64 — an INTEGER cell widened, a FLOAT cell as stored. No number is built; the hot-loop accessor. Contract: the cell is present.

METHOD hasSourceText() RETURNS boolean

METHOD sourceTextAt(int64 i) RETURNS String

Cell i's original CSV text — only meaningful when hasSourceText(). This is what CsvSink writes for an untransformed column, so a 42 in a float column comes back 42 rather than 42.0.

METHOD isNullOrBlank(int64 i) RETURNS boolean

Null, or (for a number) nothing more — a numeric cell is never blank.

METHOD withName(String newName) RETURNS IColumn

A fresh owned copy under newName — the name/set-transform seam.

METHOD filterRows(boolean[] keep) RETURNS IColumn

METHOD gatherRows(int64[] rows) RETURNS IColumn

A fresh owned copy whose cell j is source cell rows[j]; a -1 row is a null cell. Repeats and reorders are allowed — the gather seam every join, sort and group-first rides.

METHOD count() RETURNS int64

Count of non-null cells.

METHOD sum() RETURNS float64

Sum of the non-null cells.

METHOD mean() RETURNS float64

Arithmetic mean of the non-null cells; 0.0 if all null.

METHOD minimum() RETURNS float64

Minimum non-null cell; 0.0 if all null.

METHOD maximum() RETURNS float64

Maximum non-null cell; 0.0 if all null.

METHOD std() RETURNS float64

Sample standard deviation of the non-null cells, Bessel-corrected (ddof = 1); 0.0 when fewer than two cells are present.

HERE RATHER THAN DELEGATED to stats::Descriptive, and that is a semantic choice before it is a dependency one. Descriptive.sampleVariance takes a DENSE AdvancedMath::Vector in which every slot is a value. A column carries a validity mask, and count() here means PRESENT cells rather than length — which is exactly why mean() divides by it. Calling out would mean materialising a dense vector from a masked column, a copy, to compute what one pass over the buffer we already own yields directly.

The dependency it also avoids is the second reason, not the first: Data → stats → AdvancedMath → Accelerate would link a BLAS framework into every consumer of Data in exchange for one scalar. Note the distinction matters because it survives a change in circumstances — a lighter stats would reopen the dependency argument and would not touch the mask argument.

ddof = 1 is not a new decision. It is Descriptive.sampleVariance's own default (its header pins ddof=1 as the unbiased sample variance) and it is pandas' default too, so a Data standard deviation agrees with both the module next door and the library this is benchmarked against. numpy defaults to 0; that disagreement is numpy's.

TWO PASSES, not Welford. The mean is already a method, the data is in cache after the first pass at these sizes, and a textbook two-pass is the one a reader can check against the formula. Welford earns its complexity on a streaming input, which a materialised column is not.

The kernel's only square root is Math.squareRoot, which RETURNS COMPLEX — it is the Math↔complex integration, so a negative input yields an imaginary result rather than an error. A sum of squares divided by a positive count cannot be negative, so the real component is the whole answer here. AdvancedMath.LinearAlgebra.squareRoot returns float64 but reaches it through FOREIGN::sqrt, which needs allows_foreign — a heavier change than this one function warrants.

METHOD appendColumn(REFERENCE REFERENCE IColumn otherCol) RETURNS NumberColumn

This column's rows followed by other's — the DataFrame.concat seam. INTEGER + INTEGER stays INTEGER; any FLOAT side promotes the whole result to FLOAT ("a number is a number" — the backing is chosen once, here).

METHOD compareRows(int64 a, int64 b) RETURNS int32

Order cell a against cell b by VALUE: -1 before, 0 equal, 1 after. Contract: both cells are present. A column has one uniform backing, so this compares int64 to int64 or float64 to float64, never across.

CLASS Row

Data/src/Row.ev:19

Constructors

INIT(IColumn[] columns, int64 i)

Pin row i of the column set columns (a frame's .columns). The plain IColumn[] param lowers to a const reference (the ArrayIterator convention); .cols is a non-owning bind, never an owning copy.

Methods

METHOD text(String colName) RETURNS String

This row's cell in column colName, as a String (any dtype). A numeric cell reads as its source text when present (byte-exact), else its canonical form; a null cell reads as "". A missing column PANICs.

METHOD bool(String colName) RETURNS boolean

The cell of a BoolColumn — FALSE when null (check isBlank first when the distinction matters). PANICs if the column is missing or not boolean.

METHOD decimal(String colName) RETURNS decimal128

The cell of a DecimalColumn — 0 when null. PANICs if missing / not decimal.

METHOD dateTime(String colName) RETURNS DateTime

The cell of a TemporalColumn as a UTC-instant DateTime (DATE → midnight UTC; not for TIME) — the epoch when null. PANICs if missing / not temporal.

METHOD timeOfDay(String colName) RETURNS TimeDuration

The cell of a TIME TemporalColumn — zero when null. PANICs if missing / not temporal (and, by the column's own contract, if the kind is not TIME).

METHOD isBlank(String colName) RETURNS boolean

Whether this row's cell in colName is null or whitespace-only. A missing column PANICs.

METHOD emptyCount() RETURNS int64

The number of this row's cells that are null or blank across all columns.

CLASS RowIterator

IMPLEMENTS ObjectIterator

Data/src/RowIterator.ev:17

Constructors

INIT(IColumn[] columns, int64 n)

Methods

METHOD hasNext() RETURNS boolean

MODIFY METHOD next() RETURNS | REFERENCE Row

METHOD peek() RETURNS | REFERENCE Row

CLASS TemporalColumn

IMPLEMENTS IColumn

Data/src/TemporalColumn.ev:31

Constructors

INIT(String columnName, TemporalKind kind, MOVE mda::Column[int64] backing, MOVE mda::Column[int32] offsetMinutes, MOVE mda::TextColumn src)

Adopt a backing in the kind's unit. offsetMinutes is the _TZ offset column (EMPTY for every other kind); src the original CSV text or EMPTY.

Methods

METHOD length() RETURNS int64

METHOD name() RETURNS String

METHOD isNull(int64 i) RETURNS boolean

METHOD nullCount() RETURNS int64

METHOD isNullOrBlank(int64 i) RETURNS boolean

A temporal is never blank: null is the only absence.

METHOD withName(String newName) RETURNS IColumn

METHOD filterRows(boolean[] keep) RETURNS IColumn

METHOD gatherRows(int64[] rows) RETURNS IColumn

A fresh owned copy whose cell j is source cell rows[j]; a -1 row is a null cell. Repeats and reorders are allowed — the gather seam every join, sort and group-first rides.

METHOD kind() RETURNS TemporalKind

METHOD getEpochMs(int64 i) RETURNS int64

The raw value in the kind's unit: DATE → epoch millis of midnight UTC, TIME → millis of day, timestamps → epoch millis (for _TZ, the instant).

METHOD getDateTime(int64 i) RETURNS DateTime

A UTC-instant DateTime. DATE → midnight UTC of that day; NTZ → the wall clock's fields; LTZ / TZ → the instant. Not for TIME (contract).

METHOD getTimeOfDay(int64 i) RETURNS TimeDuration

The time of day of a TIME column (contract: kind is TIME).

METHOD getOffsetMinutes(int64 i) RETURNS int32

The _TZ offset in minutes east of UTC; 0 for every other kind.

METHOD hasSourceText() RETURNS boolean

METHOD sourceTextAt(int64 i) RETURNS String

The cell's original CSV text. Contract: hasSourceText().

METHOD canonicalTextAt(int64 i) RETURNS String

The cell as ISO-8601: YYYY-MM-DD, HH:MM:SS[.fff], or a timestamp with no suffix (NTZ), Z (LTZ) or the wall clock and its ±HH:MM (TZ).

METHOD valueAt(int64 i) RETURNS int64

The raw int64 in the kind's unit — epoch days (DATE), millis of day (TIME), epoch millis (timestamps; the instant for _TZ). What where compares.

METHOD appendColumn(REFERENCE REFERENCE IColumn otherCol) RETURNS TemporalColumn

This column's rows followed by other's — the DataFrame.concat seam. Contract: other->kind() == kind() (the caller checks and raises DataError).

METHOD compareRows(int64 a, int64 b) RETURNS int32

Order cell a against cell b by the raw int64 in this kind's unit — chronological for every kind, since a column holds one kind. For _TZ that orders by the true instant, not by the local wall clock. Contract: both cells are present.

CLASS TextColumn

IMPLEMENTS IColumn

Data/src/TextColumn.ev:19

Constructors

INIT(String columnName, MOVE mda::TextColumn s)

Adopt an mda text column under a name. The sole constructor: build the store first (mda::TextColumn s := CREATE mda::TextColumn(values)), then name it. Both the CSV-reader path (raw Arrow buffers) and the builder path (a String[]) construct their store through mda, so Data never has a second way to make a column.

Methods

METHOD length() RETURNS int64

METHOD name() RETURNS String

METHOD isNull(int64 i) RETURNS boolean

Null = the mask bit is clear. Distinct from blank (present whitespace).

METHOD nullCount() RETURNS int64

METHOD get(int64 i) RETURNS String

Materialise cell i (the per-cell path). A null cell reads as "".

METHOD blankCount() RETURNS int64

The number of cells that are null OR whitespace-only — the axis requireValues drops rows on. Vectorised: one pass over the byte seam, no per-cell String. An empty slice counts as blank.

METHOD countWhere(LAMBDA pred) RETURNS int64

The number of cells satisfying pred — the per-cell escape hatch (cell: DynamicString -> boolean). Materialises one DynamicString per cell, so prefer a vectorised method where one exists.

METHOD unique() RETURNS TextColumn

The distinct present values, first-seen order (delegates to the store).

Data and mda each declare a TextColumn (deliberately: ours is the named public column, theirs is the Arrow storage it wraps). An unqualified CREATE TextColumn(...) resolves to OURS — a module's own class wins the bare name; mda::TextColumn reaches the storage (Bug #301, fixed).

METHOD sortedIndices(boolean ascending) RETURNS int64[]

A STABLE permutation reading the column in byte-wise order, nulls last.

METHOD isNullOrBlank(int64 i) RETURNS boolean

Null, or present-but-whitespace-only. An empty present slice reads blank.

METHOD withName(String newName) RETURNS IColumn

A fresh owned copy under newName — the name/set-transform seam.

METHOD filterRows(boolean[] keep) RETURNS IColumn

A fresh owned copy keeping only keep-flagged rows — the row-drop seam.

METHOD gatherRows(int64[] rows) RETURNS IColumn

A fresh owned copy whose cell j is source cell rows[j]; a -1 row is a null cell. Repeats and reorders are allowed — the gather seam every join, sort and group-first rides.

METHOD renamed(String newName) RETURNS TextColumn

A copy under a new name (data + validity verbatim).

METHOD lowercase() RETURNS TextColumn

ASCII-lowercased (single pass over the byte seam; lengths unchanged).

METHOD uppercase() RETURNS TextColumn

ASCII-uppercased (single pass over the byte seam; lengths unchanged).

METHOD trimmed() RETURNS TextColumn

ASCII-whitespace trimmed both ends, per cell (a null stays null).

METHOD titled() RETURNS TextColumn

Title-cased per cell via the kernel's word-aware toTitleCase (boundaries at space / - / '); a null stays null.

METHOD replaced(Regex::Regex pattern, String repl) RETURNS TextColumn

Every match of pattern replaced by repl, per cell (regex needs cell boundaries; the compiled pattern is reused across cells). A null stays null. Global replace, per Regex.replace's contract.

METHOD rebuiltFrom(String[] values, boolean[] present) RETURNS TextColumn

A new column (this column's name) from parallel values + present arrays: cell i is values[i] when present[i], else null. The materialisation the per-cell ColumnExpr.map builds its result from (the lambda must be invoked in the method that declares the LAMBDA parameter for its signature to infer — so map collects, this rebuilds).

METHOD appendColumn(REFERENCE REFERENCE IColumn otherCol) RETURNS TextColumn

This column's rows followed by other's — the DataFrame.concat seam. Same name as this column; nulls carried from both sides.

METHOD compareRows(int64 a, int64 b) RETURNS int32

Order cell a against cell b: -1 before, 0 equal, 1 after. Contract: both cells are PRESENT — the null policy lives in DataFrame.sortBy, so each kind only has to order real values.

Compared byte-wise over UTF-8, which is code-point order, and which avoids building two Strings per comparison in a sort's inner loop.

ENUM Agg

Agg — a per-group reduction (GroupBy.agg). COUNT counts PRESENT cells of the named column (GroupBy.count() counts rows instead); SUM skips nulls and is 0 over none; MEAN, MIN and MAX skip nulls and are NULL over none; FIRST takes the group's first row and keeps the column's kind.

SUM/MEAN/MIN/MAX need a NumberColumn — exact decimal and temporal reductions each want their own accumulator, so they wait for a consumer.

Data/src/enums.ev:94

Case Description
? —
? —
? —
? —
? —
? —

ENUM ColumnBacking

ColumnBacking — what type inference concluded a CSV column actually holds (DESIGN.md §4.3). Distinct from ColumnType, which is what the developer asked for: ColumnType is the request, ColumnBacking is the verdict.

TEXT — no usable numeric reading: a non-numeric cell, an all-empty column, or an identifier whose leading zeros must survive (007) INTEGER — every present cell parses as int64 FLOAT — every present cell parses, and at least one is fractional BOOL — every present cell is true or false (case-insensitive); a column mixing those with anything else is TEXT, and 0/1 is INTEGER — booleans are inferred only from the words DECIMAL — declared only (ColumnType.DECIMAL); inference never produces it TEMPORAL — every present cell is an ISO-8601 date, or every present cell is an ISO-8601 timestamp of ONE zone class (none / Z / ±HH:MM); mixing dates with timestamps, or zone classes, is TEXT. TIME is never inferred (declare it)

A declared NUMBER column still consults inference, because int64-vs-float64 backing is chosen for the developer rather than spelled at the call site.

Data/src/enums.ev:63

Case Description
? —
? —
? —
? —
? —
? —

ENUM ColumnType

ColumnType — how a CSV column should be typed on read (DESIGN.md §7.4).

This is the typed alternative to pandas' stringly-typed dtype={"zip": str} dict, whose values sprawl across str / "str" / np.int64 / "int64" / "Int64" / "string" / float. Here there are three, and the dev picks the intuitive category — the int64-vs-float64 backing of a NUMBER column is chosen for them by inference (§4.3), not spelled at the call site.

AUTO_SELECT — infer the type (the default for any column not declared) TEXT — force text, even if every cell parses as a number NUMBER — force numeric; a cell that won't parse is a located FAILURE BOOL — force boolean (true/false, case-insensitive); a present cell that is neither is a located FAILURE DECIMAL — force exact base-10 (decimal128); a present cell that is not exact decimal text (or exceeds 34 digits) is a located FAILURE. Never inferred — a numeric column stays NUMBER unless declared DATE / TIME / TIMESTAMP — force a TemporalColumn of that kind. Without a pattern the cell must be ISO-8601 (YYYY-MM-DD, or a timestamp whose zone suffix picks NTZ / LTZ / TZ); with a column(name, type, pattern) pattern the cell is read by DateTimeFactory->parseDateTime (TIMESTAMP → NTZ). A present cell that does not parse is a located FAILURE

(AUTO_SELECT, not AUTO — AUTO is a reserved keyword.)

Data/src/enums.ev:32

Case Description
? —
? —
? —
? —
? —
? —
? —
? —

ENUM Compare

Compare — the operator of a vectorised row filter (DataFrame.where / whereText). A null cell never satisfies any of them, NE included.

Data/src/enums.ev:106

Case Description
? —
? —
? —
? —
? —
? —

ENUM JoinKind

JoinKind — which left rows survive a DataFrame.join.

INNER — only left rows with at least one match LEFT — every left row; an unmatched one carries NULLs on the right side

RIGHT is LEFT with the frames swapped, and OUTER waits for a consumer. A key cell that is NULL never matches — not even another NULL — which is SQL's rule and the same rule where/whereText follow.

Data/src/enums.ev:81

Case Description
? —
? —

ENUM TemporalKind

TemporalKind — what a TemporalColumn's int64 means (Step-1.5 decision S2).

DATE epoch days TIME millis of day TIMESTAMP_NTZ a zone-less wall clock, stored as if UTC TIMESTAMP_LTZ a true instant (epoch millis) TIMESTAMP_TZ a true instant plus a per-cell offset (minutes east of UTC)

Data/src/enums.ev:123

Case Description
? —
? —
? —
? —
? —

INTERFACE IColumn

IColumn — the type-erased face of a column. Each concrete column (TextColumn, NumberColumn, BoolColumn) is homogeneous by element type, but a DataFrame holds an ordered set of columns of differing types that reads as heterogeneous; it holds them as an Array[IColumn]. This is the name-shape-and-null contract every column kind shares, independent of the cell type.

Distinct from mda::IColumn (the substrate's own face): this one adds name(), because a column's name is a Data concern, not a storage concern. Widths are int64 throughout — the kernel's container width, and Arrow's LargeUtf8 offsets layout at the substrate (DESIGN.md §3).

Data/src/interfaces.ev:19

Methods

METHOD length() RETURNS int64

The number of cells (rows) in the column.

METHOD name() RETURNS String

The column's name.

METHOD isNull(int64 i) RETURNS boolean

Whether the cell at i is null (reads the validity mask, never the cell).

METHOD nullCount() RETURNS int64

The number of null cells.

METHOD isNullOrBlank(int64 i) RETURNS boolean

Whether cell i is null OR present-but-whitespace-only — the frame's "no usable value" two-state that requireValues/dropEmptyColumns drop on. For a numeric column this is exactly isNull (a number is never blank); a text column adds the whitespace-only reading of its bytes.

METHOD withName(String newName) RETURNS IColumn

A fresh, independently-owned copy of this column under newName, cells and validity verbatim (a CSV-loaded numeric column keeps its source text, so it still round-trips byte-exact). The seam the name/set transforms (normalizeHeaders/select/dropDuplicateColumns/dropEmptyColumns) build a derived frame from.

METHOD filterRows(boolean[] keep) RETURNS IColumn

A fresh copy keeping only the rows where keep[i] is TRUE — keep.length must equal this column's length. Cells, validity and any source text are carried across position-aligned, so a surviving row still round-trips byte-exact. The row-drop seam requireValues builds on.

METHOD gatherRows(int64[] rows) RETURNS IColumn

A fresh owned copy whose cell j is source cell rows[j]; -1 yields a null cell. Repeats and reorders allowed — the gather seam for join / sort / group-first / concat.

METHOD compareRows(int64 a, int64 b) RETURNS int32

Order cell a against cell b: -1 before, 0 equal, 1 after. Contract: BOTH cells are present — DataFrame.sortBy owns the null policy, so a kind only has to order real values.

STRUCT AggSpec

One measure of a GroupBy.agg call: which column, which reduction, and the name the result carries.

Data/src/structs.ev:30

Fields

STRUCT ColumnSpec

ColumnSpec — one entry of a CsvSource->requireColumns(...) schema contract: a column name bound to the type it must have (DESIGN.md §7.4).

Data/src/structs.ev:38

Fields

STRUCT CsvOptions

CsvOptions — the dialect knobs a CsvSource / CsvSink reads and writes by. delimiter / quote are binary (ASCII): CSV's structural characters are always ASCII, and the tokenizer scans the file as binary[] (what File.read() hands back), so keeping the markers at that width avoids a per-byte cross-width conversion in the inner loop. Non-ASCII delimiters are out of scope.

nullSentinels are the cell texts read as NULL in addition to an empty field (DESIGN.md §7.5) — pandas' na_values. A whitespace-only field is NOT null; it is a present blank, which is the two-state distinction isNull vs isBlank rests on.

Data/src/structs.ev:19

Fields