Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Work in progress. This course has not yet received the author’s final audit. Chapters, exercises, and commands may still change before the course is marked complete.

Build a Typed Database Expression Engine in Rust

A hand-written loop for i32 + i32 is easy:

for row in 0..left.len() {
    output.push(match (left.get(row), right.get(row)) {
        (Some(left), Some(right)) => Some(
            std::ops::Add::add(std::num::Wrapping(left), std::num::Wrapping(right)).0,
        ),
        _ => None,
    });
}

The design problem appears when the engine must also borrow strings without copying, read constants and Indexed views, promote mixed numeric types, reject bad arity and lengths, and choose a function from runtime names. Repeating those decisions in every loop makes each new function a new place for type drift, null bugs, and inconsistent errors.

This course builds the connections that move those decisions out of the row loop. You will first write the small cases by hand. Only after their duplication is visible will you introduce the catalogs and generic adapters that remove it.

The map has three reading directions:

  1. DataType tells the planner what a value means; PhysicalType selects storage.
  2. Scalar, ScalarRef, Array, and ArrayBuilder form one compile-time family, while erased enums cross runtime boundaries through checked conversions.
  3. ColumnViewImpl normalizes array, constant, Indexed, and typed-null representations before one selected typed expression enters its row loop.

The numeric chapters keep scalar hooks statically typed and erase only whole-batch evaluators. For signed addition, subtraction, and multiplication, std::num::Wrapping<T> makes the chosen cross-profile overflow behavior explicit while still using the standard operator traits.

Nullability is value state—Option or validity—not a DataType::Nullable variant. A one-level List adds offsets and independent outer/child validity; it does not add an aggregate engine.

What you need to know

You should be comfortable with Rust enums, traits, references, Option, and ordinary Cargo use. The course introduces generic associated types, checked runtime erasure, and return-position impl Trait in the concrete places that need them.

Every chapter names prerequisites, exact starter targets, required work, extensions, and a copied test. Passing the test is necessary; you should also be able to explain why the new boundary exists and which failure it prevents.

Continue to Environment Setup.

Work in progress. This course has not yet received the author’s final audit. Chapters, exercises, and commands may still change before the course is marked complete.

Environment Setup

Install rustup, update stable Rust, and install mdBook:

rustup update stable
rustc --version

The repository selects rolling stable and Rust Edition 2024. It does not claim an older minimum supported Rust version.

Check out the starting state

Clone the repository, then create a working branch from main:

git fetch origin
git switch --create course-work --track origin/main

Choose another branch name if course-work already exists. Verify the untouched starter:

cargo check -p type-exercise-starter --lib --locked
cargo test -p type-exercise-starter --lib --locked

Both commands should pass. Chapter tests do not exist in the starter until you copy them.

Follow the learner boundary

  • Work only in implementation files under type-exercise-starter/src/.
  • Do not edit src/tests.rs or copied files under src/tests/.
  • Do not read, search, diff, or copy type-exercise/, archived/, Git history, or an online solution while implementing a chapter.
  • Use the chapter, copied destination test, compiler diagnostics, and official Rust documentation.
  • Add only the types, modules, dependencies, and public APIs owned by the current chapter.

The only permitted reference-to-starter operation is:

cargo x copy-test --chapter <N>

Run it without opening the source test. The command copies the cumulative tests through chapter N, removes later managed tests, and regenerates the module list. Afterward, you may read the copied destination. Its first focused run should be red until you implement the chapter.

Preview the course

mdbook serve course --open

Continue to Chapter 1: Connect One Type Family by Hand.

Work in progress. This course has not yet received the author’s final audit. Chapters, exercises, and commands may still change before the course is marked complete.

Chapter 1: Connect One Type Family by Hand

In this chapter, you will use generic associated types (GATs) to connect the different representations of one database value: an owned scalar, a borrowed scalar reference, and a nullable array. We will make those connections for i32 and String. The same relationships will later let us implement primitive arrays once and write generic expression code without repeating it for every physical type.

A database execution engine rarely works with one Rust representation of a value. A string may arrive as an owned String, be read from an array as an &str, and live inside a compact column with thousands of other strings. These are different Rust types, but the engine must know that they belong to the same logical family.

The relationships we will build look like this:

owned scalar S  ── RefType<'a> ──> borrowed scalar S::RefType<'a>
      │                                      │
      └──────────── ArrayType ───────────────┘
                             │
                             ▼
                       concrete array

For the integer family, the owned and borrowed representations are both i32 because copying an integer is cheap. For the string family, the owned representation is String, while the borrowed representation is &'a str. The lifetime 'a ties the borrowed string to the scalar or array that stores its bytes.

What is in the starter

The Day 1 starter is deliberately small. It exposes only the two families used in this chapter: Int32 and String. PhysicalType and PhysicalFamily contain those two variants; ScalarImpl, ScalarRefImpl, and ArrayImpl contain their two erased variants. The starter also provides placeholder PrimitiveArray<T> and StringArray types and their builders. These declarations let the crate compile, but they do not implement the family relationships or store any values yet.

The Scalar, ScalarRef, Array, and ArrayBuilder traits begin as unbounded shells. They name the associated types and operations you will connect, but their supertraits, where clauses, and reciprocal associated-type bounds are learner work. Later physical types and later-day relationships appear only in comments or docstrings; they are not executable scaffolding for you to work around.

The comments beside each missing relationship name the checkpoint that owns it. Treat those comments as the implementation boundary: complete the two Day 1 families, but do not add the later families yet.

Copy the supplied Chapter 1 test and run it once before editing the starter:

cargo x copy-test --chapter 1 --checkpoint 1
cargo test -p type-exercise-starter chapter_1 --locked

The test is cumulative course material; do not edit the copied file. Work only in the learner files named below.

Checkpoint 1: Implement the Scalar and ScalarRef traits

Open src/scalar.rs. The starter already distinguishes owned Int32 and String values in ScalarImpl and borrowed Int32 and String values in ScalarRefImpl<'a>. What it does not yet express is that each owned type has exactly one borrowed type and one array type, and that the borrowed type points back to the same family.

Complete only the owned↔borrowed bounds and associated-type relationship on Scalar and ScalarRef. Scalar::ArrayType and ScalarRef::ArrayType remain unconstrained associated-type placeholders in this checkpoint; do not require them to implement Array or tie them reciprocally yet. Use RefType<'a> and ScalarType to make the owned and borrowed directions agree: if String::RefType<'a> is &'a str, then that reference must identify String as its owned scalar. Checkpoint 3 will connect both scalar forms to the concrete array and builder once those implementations exist.

Then implement the owned↔borrowed relationship for the two families. The array names remain placeholders until Checkpoint 3:

i32    <──owned/borrowed──> i32       <──array──> I32Array
String <──owned/borrowed──> &'a str   <──array──> StringArray

This is where the GAT matters. A normal associated type could say that String has some reference type, but it could not produce a different &'a str for every lifetime chosen by the caller. type RefType<'a> preserves that caller-chosen lifetime.

Why spend this effort on relationships before implementing an expression engine? Consider nullable equality, a common database scalar operation:

use crate::Scalar;

fn nullable_eq<'a, S>(
    left: Option<S::RefType<'a>>,
    right: Option<S::RefType<'a>>,
) -> Option<bool>
where
    S: Scalar,
    S::RefType<'a>: PartialEq,
{
    match (left, right) {
        (Some(left), Some(right)) => Some(left == right),
        _ => None,
    }
}

assert_eq!(nullable_eq::<i32>(Some(7), Some(7)), Some(true));
assert_eq!(nullable_eq::<String>(Some("db"), Some("rust")), Some(false));
assert_eq!(nullable_eq::<String>(None, Some("rust")), None);

The function describes the database rule once: compare two non-null values, otherwise produce NULL. The type family decides whether the compared values are copied integers or borrowed strings:

Chapter 1 does not build the generic expression framework yet. The supplied Checkpoint 1 compile witness uses this same idea to prove that each owned scalar and borrowed scalar point back to one another. When this checkpoint passes, generic code can name S::RefType<'a> without separately teaching it the Int32 and String cases; no Array or builder relationship is required yet.

Run the focused test again:

cargo x copy-test --chapter 1 --checkpoint 1
cargo test -p type-exercise-starter chapter_1 --locked

Checkpoint 2: Add scalar type erasure

The traits from Checkpoint 1 work when Rust knows the concrete type S at compile time. A database plan does not always have that information in its Rust type. A scan may read a runtime schema, or an expression node may hold a value whose physical type is known only after binding. We therefore need one runtime container for all scalar families supported so far.

That is the role of ScalarImpl and ScalarRefImpl<'a> in src/scalar.rs. The first owns a value; the second can borrow one. For Day 1, each enum contains only Int32 and String. The enums erase the concrete Rust type at the runtime boundary while their variants preserve enough information to recover it safely.

Implement the Day 1 erased methods and conversions in src/scalar.rs and the display/error behavior of the existing TypeMismatch carrier in src/physical_type.rs:

  • ScalarImpl::physical_type and ScalarRefImpl::physical_type report the variant’s PhysicalType.
  • ScalarRefImpl::to_owned_scalar turns an erased borrowed value into the matching erased owned value.
  • From<T> moves a correctly typed value into its erased enum and cannot fail.
  • TryFrom<ScalarImpl> and TryFrom<ScalarRefImpl<'a>> recover a requested concrete type.
  • A matching variant returns the value.
  • A nonmatching variant returns TypeMismatch; it must not panic or reinterpret the value.

For example, converting 42_i32 into ScalarImpl and back to i32 succeeds. Asking for a String from that same ScalarImpl::Int32 returns an error. This fallibility is why the generic traits alone are not enough: generics prevent a mismatch inside statically typed code, while an erased runtime boundary must check the variant it receives.

Keep the distinction between owned and borrowed erasure visible. ScalarImpl::String owns a String; ScalarRefImpl::String holds an &str with the caller’s lifetime. Do not allocate a new String merely to erase a borrowed value.

Run the same focused test. Its scalar-erasure checkpoint should now round-trip both families and reject a cross-family downcast.

cargo x copy-test --chapter 1 --checkpoint 2
cargo test -p type-exercise-starter chapter_1 --locked

Checkpoint 3: Implement primitive and string arrays

Now connect the array-type placeholders from Checkpoint 1 to concrete arrays. Open src/array.rs, src/array/primitive_array.rs, and src/array/string_array.rs. Add the reciprocal Scalar↔Array and Array↔ArrayBuilder bounds here, then implement the storage and access methods for I32Array and StringArray.

Why arrays? Database execution engines usually process columns in batches instead of dispatching one operator for every row. Conceptually, a vectorized binary expression performs the same scalar operation across two input arrays:

for row in 0..input_len {
    result.push(scalar_func(left[row], right[row]));
}
return result;

Later chapters will build the reusable vectorization layer. In this chapter, the goal is the representation underneath it: the array must return the scalar reference associated with its family, preserve nulls, and support append-only construction.

Use the small Arrow-style layouts required by the supplied tests. They are teaching layouts inspired by Arrow’s columnar separation; this chapter does not claim full Apache Arrow compatibility.

For PrimitiveArray<i32>, store:

  • one contiguous Vec<i32> with one value slot per row; and
  • one packed BitVec validity bitmap, where true means the row is non-null.

A null integer row still occupies a value slot, using the type’s default value as an ignored placeholder. Nullness comes from the validity bit, not from wrapping every stored value in Option<i32>.

For StringArray, store:

  • one contiguous Vec<u8> containing the UTF-8 bytes for all rows;
  • an offsets vector with row_count + 1 entries; and
  • one packed BitVec validity bitmap.

Row i occupies the half-open byte range offsets[i]..offsets[i + 1]. The first offset is zero, the offsets never decrease, and the last offset is the byte-buffer length. A null row and an empty string may repeat an offset; the validity bit distinguishes them. Because the bytes live in the array, StringArray::get returns an &str borrowed from that buffer rather than allocating a String.

Implement the Day 1 array surface described by the starter comments:

  • array access: get, len, is_empty, iter, and the read-only buffer accessors used by the tests;
  • construction: with_capacity, push, and finish on each builder; and
  • Array::from_slice, which builds an array through its associated builder.

Preserve the row count and null position for normal, empty, and all-null inputs. String offsets count UTF-8 bytes, not characters.

cargo x copy-test --chapter 1 --checkpoint 3
cargo test -p type-exercise-starter chapter_1 --locked

When this checkpoint passes, the scalar relationship from Checkpoint 1 becomes observable: I32Array::get produces Option<i32>, while StringArray::get produces Option<&str> borrowing the array.

Checkpoint 4: Add array type erasure with a macro

Concrete arrays are ideal for generic code, but a database operator often receives a column selected from a runtime schema. ArrayImpl in src/array.rs is the erased boundary for that case. On Day 1 it has only Int32(I32Array) and String(StringArray) variants.

Implement the common erased-array operations and the checked conversions between each concrete array and ArrayImpl. As with scalar erasure, upcasting with From cannot fail, while downcasting with TryFrom must return TypeMismatch for the wrong variant. Support both owned recovery and borrowed recovery so callers can inspect an erased array without cloning its buffers.

The two families need the same conversion shape. Write that shape once as a macro_rules! macro, then invoke it for Int32 and String. Keep the family inventory in src/variant_catalog.rs to exactly the two Day 1 rows. The catalog supplies the type names to the macro; it must not contain the later physical families yet.

The point of this macro is narrow: remove repetitive enum conversion code while keeping each generated implementation ordinary, inspectable Rust. It is not a generic reflection system. Later chapters will extend the catalog and reuse the same expansion boundary.

Finish by checking these behaviors:

  • an I32Array and a StringArray each round-trip through ArrayImpl;
  • borrowed recovery returns a reference to the original concrete array;
  • asking for I32Array from ArrayImpl::String returns TypeMismatch;
  • erased get preserves nulls and returns the matching ScalarRefImpl variant; and
  • the physical-family catalog contains exactly Int32 and String.

Run the focused test, then the starter library tests:

cargo x copy-test --chapter 1 --checkpoint 4
cargo test -p type-exercise-starter chapter_1 --locked
cargo test -p type-exercise-starter --lib --locked

Before continuing, make sure you can explain three boundaries in your own words:

  1. Why does String need RefType<'a> = &'a str, while i32 can use RefType<'a> = i32?
  2. Why can generic code trust a Scalar relationship, while erased code must perform a checked downcast?
  3. Which bytes represent a null string row, and which structure tells you that it is null rather than empty?

You have connected the first two concrete families by hand. Chapter 2 will extend the physical-family catalog and let the macros reproduce those connections for more types without turning the Day 1 starter into a completed framework.

Next: Chapter 2 scales the family without copying every connection.

Work in progress. This course has not yet received the author’s final audit. Chapters, exercises, and commands may still change before the course is marked complete.

Chapter 2: Scale the Physical Type Family

Chapter 1 connected Int32 and String by hand. An owned scalar, its borrowed form, its nullable array, and its erased runtime variant now agree through Rust’s associated types. That was useful while there were two families. Adding five more primitive families by copying those implementations would make the boilerplate larger than the storage idea.

This chapter separates ordinary generic storage from the finite runtime catalog. Six explicit aliases name the supported primitive arrays, while one generic PrimitiveArray<T> implementation owns their identical storage behavior. Its ordinary trait bounds require the complete scalar, array, builder, and erased-conversion relationship. The physical catalog remains for work Rust generics cannot express: erased enum variants and their variant-specific conversions. Then we will add Decimal as the important exception: its precision and scale are chosen at runtime. DecimalArray therefore wraps reused PrimitiveArray<i128> coefficient and validity storage with one checked DecimalType shared by the whole array.

The goal is not to hide the type system behind a general framework. It is to keep two kinds of variation separate:

Static family identityRuntime type metadata
What varies?The Rust scalar, borrowed scalar, array, and builder typesValues that describe one physical family at runtime
Examplesi64I64Array; f64F64Array; StringStringArrayDecimalType { precision, scale } shared by a Decimal array
Where is it enforced?Explicit public aliases plus the complete Scalar/Array/conversion bounds on the generic implementationChecked constructors plus PhysicalType::Decimal(decimal_type) at erased boundaries

What is in the starter

Begin from your completed Chapter 1 workspace. src/variant_catalog.rs contains exactly the Int32 and String rows. Those rows already drive scalar and array erasure. src/physical_type.rs, src/scalar.rs, and src/array.rs therefore expose two physical families, and src/array/primitive_array.rs has the working I32Array layout you implemented.

The Day 2 starter does not predeclare the rest of the solution. Comments in the active files mark where the primitive variants and aliases belong. src/data_type.rs, src/decimal.rs, and src/array/decimal_array.rs are still docstrings rather than executable declarations, and their modules remain commented in src/lib.rs and src/array.rs. You will make those files executable only when their checkpoints introduce the concepts.

Copy the cumulative supplied test before editing:

cargo x copy-test --chapter 2
cargo test -p type-exercise-starter chapter_2 --locked

The Chapter 2 test is one final contract, not four progressive test files. Its first run should fail because the new arrays, logical types, and Decimal types do not exist yet. Do not edit the copied test. Checkpoints 1 and 2 can use a library check to catch local compiler errors. Checkpoints 3 and 4 share one compile boundary because the final DataType includes Decimal; enable their modules together after both implementations exist. The focused Chapter 2 test becomes green after all four checkpoints are complete.

cargo check -p type-exercise-starter --lib --locked

Checkpoint 1: Generalize the primitive array family

Open src/array/primitive_array.rs. Chapter 1 implemented Array and ArrayBuilder directly for the I32Array aliases. The storage does not depend on i32: every copyable primitive family uses one contiguous Vec<T> plus one packed validity bitmap. What changes from family to family is the Rust scalar type and the public alias name.

Generalize that implementation without inventing another marker trait. Write the six public primitive aliases explicitly, for example:

pub type F64Array = PrimitiveArray<f64>;
pub type F64ArrayBuilder = PrimitiveArrayBuilder<f64>;

Implement Array once for PrimitiveArray<T> and ArrayBuilder once for PrimitiveArrayBuilder<T>, with bounds connecting T to the matching Scalar, ScalarRef, and erased ArrayImpl family. Those existing relationships are already the exact admission rule: an arbitrary PrimitiveArray<T> remains useful as internal storage, but it does not become a database Array unless T satisfies the complete static family contract. Do not duplicate that contract with a private marker trait.

Keep the Chapter 1 layout unchanged. push(None) still appends a default placeholder and a false validity bit. get consults validity before returning the copied value. NaN, infinity, and signed zero are stored as their original floating-point bit patterns; do not add an equality or ordering requirement just to make the generic implementation convenient.

The six aliases are ordinary Rust declarations, not generated execution code. Re-export them from src/array.rs; their scalar and erased-enum relationships become complete when Checkpoint 2 adds the remaining physical catalog rows.

cargo check -p type-exercise-starter --lib --locked

Checkpoint 2: Make the catalog own the repeated relationships

Now open src/variant_catalog.rs. Each row names six facts that otherwise have to stay synchronized:

storage kind, erased variant, array, builder, owned scalar, borrowed scalar

Extend the inventory with the static Day 2 families: Int16, Int64, Bool, Float32, and Float64. Int32 remains in place, and String remains the one borrowed row because its array yields &str rather than copying an owned String.

The existing catalog callbacks in src/scalar.rs and src/array.rs should now generate the new erased scalar and array variants, scalar-family relationships, physical-type dispatch, and variant-specific checked conversions without five hand-written copies. They do not generate the primitive aliases or duplicate the generic Array implementation from Checkpoint 1. Add the matching variants to PhysicalType and PhysicalFamily in src/physical_type.rs, and keep PHYSICAL_FAMILY_CATALOG in the same public order. The supplied test treats that public list as an audit surface: an omitted, duplicated, or misnamed family is a failure even if some generated code still compiles.

This is a declarative macro, not runtime reflection. After expansion, Rust still sees concrete items such as impl Scalar for f64, ArrayImpl::Float64(F64Array), and a checked TryFrom<ArrayImpl> for F64Array. The compiler checks the same reciprocal relationships from Chapter 1 for every static row:

owned scalar <-> borrowed scalar <-> concrete array <-> builder

Do not implement the static Scalar/Array family contract for i128. Checkpoint 4 will add Decimal’s catalog row after the descriptor-bearing types exist. The Decimal wrapper may reuse PrimitiveArray<i128> as internal coefficient and validity storage, but its builder still needs a runtime DecimalType before it can accept any row.

cargo check -p type-exercise-starter --lib --locked

Checkpoint 3: Separate logical type from physical storage

So far, PhysicalType answers an execution question: which scalar and array representation is in memory? A planner asks a different question. SQL CHAR(7) and VARCHAR have different logical meaning, but this course stores both in the String physical family. That distinction belongs in a planner-visible DataType, not in StringArray.

Replace the docstring in src/data_type.rs with DataType and its methods. Add the primitive logical variants and the two string variants, then map them explicitly:

Logical DataTypePhysical storage
SmallIntInt16
IntegerInt32
BigIntInt64
BooleanBool
RealFloat32
DoubleFloat64
Varchar, Char { width }String

Implement physical_type, is_string, and is_numeric. Boolean is not numeric. The width in Char { width } remains logical metadata even though it does not change the physical array.

Do not add a nullable logical variant or List. Keep one primitive array representation with its validity bitmap. Nullability is a physical property beside PhysicalType, expressed as Nullability::{NonNull, Nullable}. Day 10 will make ColumnViewImpl carry that property and make expressions derive their output property with Expression::output_nullability; BoundExpression only delegates it. An ordinary ColumnViewImpl::array remains conservatively Nullable. A checked try_non_null_array can establish NonNull once, after which the selected dense loop reads the same array’s values() and leaves its bitmap structurally present but unused. This does not require a second Arrow array type or a cached null count. List arrives with its own scalar and array relationships on Day 11.

Checkpoint 4 adds the Decimal variants and checked constructor to this same file. After that work, uncomment the data_type and decimal modules and exports in src/lib.rs; do not enable any later-day module.

Checkpoint 4: Keep Decimal metadata with the physical value

An f64 value carries its interpretation in its bits. An i128 coefficient does not tell you whether 12345 means 12345, 123.45, or 12.345. Decimal therefore cannot use the static primitive relationship unchanged.

Adding a dependency is the one exception to this chapter’s normal source-only editing boundary. From the repository root, add anyhow to the learner crate before continuing:

cargo add anyhow@1 --package type-exercise-starter

This updates both type-exercise-starter/Cargo.toml and the workspace lockfile, so the documented --locked checks below remain reproducible. Then implement DecimalType and Decimal in src/decimal.rs with anyhow::Result. This chapter needs readable checked failures, not a public Decimal-specific error taxonomy. DecimalType owns the precision and scale and accepts only:

1 <= precision <= 38
0 <= scale <= precision

The scale is an unsigned u8, so it is nonnegative by construction. A Decimal pairs one checked i128 coefficient with a DecimalType; its represented value is unscaled * 10^(-scale). A coefficient is valid when its absolute value is strictly less than 10^precision. Use an overflow-safe absolute value so i128::MIN returns an ordinary error instead of panicking.

Next implement DecimalArray and DecimalArrayBuilder in src/array/decimal_array.rs. DecimalArray is a logical metadata wrapper around PrimitiveArray<i128>:

Stored stateRole
DecimalType { precision, scale }One checked descriptor shared by the entire array
PrimitiveArray<i128>One flat coefficient slot and one validity bit per row

Do not cache a null count. The reused primitive representation already owns the coefficient buffer and validity bitmap.

Require the descriptor before the first push with DecimalArrayBuilder::try_with_type. Store zero as the ignored coefficient for a null row, just as other primitive arrays use a placeholder value. Empty and all-null arrays must retain their DecimalType; the descriptor cannot be inferred from a non-null row because such a row may not exist.

Validate before mutation. try_from_raw_parts rejects different value/validity lengths and any valid coefficient outside the declared precision. try_push rejects a Decimal whose descriptor does not exactly match the builder’s descriptor, without appending either a coefficient or a validity bit. A failed push must leave the builder in the same logical state it had before the call.

Now complete the Decimal path through the runtime types:

  • add DataType::Decimal(DecimalType) and the checked DataType::decimal constructor;
  • add PhysicalType::Decimal(DecimalType) and the descriptor-free PhysicalFamily::Decimal audit tag;
  • add the decimal row to for_each_physical_family! and to PHYSICAL_FAMILY_CATALOG;
  • enable and re-export decimal_array from src/array.rs; and
  • make ScalarImpl, ScalarRefImpl, and ArrayImpl report the exact descriptor from physical_type().

Do not add a try_decimal(expected) convenience method. A caller that requires one precision and scale first compares the erased value’s physical_type() with PhysicalType::Decimal(expected). Only after equality does it use the existing checked conversion—Decimal::try_from, <&DecimalArray>::try_from, or owned DecimalArray::try_from—when it actually needs a typed value. A descriptor mismatch is a physical-type mismatch at the caller; converting the wrong erased family returns an ordinary anyhow failure such as expected a Decimal value, got Int32. Code that only carries an erased value forward does not need to force a Decimal conversion.

Decimal does not implement the Chapter 1 Scalar/Array static-family contract because that contract fixes the physical type in the Rust type relationship and constructs builders with ArrayBuilder::with_capacity(capacity). Decimal’s precision and scale are runtime values, and an empty or all-null builder cannot infer them from a row. DecimalArrayBuilder::try_with_type(decimal_type, capacity) must therefore receive the descriptor up front. The Decimal catalog arm generates only the erased enum plumbing for this metadata-bearing wrapper; it neither duplicates primitive storage nor repeats precision and scale in every row.

This chapter does not implement Decimal arithmetic, comparison, rounding, casts, or implicit coercion. It establishes the representation and checked runtime boundary that those operations would have to preserve.

Run the final contract and the starter library tests:

cargo test -p type-exercise-starter chapter_2 --locked
cargo test -p type-exercise-starter --lib --locked

Before continuing, make sure you can explain three boundaries in your own words:

  1. Which existing scalar, array, and conversion bounds admit PrimitiveArray<f64> to the generic database-array implementation, while an arbitrary PrimitiveArray<T> does not qualify?
  2. Why do Char { width } and Varchar remain distinct logical types even though both map to PhysicalType::String?
  3. Why can DecimalArray reuse PrimitiveArray<i128> storage while DecimalArrayBuilder still cannot use the metadata-free ArrayBuilder::with_capacity constructor?

You now have one compile-time inventory for the repeated static relationships and one explicit, checked path for runtime metadata. Chapter 3 will use those physical families through several nullable column encodings.

Next: Chapter 3 reads several nullable column encodings.

Work in progress. This course has not yet received the author’s final audit. Chapters, exercises, and commands may still change before the course is marked complete.

Chapter 3: Read Nullable Columns Without Materializing Them

Chapter 2 gave the executor several physical families. A row loop still should not need separate code for an array, one scalar repeated across a batch, or an index into shared values. Those are different representations of a column, not different scalar operations.

This chapter gives them one borrowed boundary. ColumnViewImpl<'a> keeps the representation known at runtime. After one checked conversion, ColumnView<'a, S> lets generic code read nullable rows as Option<S::RefType<'a>>. The views borrow their buffers, so constants and indexed columns do not need to materialize another array first.

What is in the starter

Begin from your completed Chapter 2 workspace. The scalar, array, erasure, logical-type, and Decimal work from the first two chapters is already present. src/column.rs contains only the Day 3 comment shells for ColumnViewImpl<'a> and ColumnView<'a, S>. The column module and its public exports remain commented in src/lib.rs.

You own two additions in this chapter:

  1. a representation-erased borrowed view with checked constructors; and
  2. a typed borrowed view that checks one physical family before row access.

Later relationships in the starter remain comments. Do not implement Day 10 nullability proofs or Day 11 List views here.

Copy the cumulative supplied test before editing:

cargo x copy-test --chapter 3
cargo test -p type-exercise-starter chapter_3 --locked

The first run should fail because ColumnViewImpl and ColumnView are not exported yet. Do not edit the copied test. The two checkpoints below use one final Chapter 3 contract, so Checkpoint 1 has a library compile gate and Checkpoint 2 makes the focused test green.

Checkpoint 1: Borrow each physical representation

Open src/column.rs and implement ColumnViewImpl<'a>. It represents four ways a batch can supply logical rows:

RepresentationBorrowed stateLogical lengthPhysical type
Array&'a ArrayImplarray lengtharray physical type
Constantone ScalarRefImpl<'a> plus a lengthrecorded lengthscalar physical type
Typed nullPhysicalType plus a lengthrecorded lengthrecorded physical type
Indexedcompact &'a [u32] keys plus &'a ArrayImpl valueskey countvalues physical type

The lifetime 'a is the ownership boundary: the view may borrow an array, keys, or a string scalar, but it does not own or copy those buffers. Implement array, constant, null, and indexed, together with len, is_empty, physical_type, and erased row access with this exact shape:

pub fn get(&self, row: usize) -> Option<ScalarRefImpl<'a>>

Keep the representation enum private behind the public ColumnViewImpl wrapper. This small split forces callers through the constructors, so they cannot bypass the indexed bounds check. It also leaves one place for later chapters to attach batch-wide metadata instead of repeating that state inside every representation variant.

A typed null needs an explicit PhysicalType because it has no non-null scalar from which to recover one. The type still matters for overload selection and output allocation, including for an empty batch. Do not add nullable variants to DataType, PhysicalType, or the scalar families; this chapter continues to represent each logical row as Some(value) or None.

For an indexed view, each compact non-null u32 key selects one row from the borrowed values array. A null logical row lives in that nullable values array rather than in the key buffer:

keys[row] = i, values[i] null -> None
keys[row] = i, values[i] set  -> Some(values[i])

array, constant, and null are direct constructors. indexed is the fallible constructor: validate every key inside it before returning a view. If any key is outside the values array, return an ordinary anyhow::Error that identifies the row, key, and values length, and expose no partially valid view.

Enable the module for this checkpoint and export only the type you have implemented:

mod column;
pub use column::ColumnViewImpl;

Then compile the learner library:

cargo check -p type-exercise-starter --lib --locked

Passing means the real column.rs implementation compiles. The focused Chapter 3 test is still expected to fail because ColumnView<'a, S> belongs to Checkpoint 2.

Checkpoint 2: Check the scalar family once

Now implement ColumnView<'a, S> and TryFrom<ColumnViewImpl<'a>>. The erased view can report its PhysicalType, but a generic row loop wants the concrete family S. Compare the view’s physical type with S::PHYSICAL_TYPE once during conversion. A mismatch returns TypeMismatch before any row is read.

After that check, recover the matching borrowed array, borrowed scalar reference, or indexed values array once and store it in the typed view. get(row) can then return Option<S::RefType<'a>> without repeating an erased downcast for every row. The GAT relationship from Chapter 1 remains visible: ColumnView<'a, i32> returns copied i32 values, while ColumnView<'a, String> returns &'a str borrowed from the original string storage.

The generic typed view covers the catalog families that implement Scalar. ColumnViewImpl can still carry an erased Decimal array and preserve its exact PhysicalType::Decimal descriptor. Decimal does not implement the static Scalar relationship, so it is not a ColumnView<Decimal> family in this chapter.

Finish the public export in src/lib.rs:

pub use column::{ColumnView, ColumnViewImpl};

Run the focused contract, then all learner-library tests copied so far:

cargo test -p type-exercise-starter chapter_3 --locked
cargo test -p type-exercise-starter --lib --locked

The focused test proves five boundaries:

  • arrays, constants, and indexed values expose the same logical-row interface;
  • the primitive families added in Chapter 2 work without family-specific row loops;
  • typed-null and empty views retain their type and length;
  • every invalid key is rejected during construction; and
  • a physical-family mismatch fails before row access.

Keep this chapter focused on borrowed execution views. The indexed form borrows compact u32 keys and an existing nullable ArrayImpl; it is not a persisted dictionary-array format and adds no key-array family, builder, or storage encoding. Leave run-length encoding as an extension. Chapter 10 adds primitive-loop specialization, and Chapter 11 reuses this representation boundary for List.

Before continuing, make sure you can explain three boundaries in your own words:

  1. Why must an all-null or empty column carry a physical type instead of inferring one from rows?
  2. Why does indexed validate every key before it returns a view?
  3. Why can ColumnView<'a, String>::get return a borrowed &'a str without materializing a new StringArray?

You can now separate the representation of a batch from the scalar operation that reads it. Chapter 4 will use this borrowed boundary to expose what concrete unary and binary row loops repeat.

Next: Chapter 4 exposes what unary and binary loops repeat.

Work in progress. This course has not yet received the author’s final audit. Chapters, exercises, and commands may still change before the course is marked complete.

Chapter 4: Expose the Cost of Concrete Loops

Chapter 3 gave arrays, constants, typed nulls, and indexed values one borrowed row interface. That solved a representation problem. It did not yet turn a scalar operation such as i32 + i32 into a batch expression.

A batch adapter has more work to do than the addition itself. It must reject the wrong inputs before indexing, preserve strict nulls, build the correct output family, and stop cleanly if a row operation fails. This chapter writes that machinery as a fixed-arity whole-batch boundary. The kernel pointer is erased, but the operation it names is still vectorized: there is no dynamically dispatched object call for every scalar row.

What is in the starter

Begin from your completed Chapter 3 workspace. src/column.rs already provides ColumnViewImpl<'a> and the checked ColumnView<'a, S> conversion. The Day 4 files contain only comment shells:

  • src/expression.rs names the first typed binary scalar function and evaluator;
  • src/operators.rs names the fixed-arity batch shell and its vectorized kernel pointer; and
  • src/lib.rs keeps both modules and their exports commented out.

You own two additions in this chapter:

  1. one typed binary scalar operation lifted over nullable borrowed columns; and
  2. one batch expression that validates a complete fixed-arity input contract before delegating to a monomorphized row loop.

The later comments are boundaries, not implementation work. Leave numeric promotion, ternary evaluation, the erased Expression trait and catalog, primitive fast paths, and asynchronous adapters for their chapters.

Copy the cumulative supplied test before editing:

cargo x copy-test --chapter 4
cargo test -p type-exercise-starter chapter_4 --locked

The first focused run should fail because the Day 4 modules and public items do not exist yet. Do not edit the copied test.

Checkpoint 1: keep one row operation small

Open src/expression.rs and define BinaryScalarFunction with three associated scalar families: Left, Right, and Output. Its method receives the borrowed scalar-reference type for each input and returns one owned output value.

That signature keeps one row operation independent from the column representation:

pub trait BinaryScalarFunction {
    type Left: Scalar;
    type Right: Scalar;
    type Output: Scalar;

    fn evaluate<'a>(
        &self,
        left: <Self::Left as Scalar>::RefType<'a>,
        right: <Self::Right as Scalar>::RefType<'a>,
    ) -> Self::Output;
}

Implement I32Add first. Use wrapping_add explicitly. Ordinary signed addition can panic on overflow in a debug build and wrap in a release build; a database expression must not change its result with the compilation profile.

Next implement evaluate_binary. It receives two ColumnViewImpl<'a> values and one typed scalar function. The adapter, not the function, owns the batch work:

  1. convert each erased input once to ColumnView<'a, F::Left> or ColumnView<'a, F::Right>;
  2. reject unequal lengths before reading a row;
  3. allocate the builder associated with F::Output for that length;
  4. for each row, call the scalar function only when both inputs are non-null; and
  5. finish the builder and erase the owned output as ArrayImpl.

The typed conversions perform the physical-family checks. They also recover the borrowed scalar shape established in Chapter 1: a mixed-family function can receive &str from a string column and i32 from a primitive column without allocating either input value. The output family is independent of both inputs; an i32, i32 -> String function must build a StringArray.

Use the public ExpressionError enum for batch failures. Its completed Chapter 4 contract has four variants: TypeMismatch(TypeMismatch), InputArityMismatch { expected, actual }, InputLengthMismatch { expected, actual, input_index }, and ScalarEvaluation { function, row, error }. Checkpoint 1 needs the type and length cases; Checkpoint 2 completes the arity and checked-scalar cases. The exact Display sentences remain your choice. Enable expression in src/lib.rs and export ExpressionError with the checkpoint’s public function, trait, and I32Add.

The copied Chapter 4 test also imports the Checkpoint 2 shells, so it cannot be green yet. Use an honest library boundary here:

cargo check -p type-exercise-starter --lib --locked

Passing means the first vectorized loop and its public surface compile. The completed focused test will later exercise arrays, constants, and indexed views; strict nulls; a borrowed mixed-family function; an output family different from the inputs; explicit wrapping overflow; and type and length rejection.

Checkpoint 2: erase one complete batch operation

Now open src/operators.rs. Define BatchExpression<const N: usize> with a static function name, an [PhysicalType; N] input contract, one output type, and a function pointer for a complete batch. Name that pointer type BatchKernel<N>. Its signature receives the expression metadata and the borrowed &[ColumnViewImpl<'_>], and returns an owned ArrayImpl or ExpressionError.

This is the important erasure boundary. A caller may select one monomorphized batch kernel at runtime, but that kernel converts each erased column to a typed ColumnView once and owns the whole row loop. Do not introduce unary, binary, or ternary checked scalar traits, and do not store a dynamically dispatched scalar operation inside the loop.

BatchExpression::new receives the complete metadata and kernel. Its inherent evaluate method accepts &[ColumnViewImpl<'_>]. Before calling the kernel, validate in this order:

  1. the input count equals its arity;
  2. every input’s physical family equals the corresponding expected family; and
  3. every input has the same logical length as the first input.

That order is observable. An empty unary input slice is an arity error, not an indexing panic. A wrong second physical family is rejected before the row loop. A binary length mismatch is rejected before the selected batch kernel runs.

Write small test kernels for arity one and arity two. Inside each kernel, recover its typed views, allocate the associated output builder, and make each row follow one strict rule:

any required input is null -> append null; do not perform the operation
all required inputs are set -> perform the typed operation once
typed operation returns Err -> stop; return a batch error and no output array

Report the three validation failures as ExpressionError::InputArityMismatch { expected, actual }, ExpressionError::TypeMismatch(TypeMismatch { expected, actual }), and ExpressionError::InputLengthMismatch { expected, actual, input_index }. Null and error are different results: a strict null is a valid row in the output, while a scalar error ends evaluation and later rows must not run. Return that failure as ExpressionError::ScalarEvaluation { function, row, error }; the public variant and fields are part of the supplied contract, while their Display wording remains flexible.

The kernel borrows every input view and returns a new owned array. Do not materialize an input representation just to simplify the loop.

Enable operators in src/lib.rs. Export BatchExpression and BatchKernel; also export ScalarError from expression. Keep the later Expression trait and runtime catalog commented out.

Run the focused contract, then the cumulative learner-library suite:

cargo test -p type-exercise-starter chapter_4 --locked
cargo test -p type-exercise-starter --lib --locked

The 14 focused cases prove the complete boundary:

  • one typed binary evaluator works over array, constant, typed-null, and indexed representations;
  • borrowed mixed-family inputs and an independent associated output family work without per-representation loops;
  • i32 addition has explicit wrapping behavior;
  • fixed-arity batch expressions reject arity, physical-family, and length errors before kernel entry;
  • strict nulls skip the typed operation and append null; and
  • a row error inside a whole-batch kernel stops later rows and returns no partial array.

Read the shared boundary as evidence

Compare the arity-one and arity-two kernels you exercised through the same shell:

DecisionUnaryBinarySame underlying rule?
Arityexactly one inputexactly two inputsyes
Physical typescheck one expected familycheck two expected familiesyes
Lengthestablish one batch lengthrequire both lengths to matchyes
Strict nullskip on one nullskip if either input is nullyes
Row failurestop at the failing rowstop at the failing rowyes
Outputtyped builder in the kerneltyped builder in the kernelyes

The shell captures the repeated boundary decisions without erasing individual scalar operations. Chapter 5 will select generic numeric batch kernels while preserving these rules. Chapter 6 will make the shared validator public and add vectorized negation and clamp. Runtime trait-object erasure comes later, after the whole-batch path has concrete behavior to preserve.

Before continuing, make sure you can explain three distinctions in your own words:

  1. Why is strict null propagation batch control flow rather than a scalar-operation object?
  2. Why must arity, physical types, and lengths be checked before the first row is evaluated?
  3. Why is a whole-batch function pointer a different boundary from a dynamically dispatched call on every scalar row?

You can now point to the exact work required to lift one scalar operation over a nullable batch.

Next: Chapter 5 makes numeric operation selection generic.

Work in progress. This course has not yet received the author’s final audit. Chapters, exercises, and commands may still change before the course is marked complete.

Chapter 5: Make Numeric Evaluation Generic

Chapter 4 separated an ordinary typed scalar function from the whole-batch boundary around it. BatchExpression<N> validates physical inputs before calling one vectorized kernel. Writing another ad hoc runtime shell—or dynamically dispatching every scalar operation—for each numeric type pair would throw away that boundary.

The remaining problem has two parts. Given logical types such as SmallInt and Double, the database must first decide whether an implicit conversion is lossless and what logical type the result has. Only then can it choose one concrete Rust scalar type for the row operation. This chapter keeps those decisions separate: an explicit promotion table owns the database policy, and a small runtime match chooses one generic typed batch kernel before its row loop begins.

What is in the starter

Begin from your completed Chapter 4 workspace. The fixed-arity batch shell in src/operators.rs is working code; preserve its validation order and its whole-batch kernel boundary. The Day 5 surface is still deliberately small:

  • src/promotion.rs contains comment shells for one promotion row, the promotion catalog, and its lookup function;
  • src/operators.rs ends with comments for the arithmetic and comparison selectors;
  • src/array/primitive_array.rs has the Arrow-style value and validity buffers but not the all-valid constructor used by this chapter’s batch fixture; and
  • src/lib.rs leaves the promotion module and the two operator enums unwired.

You own three connected additions: the logical promotion policy, generic arithmetic selection, and generic numeric comparison. You will also add the small PrimitiveArray::from_values helper needed to construct a non-null batch directly. Leave shared arity validation and ternary evaluation for Chapter 6, runtime expression erasure for Chapter 8, and logical name binding for Chapter 9.

Copy the cumulative supplied test before editing:

cargo x copy-test --chapter 5
cargo test -p type-exercise-starter chapter_5 --locked

The focused run should fail on the missing promotion items, operator selectors, and PrimitiveArray::from_values. Do not edit the copied test.

Checkpoint 1: make widening a database policy

Open src/promotion.rs and define the public shape already named by the starter:

pub struct NumericPromotion {
    pub left: DataType,
    pub right: DataType,
    pub output: DataType,
}

pub const NUMERIC_PROMOTIONS: &[NumericPromotion] = /* every supported ordered pair */;

pub fn promote_numeric(
    left: impl Borrow<DataType>,
    right: impl Borrow<DataType>,
) -> Option<DataType>;

This is an ordered-pair catalog, not a request to let Rust choose an as cast. Enter both operand orders whenever both are supported. The complete policy for the five non-Decimal numeric types is:

left ↓ / right →SmallIntIntegerBigIntRealDouble
SmallIntSmallIntIntegerBigIntRealDouble
IntegerIntegerIntegerBigIntDoubleDouble
BigIntBigIntBigIntBigIntrejectreject
RealRealDoublerejectRealDouble
DoubleDoubleDoublerejectDoubleDouble

The unusual-looking rows state the rule. Every i16 value is exact in f32, so SmallInt with Real may stay Real. Every i32 value is exact in f64 but not in f32, so Integer with Real widens to Double. Neither f32 nor f64 represents every i64 value, so every BigInt/floating-point pair is rejected even though Rust can spell the cast.

Decimal is also a numeric logical type, but it gets no row in this table. Precision, scale, rounding, overflow, and division scale need a separate contract before an implicit Decimal operation is meaningful. A physical representation alone does not supply those semantics.

Implement promote_numeric as a catalog lookup that returns the row’s logical output or None. Do not infer a fallback from enum order or substitute a duplicate row: the supplied test audits all 25 ordered input pairs and the exact 21 supported catalog keys.

Enable promotion in src/lib.rs and export NumericPromotion, NUMERIC_PROMOTIONS, and promote_numeric. The final focused test also imports the later operator selectors, so it cannot be green at this checkpoint. Use the library boundary instead:

cargo check -p type-exercise-starter --lib --locked

Passing means the logical policy and its public lookup compile independently from physical evaluation.

Checkpoint 2: choose one arithmetic kernel before the rows

Start with the fixture helper in src/array/primitive_array.rs. It keeps the existing representation and marks every supplied value valid:

impl<T> PrimitiveArray<T> {
    pub fn from_values(values: Vec<T>) -> Self {
        let validity = BitVec::repeat(true, values.len());
        Self { values, validity }
    }
}

This constructor is not a second array format and does not change null handling. It is simply the direct counterpart to building a batch whose rows are all non-null.

Now extend src/operators.rs with the public ArithmeticOperator variants Add, Subtract, Multiply, and Divide. Describe the five concrete numeric types with standard operator bounds, not another trait whose methods re-name arithmetic:

trait Numeric:
    Scalar
    + Copy
    + PartialOrd
    + Add<Output = Self>
    + Sub<Output = Self>
    + Mul<Output = Self>
    + Div<Output = Self>
{ /* representation used by standard Add/Sub/Mul */ }

Implement the small representation bridge explicitly for i16, i32, i64, f32, and f64. Express addition, subtraction, and multiplication through the standard Add, Sub, and Mul traits. For signed integers, apply those traits to std::num::Wrapping<T> and recover .0; this keeps the course’s deterministic wrapping result in debug and release builds. Standard Add on a bare signed integer does not itself choose one cross-profile overflow policy, so changing overflow into an error would be a separate product-semantic decision rather than part of this generic refactor. Floating-point implementations use the standard traits directly and retain ordinary IEEE results.

Division stays the one small custom fallible operation because stable std has no single checked division trait covering both the course’s integers and floats. Integer division reports DivisionByZero for zero and DivisionOverflow for MIN / -1. Treat both 0.0 and -0.0 floating-point divisors as division by zero; other results such as infinity or NaN remain values.

The important Rust boundary is where all three generic types become concrete. The physical builder matches the validated (left, right, output) tuple once and stores the selected monomorphized whole-batch function pointer in one concrete NumericBinaryExpression. Require O: TryFrom<L, Error = Infallible> + TryFrom<R, Error = Infallible> for the lossless conversions admitted by the promotion table. This uses Rust’s standard conversion vocabulary; do not add a parallel conversion trait.

That function pointer owns the complete vectorized evaluation. It converts each erased column to its typed view once, then the row loop receives L and R values directly, converts them to O with TryFrom, and applies the selected standard operation. Do not accept ScalarRefImpl, create a per-scalar erased operation object, re-run logical promotion, or match physical variants inside every row. The caller must obtain the logical output from promote_numeric first; an unsupported pair never reaches the physical builder.

Keep build_numeric_binary_expression and its returned shell crate-private. Export ArithmeticOperator from the crate root, but do not turn the physical constructor into a public user API: Chapter 9 will place logical name binding in front of it.

The copied test still imports numeric comparison, so use the library compile boundary again:

cargo check -p type-exercise-starter --lib --locked

Passing means all four arithmetic choices share one monomorphized batch evaluator without widening the public runtime boundary.

Checkpoint 3: return Boolean through the same common type

Add the six public ComparisonOperator variants: Less, LessOrEqual, Greater, GreaterOrEqual, Equal, and NotEqual. A crate-private NumericCompare<L, R, O> reuses the same typed TryFrom conversions and tuple-selected batch kernel, but it builds bool. Keep build_numeric_comparison_expression crate-private.

Both inputs may be converted to f64 for comparison while the batch kernel builds a BoolArray. The runtime selector still chooses once before the rows.

Rust’s floating-point comparisons supply the required NaN behavior: <, <=, >, >=, and = are false when either relevant comparison is unordered, while != is true. Do not turn NaN into a batch error. Strict null handling remains different: if either input row is null, the batch kernel appends null and performs no comparison.

Export ComparisonOperator beside ArithmeticOperator, then run the completed contract:

cargo test -p type-exercise-starter chapter_5 --locked
cargo test -p type-exercise-starter --lib --locked

The 9 focused cases and 42 cumulative learner tests prove the whole Day 5 boundary:

  • the catalog contains exactly the approved ordered promotions and rejects every lossy pair;
  • arithmetic works in both mixed operand orders and builds the promoted physical family;
  • signed overflow wraps, while division by zero and signed division overflow stop the batch;
  • a strict null prevents even a failing divide from being called;
  • nonzero IEEE results and all six comparison operators retain their defined behavior; and
  • comparison reuses the Chapter 4 arity, physical-type, length, null, and complete-output rules.

Read the two decisions separately

The promotion table and the generic kernel solve different problems. The table answers a logical question before evaluation: “Is this implicit conversion allowed, and what is the result type?” The physical match answers a Rust question once: “Which concrete Scalar implements this operation?” The selected kernel then answers the batch question for every row. Collapsing those three stages into as f64, a per-row type match, or another handwritten loop would make the code shorter by hiding the policy you need to audit.

Before continuing, make sure you can explain these boundaries in your own words:

  1. Why may SmallInt + Real produce Real while Integer + Real produces Double?
  2. Why is every BigInt/floating-point pair absent even though Rust provides an as conversion?
  3. Why does the physical builder select (L, R, O) once instead of matching scalar variants in each row?
  4. Why is null / 0 a null row rather than a division error?

You now have generic numeric operation selection without changing the batch contract that made the concrete loops correct. Chapter 6 will publish their validator across arities and add a real vectorized ternary path.

Next: Chapter 6 makes expression arity systematic.

Work in progress. This course has not yet received the author’s final audit. Chapters, exercises, and commands may still change before the course is marked complete.

Chapter 6: Make Arity Systematic

Chapter 5 chose one typed binary kernel before each batch, then let that kernel own the typed row loop. Its validator already accepts a slice of expected input types, but it remains private inside the operator module. A new three-input function would still be easy to implement as another special case, repeating the arity, physical-type, length, null, output, and error work that Chapters 4 and 5 separated from physical selection.

This chapter makes that boundary explicit. You will publish the shared validator already used by the existing batch kernels, add one typed three-input kernel with the same batch contract, and execute clamp(value, lower, upper) through it. The result is not a generic expression framework. It is one more concrete arity that shows which parts of evaluation vary with the number of inputs and which parts stay unchanged.

What is in the starter

Begin from your completed Chapter 5 workspace. In src/operators.rs, the unary and binary expressions already own monomorphized whole-batch kernels. Both paths call validate_expression_inputs, but that helper is private. The file ends with comment shells for the Day 6 additions:

  • the public shared validator;
  • one mixed-family three-input clamp kernel and its physical selector;
  • the physical numeric neg and clamp builders; and
  • the exact re-export to add in src/lib.rs.

You own those additions and the ScalarError::InvalidClampBounds variant used by clamp. Keep the existing binary row loops intact. Logical function registration waits until Chapter 9, and concrete four- or five-input builtins are not part of this chapter.

Chapter 6 has three cumulative supplied checkpoints. Copy the first one before editing:

cargo x copy-test --chapter 6 --checkpoint 1
cargo test -p type-exercise-starter chapter_6 --locked

The focused run should fail because the Day 5 validator is still private. Do not edit the copied test. Checkpoints 1 and 2 deliberately avoid importing later constructors; Checkpoint 3 copies the completed Chapter 6 test.

Checkpoint 1: share validation across arities

Open src/operators.rs. validate_expression_inputs already checks a batch before either the unary or binary row loop allocates an output or calls a scalar function. Make that helper public without changing its order:

  1. compare the actual and expected arities;
  2. compare physical types in input order;
  3. compare every later input length with input zero; and then
  4. allow row evaluation to begin.

That precedence is observable when more than one fact is wrong. A type error in a later column must win over an earlier length mismatch because all physical types are checked before any length. The helper returns the batch length only after every check passes.

Its expected types are a slice rather than a two-element array. The supplied test uses that fact directly: four valid two-row inputs return Ok(2), while wrong arity, a later wrong type, and a later wrong length produce their existing ExpressionError categories. This does not require a four-input expression. It proves that validation itself is not binary-specific.

Publish only the validator from src/lib.rs; ExpressionError is already public:

pub use operators::validate_expression_inputs;

Run the same checkpoint again:

cargo x copy-test --chapter 6 --checkpoint 1
cargo test -p type-exercise-starter chapter_6 --locked

Passing this checkpoint means the shared boundary works for an arbitrary expected-type slice. The ternary API is still absent.

Checkpoint 2: add one typed ternary batch loop

Start build_numeric_clamp_expression with the exact mixed tuple used by this checkpoint: (i16, i32, i64) -> i64. A private NumericClampExpression stores the function name, those three physical input types, the output type, and one whole-batch function pointer. It does not store a scalar operation object.

The selected evaluate_numeric_clamp<i16, i32, i64, i64> kernel follows the boundary you already built:

  1. call validate_expression_inputs before allocating output;
  2. convert the three erased columns to ColumnView<i16>, ColumnView<i32>, and ColumnView<i64> once;
  3. read all three positions for each row;
  4. use TryFrom to promote the three present values inside that row; and
  5. append the typed result, a strict null, or the existing row-carrying scalar error.

Add ScalarError::InvalidClampBounds in src/expression.rs. The supplied witness uses it in a small mixed-family clamp with i16, i32, and i64 inputs and an i64 output. It also verifies that a null in any input skips the operation and that invalid bounds preserve the function name and failing row.

Copy and run the cumulative second checkpoint:

cargo x copy-test --chapter 6 --checkpoint 2
cargo test -p type-exercise-starter chapter_6 --locked

Passing now means one direct typed ternary batch kernel is complete. The full numeric clamp selector is still missing.

Checkpoint 3: select a real ternary kernel once

Finish src/operators.rs with the crate-private physical builders named by the starter: build_numeric_neg_expression and build_numeric_clamp_expression. They receive already-selected physical families, just as Chapter 5’s binary builder does. Chapter 9 will place logical name binding in front of them.

Numeric negation owns one typed unary batch kernel. For signed integers, apply the standard Neg trait to std::num::Wrapping<T> and recover .0; negating MIN then has the same wrapping result in debug and release builds. Floating-point negation uses the ordinary standard operation. Its row loop remains strict over nulls.

Clamp is the observable three-input path. Generalize the private evaluate_numeric_clamp<A, B, C, O> batch kernel from Checkpoint 2. Require O: TryFrom<A> + TryFrom<B> + TryFrom<C> with Infallible errors, then promote the value, lower bound, and upper bound to O inside each present row. Bounds are valid only when lower.partial_cmp(&upper) is Less or Equal. A lower bound greater than the upper bound, or an unordered floating-point comparison involving NaN, returns InvalidClampBounds.

Choose the whole-batch kernel from the exact (value, lower, upper, output) physical tuple once, before evaluation begins. It validates the batch, converts the three columns once, and runs its single typed row loop. Do not materialize three promoted arrays or match erased scalar variants inside each row.

The legal tuple comes from applying Chapter 5’s lossless promotion table twice: first to (value, lower), then to that result and upper. The second result is the output family. For example, (i16, i32, i64) -> i64 and (i32, f32, i16) -> f64 are legal. A tuple that needs a missing promotion, such as one mixing i64 with a floating-point family, never reaches this physical builder.

Copy the final checkpoint and run the completed contract:

cargo x copy-test --chapter 6 --checkpoint 3
cargo test -p type-exercise-starter chapter_6 --locked
cargo test -p type-exercise-starter --lib --locked

The focused cases now cover generic validation beyond ternary arity, the direct mixed-family ternary witness, strict null propagation, row-carrying invalid-bound errors, wrapping numeric negation, every legal two-step clamp promotion tuple, and rejection of greater or unordered bounds. The cumulative library run keeps the Chapter 1–5 type, array, column-view, and expression contracts in the same learner workspace.

Read the shared boundary

Unary, binary, and ternary expressions have different typed whole-batch kernels, but the surrounding contract is the same. The shared validator answers whether row evaluation may begin. Each kernel then recovers typed borrowed columns once, applies its arity-specific strict-null rule, and builds the associated output array. The physical clamp selector chooses one concrete instantiation before that work starts.

Before continuing, make sure you can explain these boundaries in your own words:

  1. Why must every physical type be checked before the first length mismatch is reported?
  2. Why does an expected-type slice prove more than a validator hard-coded for three inputs?
  3. Why must the clamp selector bind its three physical input types and output type to one batch kernel before row evaluation begins?
  4. Why is an unordered NaN bound an error while a null input skips the clamp scalar call?

You now have a real ternary expression without scalar-operation erasure or duplicated batch checks. Chapter 7 will apply the same separation to three-valued Boolean logic, where nulls are part of the operator’s truth table rather than always strict.

Next: Chapter 7 adds three-valued Boolean logic with SQL null semantics.

Work in progress. This course has not yet received the author’s final audit. Chapters, exercises, and commands may still change before the course is marked complete.

Chapter 7: Implement Three-Valued Boolean Logic

SQL engines do not stop at two truth values. A missing value makes NULL AND FALSE false and NULL OR TRUE true, so nulls must flow into the Boolean scalar function instead of always short-circuiting.

Prerequisites: Chapter 6, the checked-expression boundary, and nullable Boolean columns.

By the end of this chapter, you will:

  • distinguish strict null short-circuiting from SQL’s non-strict null semantics;
  • implement AND, OR, and NOT over TRUE/FALSE/NULL; and
  • publish one checked expression whose validation and row loop follow the same arity-before-type-before-length contract as the earlier shells.
cargo x copy-test --chapter 7
cargo test -p type-exercise-starter chapter_7 --locked

The first run should fail on the missing Boolean operator, truth table, or expression builder.

Two null policies

The Day 4–6 shells skip the scalar function for any strict null input: the row is null, and the function never sees it. That is the Strict policy. SQL’s three-valued logic needs more: a null operand can still decide the result when the other operand is absorbing (FALSE AND ..., TRUE OR ...). That is the NonStrict policy, where nulls are passed to the scalar function and the truth table decides.

Checkpoint 1: pin the truth table

  • Target: type-exercise-starter/src/boolean_logic.rs::{NullEvaluationPolicy, BooleanOperator, BooleanTruthRow, BOOLEAN_TRUTH_TABLE}.
  • Change: declare both policies, the three operators, and the 21 required nullable-Boolean rows (nine AND, nine OR, three NOT), with FALSE absorbing for AND, TRUE absorbing for OR, and NOT NULL staying null.
  • Preserve: the row order and values match the supplied expected table exactly.
  • Run: the Chapter 7 focused test.
  • Passing means: the table rows are exactly the required three-valued truth table.

Checkpoint 2: evaluate one operator

  • Target: type-exercise-starter/src/boolean_logic.rs::{BooleanExpression, build_boolean_expression}.
  • Change: validate arity (two for AND/OR, one for NOT), physical types, and lengths before any row work, then build Boolean rows. build_boolean_expression selects the SQL NonStrict policy; BooleanExpression::new(operator, policy) exposes the strict variant for comparison.
  • Preserve: the row error and null behavior stay inside the checked-expression contract; the error representation is your readable choice.
  • Run: the focused and cumulative tests.
  • Passing means: evaluation reproduces the full truth table, strict short-circuits before the truth table, and wrong arity/type/length fail closed.

Required and extension work

Both policies, all three operators, the exact truth table, and the checked builder are required. Nested expression trees and short-circuit execution plans are extensions outside this course.

cargo test -p type-exercise-starter chapter_7 --locked
cargo test -p type-exercise-starter --lib --locked

Next: Chapter 8 erases typed expressions behind one object-safe boundary.

Work in progress. This course has not yet received the author’s final audit. Chapters, exercises, and commands may still change before the course is marked complete.

Chapter 8: Erase Typed Expressions at Runtime

The engine now has typed kernels, but a runtime function name cannot carry a Rust generic parameter. This chapter places the typed shells behind one object-safe Expression interface.

Prerequisites: Chapters 6-7, trait objects, and checked enum recovery.

By the end of this chapter, you will:

  • expose name, input types, output type, and evaluation through dyn Expression;
  • select builtin physical expressions from one catalog; and
  • preserve the typed evaluator’s arity, type, length, null, and scalar errors.
cargo x copy-test --chapter 8
cargo test -p type-exercise-starter chapter_8 --locked

The first run should fail on the object-safe expression boundary or physical catalog.

Keep erasure outside the row loop

The runtime path is:

physical name → Box<dyn Expression> → typed whole-batch kernel → typed row loop

The object erases one already-vectorized evaluator. Its stored batch-kernel pointer selects the typed implementation once; that implementation validates the batch, converts columns to typed views, and enters the row loop. It must not erase a scalar callback or match on ScalarRefImpl to select an operator for every row.

Checkpoint 1: make the batch contract safe to erase

  • Target: type-exercise-starter/src/expression.rs::{Expression, BinaryExpression, BinaryBatchKernel, ExpressionError}.
  • Change: keep name, arity, input_types, output_type, and evaluate free of associated types, and require Any + Send + Sync for checked recovery and sharing, so the trait is object-safe from this chapter on; BinaryExpression::new pairs runtime physical metadata with one whole-batch kernel pointer.
  • Preserve: metadata is borrowed or copied from the selected expression; runtime inputs stay borrowed.
  • Run: the Chapter 8 focused test.
  • Passing means: a builtin evaluates through Box<dyn Expression> with the same result as its typed adapter.

Wire the erased boundary and catalog into the starter crate root like the earlier chapters:

pub use expression::{
    BinaryBatchKernel, BinaryExpression, BUILTIN_EXPRESSION_NAMES, Expression,
    build_builtin_expression,
};

Checkpoint 2: erase the fixed-arity batch shell

  • Target: the Expression implementation for BatchExpression<N> in type-exercise-starter/src/operators.rs, plus the original BinaryExpression implementation in type-exercise-starter/src/expression.rs.
  • Change: publish physical metadata and call the selected whole-batch kernel. One declarative catalog row owns each built-in’s name, input and output physical types, kernel, and optional loop specialization; that same row list generates both the public name list and constructor lookup. The typed i32_add and string_concat kernels own their row loops; BinaryExpression never stores or invokes a scalar callback. Its erased boundary also checks that the returned array’s physical type matches the declared output type.
  • Preserve: arity is checked before indexing; type and length errors keep their original shape.
  • Run: focused and cumulative tests.
  • Passing means: erasure adds selection, not a second evaluator.

Checkpoint 3: build the physical catalog

  • Target: type-exercise-starter/src/expression.rs::{define_builtin_expressions, build_builtin_expression, BUILTIN_EXPRESSION_NAMES}.
  • Change: make registered names and constructors one source of truth.
  • Preserve: missing names return None; catalog metadata must match the actual expression.
  • Run: the Chapter 8 catalog and delegation tests.
  • Passing means: every listed physical builtin is constructible and no unlisted name succeeds.

Required and extension work

Checked runtime erasure and a complete physical catalog are required. The vectorized kernels from Chapters 4–6 keep the same batch behavior; this chapter changes how the engine selects them. Dynamic plugin loading and per-row erased dispatch are extensions outside this course.

cargo test -p type-exercise-starter chapter_8 --locked
cargo test -p type-exercise-starter --lib --locked

Next: Chapter 9 binds logical calls to one physical kernel.

Work in progress. This course has not yet received the author’s final audit. Chapters, exercises, and commands may still change before the course is marked complete.

Chapter 9: Bind and Coerce Logical Calls

Physical kernels exist, but a parsed call contains logical types and a name. The binder must choose one kernel, apply only approved widening promotion, and reject unsupported signatures before a batch runs.

Prerequisites: Chapters 2, 5, 7, and 8; HashMap; closures; logical versus physical types.

By the end of this chapter, you will:

  • bind functions of any arity through slice-based metadata;
  • keep logical metadata consistent with the chosen physical expression; and
  • support correct numeric comparisons, string comparisons, contains, and existing concat.
cargo x copy-test --chapter 9
cargo test -p type-exercise-starter chapter_9 --locked

The first run should fail on slice-based binding, comparison semantics, or contains.

Separate four kinds of conversion

  • Numeric promotion converts values to a planner-selected common type.
  • Erased downcast checks whether a runtime enum contains the requested physical family.
  • Trait-object downcast recovers one concrete expression type through Any.
  • Lifetime shortening reborrows a value for a shorter valid lifetime.

Only the first is logical coercion. Do not call all four “casts” or restore an obsolete GAT upcast helper.

Checkpoint 1: generalize the registry

  • Target: type-exercise-starter/src/binder.rs::{BindError, BoundExpression, FunctionRegistry::register, register_unary, register_binary, register_ternary, bind}.
  • Change: store logical inputs as a slice/boxed slice and check requested arity before a factory indexes it.
  • Preserve: unknown name, wrong arity, unsupported arguments, missing physical expression, and metadata mismatch remain distinct errors.
  • Run: the Chapter 9 focused test.
  • Passing means: neg, arithmetic, clamp, and custom slice factories share one planning boundary across unary, binary, and ternary arities.

BoundExpression::new maps logical inputs and output to physical types and compares them with the selected expression’s metadata. A valid bound expression records that proof once; evaluation then delegates without rebinding each batch.

Checkpoint 2: bind comparisons and strings

  • Target: type-exercise-starter/src/operators.rs::{ComparisonOperator} and binder factories registered by FunctionRegistry::with_builtins.
  • Change: support <, <=, >, >=, =, !=, contains, and concat for their approved logical signatures.
  • Preserve: names match behavior. Any ordered float comparison with NaN is false; equality is false and inequality is true. Null input produces null before comparison.
  • Run: focused and cumulative tests.
  • Passing means: equal operands distinguish strict/inclusive operators and NaN never panics.

String Char and Varchar both use physical String, but their logical metadata stays distinct. This course does not enforce Char width.

Wire the binder into the starter crate root like the earlier chapters:

mod binder;
pub use binder::{BindError, BoundExpression, FunctionRegistry};

Checkpoint 3: bind three-valued Boolean functions

  • Target: type-exercise-starter/src/binder.rs::bind_boolean and the builtin registry entries for boolean_and, boolean_or, and boolean_not.
  • Change: register the Day 7 expressions through the same slice registry: boolean_and and boolean_or take two Boolean inputs, boolean_not takes one; the bound output is Boolean.
  • Preserve: arity is checked before a factory indexes its slice; unsupported signatures are bind errors; evaluation keeps the SQL three-valued semantics from Day 7.
  • Run: the focused and cumulative tests.
  • Passing means: one-input boolean_not binds (never rejected by a two-arity signature) and bound Boolean evaluation matches the Day 7 truth table.

Checkpoint 4: keep promotion lossless

  • Target: type-exercise-starter/src/promotion.rs::promote_numeric and its callers in type-exercise-starter/src/binder.rs::{bind_arithmetic, bind_comparison}.
  • Change: apply the same approved common type in both paths and both operand orders.
  • Preserve: unsupported or precision-losing pairs are bind errors; no silent narrowing.
  • Run: the full Chapter 9 contract.
  • Passing means: logical output metadata agrees with the chosen physical output.

Required and extension work

Slice-based binding, lossless promotion, six comparisons, contains, and concat are required. Narrowing casts, parsing casts, SQL-complete coercion, Decimal arithmetic, and overload selection from untyped NULL are extensions that need separate semantics.

cargo test -p type-exercise-starter chapter_9 --locked
cargo test -p type-exercise-starter --lib --locked

Next: Chapter 10 specializes one representative dense loop.

Chapter 10: Specialize One Primitive Loop

Work in progress. This course has not yet received the author’s final audit. Chapters, exercises, and commands may still change before the course is marked complete.

Correct generic evaluation comes first. Now you can optimize one common case and prove that every other representation still follows the established path.

Prerequisites: Chapter 9 and basic benchmarking discipline.

By the end of this chapter, you will:

  • select an all-valid i32 loop once per batch;
  • preserve the general path for nulls, dictionaries, and other operators; and
  • compare loop shapes without moving validation into the measured row loop.
cargo x copy-test --chapter 10
cargo test -p type-exercise-starter chapter_10 --locked

The first run should fail on fast-path selection while the earlier general evaluator stays green.

Prove the fast-path preconditions

A checked ColumnViewImpl::try_non_null_array proves once that an array has no null rows and records Nullability::NonNull beside its PhysicalType. A constant value is already one non-null scalar repeated for the batch. These facts permit four dense binary loop shapes over the same primitive array representation:

  • array / array;
  • array / constant;
  • constant / array; and
  • constant / constant.

A dictionary, typed null, nullable array, type mismatch, arity mismatch, or length mismatch must return to the general contract or the same structured error.

Checkpoint 1: select once per batch

  • Target: type-exercise-starter/src/physical_type.rs::Nullability, type-exercise-starter/src/column.rs::{ColumnViewImpl::nullability, ColumnViewImpl::try_non_null_array}, and type-exercise-starter/src/expression.rs::{Expression::output_nullability, PrimitiveLoop, PrimitiveBinaryExpression::evaluate_with_loop, BinaryExpression::new_with_loop}.
  • Change: keep one primitive array representation, establish physical nullability at the column boundary, and choose the dense i32 path only after ordinary validation succeeds. The builtin catalog stores that selection as another whole-batch kernel; it does not reintroduce a scalar callback.
  • Preserve: output values, nulls, and errors are identical to evaluate.
  • Run: the Chapter 10 focused test.
  • Passing means: all four dense shapes report their selected loop and every fallback reports PrimitiveLoop::General.

Checkpoint 2: forward through binding

  • Target: type-exercise-starter/src/binder.rs::{BoundExpression::output_nullability, BoundExpression::evaluate_with_loop}.
  • Change: delegate nullability propagation and evaluation to the already-selected physical expression.
  • Preserve: logical selection does not choose a fast path; batch representation does.
  • Run: focused and cumulative tests.
  • Passing means: binding and non-primitive catalog entries remain unchanged.

Required and extension work

Representative i32 specialization and semantic fallbacks are required. Fast paths for every numeric family and operator are extensions. Do not duplicate the full evaluator to chase a benchmark.

cargo test -p type-exercise-starter chapter_10 --locked
cargo test -p type-exercise-starter --lib --locked

After the tests pass, you may run the maintained reference benchmark without reading its source:

cargo bench -p type-exercise --bench expression

It reports the four dense shapes and three fallbacks separately. Setup and dictionary validation stay outside the timed row loop. The measurements are machine-specific observations, not a completion gate.

Next: Chapter 11 builds a one-level List column.

Chapter 11: Build a One-Level List Column

Work in progress. This course has not yet received the author’s final audit. Chapters, exercises, and commands may still change before the course is marked complete.

List is not another primitive enum row. Each outer row points through offsets into one child array, and the outer row can be null independently of any child value.

Prerequisites: Chapters 2–3, checked erased arrays, and slice ranges.

By the end of this chapter, you will:

  • store one-level List values with explicit child type, offsets, and outer validity;
  • distinguish a null list, an empty list, and a list containing a null child; and
  • expose List arrays, constants, dictionaries, and typed nulls through checked views.
cargo x copy-test --chapter 11
cargo test -p type-exercise-starter chapter_11 --locked

The first run should fail on the missing List types, invariants, and column integration.

Keep the two null layers independent

For n outer rows:

validity.len() == n
offsets.len() == n + 1
offsets[0] == 0
offsets are monotone
offsets[n] == child.len()

A null outer row and an empty non-null row both repeat an offset. Their validity bits differ. A non-null row may span child values that include their own nulls.

Checkpoint 1: add typed List scalars

  • Target: type-exercise-starter/src/array/list_array.rs::{ListScalar, ListScalarRef, ListError} and List variants in type-exercise-starter/src/{data_type,physical_type,scalar}.rs.
  • Change: retain the child physical type even for empty and all-null values; make get, slice, and owned conversion checked.
  • Preserve: nested List child types return ListError::NestedList.
  • Run: the Chapter 11 focused test.
  • Passing means: borrowed and owned List values cannot lose or invent child types.

Checkpoint 2: construct valid outer arrays

  • Target: type-exercise-starter/src/array/list_array.rs::{ListArray, ListArrayBuilder, try_from_rows, try_from_raw_parts}.
  • Change: validate child family, offsets, validity, and null spans before returning an array.
  • Preserve: ListArray::len() is the outer row count, never the flattened child length; a failed row does not expose partial output.
  • Run: focused and cumulative tests.
  • Passing means: zero-row, all-null, empty-row, and mixed arrays retain exact invariants.

Checkpoint 3: integrate Column views

  • Target: type-exercise-starter/src/column.rs::{ListColumnView, ColumnViewImpl::try_as_list} and List erasure in type-exercise-starter/src/array.rs.
  • Change: support List array, constant, Indexed, and typed-null representations.
  • Preserve: Indexed validation and expected/actual type errors from Chapter 3.
  • Run: the full Chapter 11 contract.
  • Passing means: one-level List values reuse the existing representation boundary.

Required and extension work

One-level storage and List inputs are required. Nested Lists, List equality as a scalar builtin, list-producing functions, and arbitrary List casts are extensions. The public type descriptor can represent a nested shape, but construction must reject it until those contracts exist.

cargo test -p type-exercise-starter chapter_11 --locked
cargo test -p type-exercise-starter --lib --locked

Next: Chapter 12 strengthens Rust type boundaries.

Chapter 12: Strengthen Rust Type Boundaries

Work in progress. This course has not yet received the author’s final audit. Chapters, exercises, and commands may still change before the course is marked complete.

The engine’s runtime behavior is complete. This chapter makes its Rust ownership and sharing claims executable without changing expression results.

Prerequisites: Chapter 11, Any, trait objects, threads, and lifetime variance.

By the end of this chapter, you will:

  • return opaque borrowed array iterators;
  • recover concrete expressions through checked Any downcasts; and
  • prove Send + Sync, captured-state, and lifetime-shortening boundaries.
cargo x copy-test --chapter 12
cargo test -p type-exercise-starter chapter_12 --locked

The first run should fail on one or more iterator, trait-object, thread, or lifetime guarantees.

Checkpoint 1: keep iterator storage private

  • Target: type-exercise-starter/src/array.rs::Array::iter and type-exercise-starter/src/array/iterator.rs::ArrayIterator.
  • Change: return impl Iterator<Item = Option<Self::RefItem<'_>>> while preserving borrowed strings and nullable rows.
  • Preserve: callers do not name the concrete iterator type.
  • Run: the Chapter 12 focused test and doctests.
  • Passing means: integer and string iteration keep their Chapter 1 ownership behavior.

Checkpoint 2: recover a concrete expression safely

  • Target: type-exercise-starter/src/expression.rs::Expression: Any + Send + Sync.
  • Change: upcast a trait object to dyn Any and use downcast_ref for checked recovery.
  • Preserve: mismatches return None; never cast raw pointers.
  • Run: the focused test.
  • Passing means: erased objects can be inspected without weakening object safety.

Checkpoint 3: prove sharing and variance

  • Target: type-exercise-starter/src/binder.rs::FunctionRegistry::{register, register_unary, register_binary, register_ternary} and type-exercise-starter/src/column.rs::ColumnViewImpl<'a>.
  • Change: require captured factories to be Send + Sync + 'static and demonstrate shortening a valid column borrow.
  • Preserve: a borrow cannot be lengthened or escape its backing array.
  • Run: focused, cumulative, and compile-fail doctests.
  • Passing means: expressions and registries can be shared across worker threads while views remain tied to their data.

Required and extension work

Opaque iteration, checked recovery, thread-safety, and covariance are required. Unsafe downcasts, custom executors, and arbitrary lifetime conversion helpers are not.

cargo test -p type-exercise-starter chapter_12 --locked
cargo test -p type-exercise-starter --doc --locked
cargo test -p type-exercise-starter --lib --locked

Next: Chapter 13 adds a batch async boundary.

Chapter 13: Add a Batch Async Boundary

Work in progress. This course has not yet received the author’s final audit. Chapters, exercises, and commands may still change before the course is marked complete.

Some engines need a uniform asynchronous interface even when a local expression is immediately ready. The useful boundary is one future per batch, not one future per row.

Prerequisites: Chapter 12, Future, Pin, and return-position impl Trait.

By the end of this chapter, you will:

  • wrap static expression evaluation in a borrowed batch future;
  • erase that future behind an object-safe asynchronous interface; and
  • preserve every synchronous result and error without evaluating twice.
cargo x copy-test --chapter 13
cargo test -p type-exercise-starter chapter_13 --locked

The first run should fail on the missing static or erased batch future boundary.

Checkpoint 1: return one static future

  • Target: type-exercise-starter/src/expression.rs::evaluate_static.
  • Change: return impl Future<Output = Result<ArrayImpl, ExpressionError>> + Send + 'a that borrows the expression and input slice.
  • Preserve: the body delegates to the existing synchronous batch evaluation exactly once.
  • Run: the Chapter 13 focused test.
  • Passing means: static sync and async paths return identical arrays and errors.

Checkpoint 2: erase the future

  • Target: type-exercise-starter/src/expression.rs::{BatchFuture, AsyncExpression, AsyncExpressionAdapter}.
  • Change: add AsyncExpressionAdapter::new, then box and pin the borrowed batch future so it can be returned from a trait object.
  • Preserve: the future lifetime covers the expression, the view slice, and every borrowed backing array.
  • Run: the focused test.
  • Passing means: erased async evaluation matches static and synchronous evaluation.

Checkpoint 3: forward a bound plan

  • Target: type-exercise-starter/src/binder.rs::BoundExpression::evaluate_async.
  • Change: delegate to the already-selected expression without repeating logical binding.
  • Preserve: arity, type, length, null, and scalar errors keep the same variants and precedence.
  • Run: focused and cumulative tests.
  • Passing means: the planning boundary remains one-time and the batch kernel remains synchronous.

Required and extension work

One ready future per batch is required. I/O, timers, retries, background threads, cancellation protocols, custom runtimes, and per-row futures are outside this course.

cargo test -p type-exercise-starter chapter_13 --locked
cargo test -p type-exercise-starter --doc --locked
cargo test -p type-exercise-starter --lib --locked

You have now moved type selection, representation dispatch, validation, promotion, and runtime selection out of the row loop while keeping each failure boundary explicit.