v8 Numerical Contracts

v8 now treats numerical behavior as compiler input. A circuit does not merely say “run attention.” It can require the query precision, K/V storage and compute precision, score accumulator, online-softmax state, value accumulator, partition threshold, partial storage, and merge order. Kernel maps state which complete contracts their implementations support. The DSL resolves one provider or fails before C is emitted.

Why the Architecture Needed Hardening

The earlier v8 path had strong kernel maps, generated C, and E2E tests, but some numerical semantics were inferred inside runtime code. A function name established one part of the contract, an environment flag selected another part, tensor metadata implied a third part, and a sequence-length threshold could silently change the reduction algorithm. Leaf kernels could all pass while the stitched model selected the wrong public route.

The Attention Bug That Exposed It

Three full-attention tests had long-standing errors of roughly 5.68e-4 to 1.06e-3. The public full-attention contract expected FP32 Q with K/V rounded through FP16, matching the GGML-style vision path. The optimized AVX2 route consumed raw FP32 K/V. Pre-rounding only K/V reduced the error to 4.77e-7, proving this was a contract mismatch rather than a tolerance problem.

Compiler Inputs

Complete Reduction Semantics

A reduction ID is not a loose dtype such as fp16 or strict. Those names are rejected as ambiguous. A registered contract includes:

• query and key compute precision;
• score accumulator and softmax-statistics precision;
• value compute and accumulator precision;
• partial storage and merge order;
• partition kind, threshold, and chunk policy.

{
  "id": "f16_online_fp32_merge",
  "q_compute": "fp16_rounded",
  "k_compute": "fp16",
  "score_accumulator": "fp32",
  "softmax_statistics": "fp32",
  "value_accumulator": "fp16_rounded_each_update",
  "partial_storage": "fp16",
  "partial_merge": "fp32_chunk_order",
  "partition": {
    "kind": "kv_chunks_by_workers",
    "threshold": 512
  }
}

Deterministic Resolution

1. The builder hydrates the current built-in circuit into cached manifests, so old converted models receive current graph and contract defaults.
2. The resolver reads each active required_contracts entry for the requested phase.
3. It matches op, phase, tensor dtype, layout, and complete reduction ID against executable kernel maps.
4. Zero providers fail as unsupported. Multiple providers fail as ambiguous.
5. The resolver selects the provider before GraphIR construction. Contract-bearing operations do not consult legacy circuit overrides or heuristic dispatch.
6. GraphIR records requirements and the resolved kernel/contract IDs. LoweredIR and call-ready IR carry that decision and hard-fail if a later stage changes it.
7. Production promotion additionally requires validated circuit, contract definition, implementation route, stitched parity, and E2E evidence.

Compatibility and Migration

The repository directory is now version/v8/circuits. Embedded BUMP metadata continues to use the historical template key, and chat templates keep their established name; changing serialized formats is not required to correct compiler architecture.

The authoritative migration preserves executable behavior while making the decision visible. Contract metadata is added to GraphIR, LoweredIR, and call-ready IR, while kernel IDs, memory layout, call arguments, and generated C remain unchanged. Gemma3, Qwen2, Qwen3, Qwen3.5, Nanbeige, and Qwen3-VL attention routes resolve without legacy selection.

Threading Is an Execution Contract

Kernel maps also declare ISA dispatch, threading runtime, possible work partitions, dispatch mechanisms, and whether scheduling can alter reduction order. Q4/Q6 GEMM and GEMV maps therefore expose serial, row, column, or output-tile policies instead of hiding threadpool behavior in generated wrappers. GraphIR records this as resolved_execution, and LoweredIR plus call-ready IR hard-fail if the kernel ID changes. When partitioning changes arithmetic order, the numerical contract must define the corresponding reduction and merge behavior.

Scalar, Production, and Threadpool Evidence

Quantized linear maps bind three exact functions: the scalar contract oracle, the public production function, and the threadpool dispatch function. The compiler validates these names and carries them into IR; it does not rank or guess functions from ISA, dimensions, or benchmark results. Dimensions and thread counts remain ordinary function inputs.

1. Compare an external backend or independent mathematical oracle with the scalar contract implementation.
2. Compare the optimized public function with that scalar implementation.
3. Compare threadpool dispatch with the same scalar implementation at representative thread counts and shapes.

Independent output rows or tiles must preserve each output's reduction order. Split-K is a different numerical contract and must declare its partial accumulator, partition, and merge order. Unavailable llama.cpp or PyTorch evidence is recorded as unresolved, never counted as a pass.

Validation Commands

make test-numerical-contracts is an executable gate, not only a schema check. It builds the engine and runs contract resolution, full attention numerics, and FP16 split-KV reduction cases. The focused commands below remain useful when attributing a failure.

make test-numerical-contracts
make test-threadpool-parity
PYTHONPATH=unittest .venv/bin/python unittest/test_attention_full.py -v
V8_QWEN3VL_ENCODER_PARITY_LLAMA_CPP_ROOT=./llama.cpp \
  make test-attention-f16-split-kv
make test-v8-qwen3vl
make v8-regression-fast

What This Prevents

• A correct leaf kernel hidden behind the wrong production wrapper.
• Prefill and decode silently using different accumulation contracts.
• KV=511 and KV=512 changing mathematics without a declared threshold.
• Thread count changing reduction structure without a declared partition policy.
• A model-specific Python conditional becoming permanent compiler architecture.
• A fast path being enabled because it is faster while long-token parity is worse.

Related: System Architecture, IR Pipeline, v8 Vision Encoder, and Kernel Tuning Methodology.