Kernel Maps and Provider Selection

How CKE v8 decides which physical C kernel executes each logical operation — and why that decision lives in 346 machine-checked JSON maps instead of in Python conditionals.

Choose Your Path

The Three-Level Separation — and Why

One CKE runtime serves many model families × quantization formats × ISA variants × execution phases, all on CPUs. That product space is why v8 separates what math is required from which provider runs from how the op is physically executed:

How A New Model Family Earns Support

Fast family bring-up is evidence of reuse only when the change lands in the correct ownership layer. A model circuit may declare a new graph composition; a kernel map may describe a genuinely new provider contract; the C source may implement that contract. The shared resolver and code generator must not grow a model-name branch to connect them.

Measuring Bring-Up Novelty

The scaling rule above is measurable. The advisory script version/v8/scripts/report_model_novelty_v8.py answers one question: when a new model family is brought up, how much code changes outside its circuit, tensor map, genuinely new kernels, and evidence fixtures?

In git-range mode (--base SHA --head SHA) it classifies every changed file into ownership buckets — circuit JSON, model/tensor maps, kernel maps, kernel C source, core compiler, converters, tests/evidence, docs, other — and reports per-bucket file counts and line deltas. The core-compiler bucket (DSL lowering, code generators, memory planner) is THE metric: its target trend is zero. A bring-up that lands entirely in its circuit, maps, kernels, and evidence is one where the architecture absorbed the family for free. In circuit mode (--circuit NAME) it reports the operations a circuit uses, how many are shared with other circuits versus unique to it, the providers bound, and their tracked status. Missing metadata is reported as an explicit null with a "not tracked yet" note; the report never fabricates numbers.

The report is advisory only: it is not a CI gate, does not enforce dsl_policy.json caps, and never fails the tree. When a snapshot is written to version/v8/.cache/reports/model_novelty_latest.json, the architecture-contract dashboard surfaces it as a purely informational section.

Anatomy of a Kernel Map

version/v8/kernel_maps/memcpy.json is the canonical annotated example — the production map behind the residual_save op. Every field group has one job:

Selection metadata is schema-enforced

The selection block is required by version/v8/schemas/kernel_provider_selection.schema.json and carries exactly four fields:

The hard rules: an equal-priority tie between explicit production providers is a fault, not a coin flip. And the map — not the code generator — owns the call ABI: dims, typed params (here _memcpy_bytes of type size_t), and a versioned call_abi block naming the exact argument list (dst, src, size) with each argument's source port. The alias and arithmetic contract is part of the map too: constraints.notes states this is a byte copy, performs no arithmetic, and is undefined for overlapping writable views.

Parameter Taxonomy: Six Field Groups

Every field in a kernel map answers exactly one of six questions. Keeping the groups distinct is what lets the resolver treat them differently: compatibility fields filter, ranking fields order the survivors, and memory/scheduling fields never influence numerics at all.

Reading a Kernel Map: Field Reference

The anatomy figure above shows the shape of one map. This section is the member-by-member reference: what each field means, which component consumes it, and what happens when it is missing or wrong.

Worked Example 1: memcpy — the Baseline Read

memcpy.json is the minimal production map: one op, two ports, one computed parameter, no ISA variants, no numerical contract — a byte copy has no arithmetic to contract. Verbatim excerpt (… marks elided lines):

{
  "id": "memcpy",
  "op": "residual_save",
  "operation_interface": "residual_save.memcpy_copy.v1",
  "selection": {"status": "production", "priority": 100, "equivalence_group": "residual_save.memcpy_copy.v1", "phases": ["prefill", "decode"]},
  "variant": "fp32_copy",
  "quant": {"weight": null, "activation": "fp32", "output": "fp32"},
  "inputs":  [{"name": "src", "dtype": "fp32", "shape": ["N"], "layout": "contiguous", "access": "read",  "storage_class": "activation", …}],
  "outputs": [{"name": "dst", "dtype": "fp32", "shape": ["N"], "layout": "contiguous", "access": "write", "storage_class": "activation", …}],
  "dims": ["N"],
  "params": [{"name": "_memcpy_bytes", "type": "size_t", "desc": "Number of bytes to copy"}],
  "constraints": {"notes": "Byte copy implementing logical residual_save; performs no arithmetic. src and dst must not overlap …"},
  "impl": {"function": "memcpy", "sources": [], "variants": [{"name": "default", "requires": [], "compile_flags": []}]},
  "call_abi": {"version": 1, "params": [
    {"name": "dst",  "source": "output:dst",        "cast": "void*", …},
    {"name": "src",  "source": "activation:src",    "cast": "const void*", …},
    {"name": "size", "source": "dim:_memcpy_bytes"} ]},
  "tests": {}
}

How to read it. The resolver reaches this map when a circuit needs residual_save in prefill or decode: the selection block makes it eligible in both phases, quant matches FP32 activations, and the single default variant passes ISA filtering vacuously (requires: []). Codegen then emits exactly memcpy(dst, src, size) — no argument reconstruction. What it deliberately lacks: numerical_contract, numerical_capabilities, non-empty tests. The audit therefore classes it as selection-managed and interface-ready, not fully "hardened" — a legitimate state for a copy with one possible behavior.

Worked Example 2: gemm_nt_q8_0_q8_0 — the Rich One

The Q8 prefill GEMM shows every group memcpy omits: a named numerical contract, ISA-gated variants, map-owned ABI for seven arguments, and validation oracles.

{
  "id": "gemm_nt_q8_0_q8_0",
  "op": "gemm",
  "selection": {"status": "production", "priority": 200,
                "equivalence_group": "q8_0_weight_q8_0_input_llama_fp32_output",
                "phases": ["prefill"]},
  "numerical_contract": "q8_0_weight_q8_0_input_llama_fp32_output",
  "quant": {"weight": "q8_0", "activation": "q8_0", "output": "fp32"},
  "inputs": [
    {"name": "A", "dtype": "q8_0", "shape": ["M", "K"], "layout": "token_major_contiguous", …},
    {"name": "B", "dtype": "q8_0", "shape": ["N", "K"], "layout": "opaque_packed", …},
    {"name": "bias", "dtype": "fp32", "shape": ["N"], …, "optional": true, …} ],
  "outputs": [{"name": "C", "dtype": "fp32", "shape": ["M", "N"], "layout": "token_major_contiguous", …}],
  "dims": ["M", "N", "K"],
  "constraints": {"alignment": {"K": 32}, "notes": "Q8_0 requires K dimension multiple of 32 …"},
  "phase_selection": {"decode": {"op_family": "gemv", "default_kernel": "gemv_q8_0_q8_0",
                                 "notes": "Single-token decode retains the certified GEMV path."}, …},
  "impl": {"function": "gemm_nt_q8_0_q8_0", "sources": ["src/kernels/gemm_batch_int8.c", …],
    "variants": [
      {"name": "avx2_m2n4", "function": "gemm_nt_q8_0_q8_0_m2n4", "priority": 240,
       "requires": ["avx2", "fma"], "shape_constraints": {"M_min": 2, "K_multiple": 32},
       "numerical_contract": "q8_0_weight_q8_0_input_llama_fp32_output"},
      {"name": "avx2",     "requires": ["avx2", "fma"],              "compile_flags": ["-mavx2", "-mfma"]},
      {"name": "avx_vnni", "requires": ["avx2", "avx_vnni", "fma"],  "compile_flags": ["-mavx2", "-mavxvnni", "-mfma"]} ]},
  "reference": {"function": "vec_dot_q8_0_q8_0_ref", "kind": "scalar_contract_oracle", …},
  "production": {"function": "gemm_nt_q8_0_q8_0", …, "reference_comparison": {"requirement": "bit_exact", …}},
  "call_abi": {"version": 1, "params": [
    {"name": "A", "source": "activation:a", "cast": "const void*"},
    {"name": "B", "source": "weight:_first_weight"},
    {"name": "bias", "source": "weight_f:_bias"},
    {"name": "C", "source": "output:c", "cast": "float*"},
    {"name": "M", "source": "dim:_m"}, {"name": "N", "source": "dim:_output_dim"},
    {"name": "K", "source": "dim:_input_dim"} ]},
  …
}

How the resolver uses it. Three facts interact:

• The equivalence-group name encodes the arithmetic. equivalence_group and numerical_contract are the same string — q8_0_weight_q8_0_input_llama_fp32_output: Q8_0 weights times Q8_0 activations producing FP32 output under llama.cpp-compatible semantics. Because priority ranks only within a group, no priority value can ever promote a provider with different arithmetic over this one; cross-group ties are a hard fault ("priority cannot choose between different equivalence groups"), not a silent substitution.
• Phases filter before priority ranks. phases: ["prefill"] means a decode op never sees this provider at all — phase_selection.decode documents that single-token decode keeps the certified gemv_q8_0_q8_0 GEMV path instead. Priority 200 is only ever compared against other prefill Q8 GEMM providers in the same group.
• Variants carry ISA eligibility. Each variant's requires list gates it by CPU features: on a machine without AVX-VNNI the avx_vnni variant is ineligible and filtered out. The avx2_m2n4 tile variant adds its own priority: 240 and shape constraints (M ≥ 2, K multiple of 32) — variant-level preference, still inside the same numerical contract, which it repeats verbatim.

Honesty note: this map has no operation_interface and no numerical_capabilities block, so the audit classes it as selection-managed and legacy-contract-shaped — a production provider carrying the older contract keys, exactly the kind of map the migration ratchet tracks.

Worked Example 3: kv_cache_store_bf16 — the Persistent-State One

Cache stores look like ordinary ops but their outputs are state, not ephemeral tensors. Verbatim excerpt:

{
  "id": "kv_cache_store_bf16",
  "op": "kv_cache_store",
  "operation_interface": "kv_cache_store.bf16.head_major.v1",
  "selection": {"status": "production", "priority": 100,
                "equivalence_group": "kv_cache_store.bf16.head_major.v1", "phases": ["decode"]},
  "variant": "bf16_cache",
  "quant": {"activation": "fp32", "output": "bf16"},
  "inputs": [
    {"name": "k", "dtype": "fp32", "shape": ["KV", "D"], "layout": "head_major_contiguous",
     "access": "read", "storage_class": "activation", "consumption": "required", …},
    {"name": "v", …} ],
  "outputs": [
    {"name": "kv_cache_k", "dtype": "bf16", "shape": ["KV", "S_max", "D"], "layout": "head_major_contiguous",
     "access": "read_write", "storage_class": "state", "consumption": "required",
     "desc": "Packed BF16 K cache slice at current pos. Physical capacity S_max rows; valid-token count is tracked separately by the caller (append index); …"},
    {"name": "kv_cache_v", …} ],
  "dims": ["num_kv_heads", "head_dim", "num_layers", "max_seq_len", "KV", "S_max", "D"],
  "constraints": {"notes": "Scheduling may round the extent to physical capacity S_max; kernels must never read or write KV rows beyond the valid-token count declared by the append index."},
  "impl": {"function": "kv_cache_store_bf16", …, "sources": ["src/kernels/kv_cache_kernels.c"]},
  "call_abi": {"version": 1, "params": [
    {"name": "kv_cache_k", "source": "runtime:kv_cache_k_layer_u16", "cast": "uint16_t*", …},
    {"name": "kv_cache_v", "source": "runtime:kv_cache_v_layer_u16", "cast": "uint16_t*", …},
    {"name": "k", "source": "activation:k", "cast": "const float*", …},
    {"name": "v", "source": "activation:v", "cast": "const float*", …},
    {"name": "layer", "source": "const:0"}, {"name": "pos", "source": "runtime:pos"},
    {"name": "num_kv_heads", "source": "dim:num_kv_heads"},
    {"name": "head_dim", "source": "dim:head_dim"}, {"name": "max_seq_len", "source": "dim:max_seq_len"} ]},
  …
}

How to read it.

• State, not output. The kv_cache_k/kv_cache_v ports are storage_class: "state" with access: "read_write" — the kernel appends into a persistent cache slice rather than producing a fresh tensor. That classification is what lets the memory planner treat the cache as long-lived state instead of a per-op allocation.
• Capacity vs validity. The shape ["KV", "S_max", "D"] is physical capacity: S_max rows exist, but the valid-token count is tracked separately by the caller through the append index (pos in the call ABI, sourced from runtime:pos). The constraint is explicit: append writes exactly the rows at the append index and never reads beyond valid rows, even if scheduling rounds extents up to S_max.
• Storage dtype and layout are part of the contract. The interface and equivalence group — kv_cache_store.bf16.head_major.v1 — bake in BF16 storage and head-major layout. An FP16 or FP32 cache store is a different group, so the resolver can never swap storage formats as a "priority" decision; quant says FP32 in, BF16 stored.
• Decode-only is deliberate. phases: ["decode"] keeps this provider out of prefill entirely; prefill KV writes use their own providers. Like memcpy it carries operation_interface but no numerical_capabilities, so the audit classes it as selection-managed and legacy-interface-ready.

With the format and three real maps in hand, the resolution algorithm below is mechanical: everything the resolver knows about a provider comes from these fields, and everything it rejects leaves a recorded reason.

The Resolution Algorithm

Provider selection lives in _provider_selection_metadata, _rank_provider_matches, and find_kernel in version/v8/scripts/build_ir_v8.py. The pipeline:

1. Validate selection metadata. Every explicit provider must declare an equivalence_group; a malformed selection block (bad status, non-integer priority, empty group) is a HARD KERNEL SELECTION FAULT. Providers with no selection block at all are implicit legacy: they stay eligible and rank below production. Explicit candidate, diagnostic, and deprecated providers are rejected here with a recorded reason.
2. Filter by compatibility. Contract identity, phase, dtype, shape, layout, ISA, and alias safety — any mismatch rejects the provider with a recorded reason.
3. Rank the survivors. The ranking tuple is direction (inference vs backward) → activation preference → lifecycle rank (production first) → priority (higher wins, within the group).
4. Fail closed on ambiguity. A tie among explicit production providers raises "equal-priority production providers are ambiguous" when they share a group, or "priority cannot choose between different equivalence groups" when they do not. Zero compatible providers is also a compile-time failure — the build never silently substitutes different numerics.

A Real Resolver Trace

Evidence, not assertion. This is not an illustrative mock-up: the X-Ray fixture version/v8/tests/fixtures/xray/provider_selection_trace.json records a real selection trace from the live resolver, and tests/test_v8_shared_provider_migration.py (test_filtering_precedes_priority_ranking) regenerates the trace and asserts it matches the fixture exactly — including that every rejection is recorded before any priority-ranking entry. In the trace, a candidate provider with priority 900 is rejected for status_not_production, a priority-800 production provider for phase_mismatch, and a priority-700 production provider for weight_dtype_mismatch — all before the compatible priority-100 production provider is selected at rank 0. Higher priority loses to incompatibility every time, because compatibility filtering happens before priority ranking. See X-Ray: Evidence and Divergence Attribution for the full evidence pipeline this trace belongs to.

Case Study: residual_save → memcpy

This is the headline example of DSL logic that legitimately remains — and of what hardened selection replaced. The two concerns are different, and the code now keeps them in different places:

• Insertion is scheduling, and stays in the DSL. should_insert_residual_save (build_ir_v8.py:3743) fires when the current op is a pre-norm, the next op starts a branch (attention or MLP), and the previous op is not already a residual save. The op is auto-inserted and stamped "kernel": "memcpy" at build_ir_v8.py:7328; the _memcpy_bytes buffer size (embed_dim × seq_len × 4) is computed during lowering at build_ir_v8.py:8610. Deciding that a residual must be saved is graph scheduling — it belongs here.
• Selection is hardened. The memcpy map is selection-managed with declared copy semantics — a byte copy, not residual arithmetic mislabeled under a copy's name. It wins or loses through the same mechanical resolver as every other hardened provider.
• The debt is named, not hidden. map_op_to_kernel (build_ir_v8.py:6712) still carries template-override branches: rope_qk via _resolve_rope_qk_kernel, rope_q, mrope_qk, position_embeddings via _resolve_position_embeddings_kernel, kv_cache_store_shared_q, and the assistant_layer_scale override. Separately, _make_decode_kv_store_op (build_ir_v8.py:8782) fabricates decode KV store ops. This is migration debt being actively burned down — tracked by the ratchet below.

Where the Weights Side Fits

Provider selection decides which kernel runs; the weights pipeline decides which bytes it reads. Converted BUMP weights carry a metadata sidecar describing tensor names, dtypes, and layouts (see GGUF to Bump and ADR 0006). Circuits and templates reference weight tensors, and the map's quant block and weights ports must match what the sidecar describes — a map that expects Q4_K weights will not pass compatibility filtering against an FP32 tensor. The conversion format itself is documented on the BUMP page; it is not re-explained here.

Memory Planning: From Port Shapes to a Checked Arena

Kernel maps declare memory symbolically; lowering turns the declarations into one arena with hard bounds. Nothing allocates outside this flow — a kernel that needs workspace names a scratch port with a sizing formula, and a kernel that keeps state marks the port storage_class: "state". The pipeline, all in build_ir_v8.py lowering with the arena emitted by codegen_core_v8.py:

The fail-closed property is the point: a map whose scratch formula overflows the arena, or whose alignment is not a power of two, fails the build with a named hard fault — the runtime never discovers the overflow by corruption. version/v8/scripts/certify_model_memory_plans_v8.py certifies per-model memory plans against a checked-in baseline without compiling or executing model code.

Contributor Recipe: Map JSON to Generated C

The full lane for adding or changing a provider, with the scripts that enforce each step:

1. Author the map. Create or edit version/v8/kernel_maps/<id>.json: id, op, a selection block (status, priority, equivalence_group, phases), typed ports with dtype/shape/layout, and quant.
2. Pass schema validation. tests/test_v8_provider_selection.py validates every map's selection block against version/v8/schemas/kernel_provider_selection.schema.json, and tests/test_v8_kernel_call_abi.py validates each declared call_abi against kernel_call_abi.schema.json; _validate_kernel_call_abi in build_ir_v8.py additionally hard-faults a malformed ABI at build time.
3. Regenerate the registry. version/v8/scripts/ck_run_v8.py rebuilds version/v8/kernel_maps/KERNEL_REGISTRY.json — never hand-edit it.
4. Run the audit. python3 version/v8/scripts/audit_kernel_map_interfaces_v8.py --check — the migration ratchet must not regress.
5. Run the unit gates. tests/test_v8_provider_selection.py, tests/test_v8_kernel_call_abi.py, tests/test_v8_shared_provider_migration.py.
6. Prove leaf parity. version/v8/scripts/parity_test_v8.py compares intermediate activations against llama.cpp dumps within declared tolerances.
7. Inspect the resolver trace. Confirm your provider is selected (or rejected) for the reason you expect — every decision is recorded in the selection trace; the fixture above shows the format.
8. Validate end to end. Codegen emits exactly the map-owned call ABI; version/v8/scripts/run_regression_v8.py (family regression lane) and stitched_parity_v8.py exercise the generated C. Promote candidate → production only with measured evidence.

Bringing up an entire model family on top of these maps — inventory, import adapter, circuit, generated-C inspection, oracle certification, standalone deployment — is walked end to end in the model bring-up guide.

Production-Readiness Checklist

• Valid selection block with a non-empty equivalence_group — a missing group is a HARD KERNEL SELECTION FAULT.
• status: production only with parity evidence attached; anything unproven stays candidate.
• call_abi version 1 whose ports cover every declared input/weight/output port — the audit cross-validates this.
• Every ISA variant gated by requires with its own compile_flags, and a portable fallback variant or provider reachable.
• Scratch sized by formula with power-of-two alignment and a declared lifetime — no hidden allocation.
• A named numerical contract and reference oracle wherever the kernel performs arithmetic.
• audit_kernel_map_interfaces_v8.py --check passes with no ratchet regression.

Anti-Patterns the System Rejects

Evolution Timeline

An honest, PR-dated history. The direction is monotonic: physics moves into the maps, heuristics leave the DSL.

• Legacy: kernel_bindings.json and a function-name ABI fallback — the name was the contract.
• PR #302: hardened provider selection (lifecycle, priority, equivalence groups) and the operation-interface format introduced.
• PR #305: shared RoPE, residual-copy, and KV-cache providers migrated to selection-managed maps.
• PR #318 / #320: layout provider selection and direct-layout attention providers — physical layout joins the map-owned contract.
• Future: resolver cleanup (rebind circuits to hardened metadata, delete the legacy branches), then attention and quantized GEMM migrations. The end state is an agnostic DSL: circuits state contracts, maps own everything physical, the resolver contains no model knowledge at all.

Migration Scoreboard

version/v8/contracts/kernel_interface_migration_baseline.json is a monotonic ratchet checked by version/v8/scripts/audit_kernel_map_interfaces_v8.py --check: floors may only rise, ceilings may only fall. The current audit values against the checked-in ratchet bounds:

Generated from version/v8/kernel_maps/KERNEL_REGISTRY.json (346 maps) + version/v8/scripts/audit_kernel_map_interfaces_v8.py at commit 14edb65a5 (2026-09). Intended direction: these metrics should be regenerated during the docs build rather than hand-maintained — that pipeline is not implemented yet, so until it is, this page is updated manually against audit output.

Is This Over-Engineered?

Fair question — the project owner has asked it directly. 346 JSON maps, a schema for four fields, a ratchet file, and an audit script can look like architecture for its own sake. The honest answer has three parts.

Diagram − + ×

Related: v8 Numerical Contracts, System Architecture, Composite Circuits (components + stitch), Code Generation, GGUF to Bump, and Architecture Links.