Bring a Model to CKE — from Math to Standalone C

This is the contributor tutorial for adding a model to C-Kernel-Engine v8: import adapter, operation inventory, numerical contracts, kernel maps, circuit, generated C, oracle-backed certification, and standalone native deployment. Evidence policy: commands labeled executed were run at the stated revision; commands labeled source-verified were read from source and --help output but not run for this page. The machine-readable record of which is which is in the evidence JSON.

The finish line

A model is “brought up” when this pipeline runs end to end and every stage's artifact survives inspection:

Pinned model artifact
  → Import adapter → weights.bump (BUMPWGT5) + manifest + config + tokenizer
  → Circuit (operation semantics, contracts)
  → Compiler: provider resolution + memory planning
  → Generated C (model_v8.c)
  → Standalone native execution (libmodel.so + ck-cli-v8)
  → Oracle-backed certification (X-Ray evidence chain)

Python tooling builds, inspects, and certifies the artifact. Python must not secretly be the model: if inference only works because a Python loop reconstructs layers and calls C kernels, you have a reference bring-up, not a CKE deployment. The distinction is architectural, and it is the subject of the first section.

Two tracks cover almost everything you will do:

• Track A — add a model on existing operations. Most new models are new arrangements of operations CKE already has. Your deliverables are the import adapter, the circuit, contract declarations, and evidence — no new C.
• Track B — implement and register one new scalar provider. Only when an operation or a numerical contract is genuinely missing do you write a scalar kernel, its kernel map, and its tests. Exactly one operation at a time.

1. Who owns what — and the prohibited shortcut

CKE separates model knowledge into components with explicit ownership. When something is wrong, the component boundary tells you where the fix is allowed to live:

2. Choose the example and inventory operations first

The tutorial spine is Qwen2 0.5B Instruct (hf://Qwen/Qwen2-0.5B-Instruct-GGUF/qwen2-0_5b-instruct-q4_k_m.gguf) through the current compiler: 24 layers, embed dim 896, 14 query heads with 2 KV heads (GQA), head dim 64, RoPE theta 1000000, SwiGLU MLP with intermediate size 4864, RMSNorm, tied embeddings, per-tensor mixed quantization (Q5_0 / Q8_0 / Q4_K / Q6_K / F32 across 290 tensors). It exercises the real path: circuit version/v8/circuits/qwen2.json, map-owned call ABIs, the region memory planner, generated C, and the native CLI.

For a new model, your first deliverable is an operation inventory, not a .c file. Identify from the model card, config, and tensor list:

• Architecture / layer sequence — e.g. norm → qkv → rope → attn → out → residual → norm → gate/up → activation → down → residual.
• Weight formats — per-tensor quant types, fused/packed projections, tied weights, biases present or absent.
• Attention / cache structure — GQA vs MHA vs MLA, KV layout, sliding window, cache dtype.
• Position encoding — RoPE variant, theta, scaling type, rotary dim.
• Normalization / activations — RMSNorm vs LayerNorm, epsilon, SwiGLU vs GeGLU, and the exact arithmetic contract (see section 3).
• Stateful ops — KV cache, recurrent state, conv state; who owns the buffer across calls.
• Modality preprocessing / bridges — audio frontend, vision encoder, projector; where the text boundary is.

Then check reuse before writing anything. The advisory tool answers exactly this question executed:

python3 version/v8/scripts/report_model_novelty_v8.py --circuit qwen2

# Model Novelty Report (ADVISORY)
- Operations used: 13
- Shared with other circuits: 13
- Unique to this circuit: 0
- Circuits compared: 31

Qwen2 needs zero unique operations: every one is shared with at least one other circuit. That is the expected shape of a Track A bring-up. A new model should not imply new kernels; a scalar implementation is written only when an operation or a numerical contract is genuinely missing.

The inventory table is the deliverable you paste into your PR description. Fill it from the GGUF/safetensors tensor list and the circuit you intend to reuse or extend:

The Parakeet bring-up page keeps a 29-row historical inventory with four honest dispositions (reusable, candidate reuse, contract extension, missing provider/runtime/composition) — use the same vocabulary. Its machine-readable form is version/v8/contracts/parakeet_tdt_0_6b_v3_inventory.json.

3. Write the mathematics and the numerical contract

Before any C exists, write the contract for the new operation. For a reduction (RMSNorm's mean of squares, or a dot product), the contract must answer all of these — an equation alone is not a contract:

• Equation — e.g. y[i] = x[i] * gamma[i] / sqrt(mean(x^2) + eps).
• I/O dims and axis meanings — token-major [T, D]? channel-major? which axis is reduced, which are batch.
• Storage vs computation vs accumulation dtype — e.g. storage Q5_0, computation fp32, accumulation fp32 in a stated order.
• Reduction axis and order — sequential over D? blocked by 32 (quant block)? tree merge across threads?
• Intermediate rounding points — where a value is stored to a narrower type mid-computation (e.g. K/V rounded through fp16 once, then reused).
• Epsilon / scaling / masking / boundary behavior — eps inside or outside the sqrt; attention scale 1/sqrt(D) applied to Q or to scores; tail lengths; window edges.
• State read/write semantics — KV cache append position, in-place normalization allowed or not.
• Aliasing rules — may output == input? (RMSNorm in Qwen2 does exactly this: A_EMBEDDED_INPUT is both.)
• Error behavior — what invalid dims, null pointers, or non-finite inputs do (reject, propagate, or clamp — pick one and write it down).

Four questions are routinely confused. Keep them separate:

1. Is the math correct? (the equation describes the model)
2. Does the scalar C implement the specified finite-precision behavior? (dtype, order, rounding points)
3. Does the optimized C preserve the contract? (vectorized/threaded variants against the scalar oracle)
4. Does the external oracle use the same contract? (PyTorch's LayerNorm and llama.cpp's rms_norm are different finite-precision programs)

Never promise bitwise equality across different reduction orders. Floating-point addition is not associative. A cancellation example you can verify in any REPL: with fp32, (1e8 + 1.0 - 1e8) evaluates to 0.0 but (1e8 - 1e8 + 1.0) evaluates to 1.0. A dot product that accumulates large positive and negative terms in a different order — sequential vs blocked-by-32 vs thread-partials-then-merge — lands on different fp32 values. The contract fixes one order per provider; certification compares within a stated tolerance, or bitwise only when both sides provably share the order.

4. Implement the scalar kernel (Track B only)

Add the kernel to the existing source group for its domain — src/kernels/audio_kernels.c for audio ops, rmsnorm_kernels.c for norms, and so on — not one file per model. A real provider with CKE's conventions (src/kernels/audio_kernels.c, used by the Parakeet path):

/* Public declaration goes in the matching header; the kernel map names this exact symbol. */
int audio_scaled_residual_add_f32(
    const float *residual,   /* input: read */
    const float *branch,     /* input: read */
    float scale,             /* parameter */
    float *output,           /* output: written; may alias residual */
    size_t elements)         /* dimension */
{
    if (residual == NULL || branch == NULL || output == NULL) {
        return -1;           /* status convention: -1 null pointer */
    }
    if (elements == 0 || !isfinite(scale)) {
        return -2;           /* -2 invalid argument */
    }
    for (size_t index = 0; index < elements; ++index) {
        /* Preserve the pinned elementwise multiply then add cast boundary. */
        volatile float scaled = branch[index] * scale;
        output[index] = residual[index] + scaled;
    }
    return 0;                /* 0 success */
}

Every convention above is load-bearing: public declaration, pointer/dim/parameter arguments, input validation with a documented status code, scalar computation that visibly implements the contract (the volatile pins the multiply-then-add rounding point so the compiler cannot fuse it into an FMA), in-place support where the contract allows it, and no allocation.

The allocation rule. The kernel never allocates workspace. The map declares requirements, the compiler resolves sizes and lifetimes, the runtime owns the memory, and the kernel uses the supplied region. The conceptual signature is status = provider(input, output, dimensions, workspace, workspace_bytes); CKE's actual convention passes scratch as typed pointer + capacity, e.g. the verified f16kv attention provider:

void attention_forward_causal_head_major_gqa_flash_strided_f16kv_workspace(
    const float *q, const float *k, const float *v, float *output,
    int num_heads, int num_kv_heads, int num_tokens, int head_dim,
    int aligned_head_dim, int kv_stride_tokens,
    float *rounded_kv, size_t rounded_kv_bytes);   /* caller-owned scratch + capacity */

whose map declares scratch: [{name: rounded_kv, dtype: fp32, shape: [2, KV, T, D], size_resolution: required, lifetime: kernel_call}]. A dynamic-size example: a T×D fp32 op needing 2×T×D fp32 scratch at T=512, D=5120 moves 10 MiB in, 10 MiB out, 20 MiB scratch — the map's shape expression is how the planner computes those bytes, and checked size arithmetic (overflow-checked products, aligned sizes) is how the runtime rejects absurd dimensions before any pointer exists. Alignment, lifetime reuse across layers, per-worker vs shared scratch, and why concurrent calls cannot share mutable scratch are covered in depth on the allocation ownership page — the short version: swapping malloc for an unbounded alloca is not a solution (stack is finite and the planner cannot see it), and tests enforce the rule: tests/test_v8_parakeet_native.py asserts malloc/calloc/realloc/free appear nowhere in the new provider bodies.

5. Test the scalar provider before optimizing

Repo convention: each kernel ships a Python ctypes parity test with a hard max-diff gate (see testing and the unittest/ suite wired into make test). For a new scalar provider, cover:

• Independently computed expected values — not the C code transliterated into Python; a different expression of the same contract.
• Every output and state update checked — including in-place buffers and caches, not just the primary output.
• Small hand-verifiable cases — 3–8 elements where you can compute the answer on paper.
• Fixed-seed randomized cases — reproducible failures.
• Nonuniform values that expose axis swaps — if x[i][j] and x[j][i] give the same test result, your test cannot catch a transposed stride.
• Boundary dims — 0/1 elements, non-multiple-of-vector-width tails, the largest supported dim.
• Supported aliasing — if in-place is in the contract, test output == input explicitly.
• Invalid dims + insufficient workspace — assert the documented status codes (-1/-2/...), and a too-small workspace_bytes rejection.
• Canary out-of-bounds checks where practical — guard bytes before/after the output region.
• Reduction-specific — cancellation, mixed magnitudes, tail lengths, near-zero variance, mask/window boundaries.

An fp64 reference (or Kahan summation) is useful to prove the equation is implemented (question 1 and 2). It misrepresents the production oracle when the production contract is itself fp32-with-a-fixed-order: then the fp64 answer is not the expected value, and an fp64-referenced test will flag correct contract-preserving code. Use fp64 to validate the scalar contract; use the contract's own arithmetic to validate providers against each other.

The debugging loop looks like this — one deliberately wrong stride, one failing test, one fix:

- output[(size_t)channel * frames + frame] = input[(size_t)frame * channels + channel];  /* strides swapped */
$ python3 -m unittest unittest.test_audio_transpose
FAIL: test_nonuniform_layout - max diff 3.75 at (t=1, c=0): expected input[1][0], got input[0][1]
+ output[(size_t)frame * channels + channel] = input[(size_t)channel * frames + frame];  /* channel-major in, token-major out */
$ python3 -m unittest unittest.test_audio_transpose
OK - 14 cases incl. axis-swap sentinel values, frame=1/channel=1 boundaries, in-place rejection

(Illustrative shape of the loop; the transpose provider is real: audio_transpose_channel_to_token_f32 in src/kernels/audio_kernels.c.)

6. Define the operation interface and the kernel map

The kernel map is the machine-checked contract between the compiler and your C function. Two real annotated examples:

Minimal map — version/v8/kernel_maps/audio_scaled_residual_add_f32.json (abridged to the fields that matter):

{
  "id": "audio_scaled_residual_add_f32",          // identity: file name, registry key
  "op": "audio_scaled_residual_add",             // logical op the circuit names
  "operation_interface": "audio.residual_add.scaled.fp32.v1",  // versioned interface contract
  "selection": { "status": "candidate", "priority": 100,
    "equivalence_group": "audio_scaled_residual_add_separate_fp32",
    "phases": ["prefill", "decode"] },              // when the resolver may pick this
  "quant": { "weight": "none", "activation": "fp32", "output": "fp32" },
  "inputs":  [ { "name": "residual", "dtype": "fp32", "shape": ["N"],
                  "layout": "contiguous", "access": "read", "storage_class": "activation" },
                { "name": "branch",   "dtype": "fp32", "shape": ["N"], ... } ],
  "weights": [],
  "outputs": [ { "name": "output", "dtype": "fp32", "access": "write", ... } ],
  "scratch": [],                                  // this op needs no workspace
  "dims":   ["elements"],
  "params": [ { "name": "scale", "dtype": "fp32" } ],
  "impl": { "function": "audio_scaled_residual_add_f32",     // exact C symbol
    "c_declaration": "int audio_scaled_residual_add_f32(const float *residual, ...);",
    "sources": ["src/kernels/audio_kernels.c"] },
  "numerical_capabilities": [ { "contract_id": "audio_scaled_residual_add_separate_fp32",
    "status": "validated",
    "implementation": { "isa_dispatch": "scalar",
      "threading": { "runtime": "serial", ..., "reduction_order_effect": "none" } },
    "arithmetic": { "partial_accumulator": "none", "merge_order": "none",
      "deterministic": true, "thread_count_changes_arithmetic_order": false } } ],
  "call_abi": { "version": 1, "params": [
    { "name": "residual", "source": "activation:residual", "ports": ["input:residual"] },
    { "name": "branch",   "source": "activation:branch",   "ports": ["input:branch"] },
    { "name": "scale",    "source": "param:scale" },
    { "name": "output",   "source": "output:output",       "ports": ["output:output"] },
    { "name": "elements", "source": "dim:elements" } ] },   // where every C arg comes from
  "tests": { "unit": ["unittest/test_audio_encoder.py"],
    "oracle": "NumPy FP32 multiply followed by FP32 add" }
}

Workspace-using map — version/v8/kernel_maps/attention_forward_causal_head_major_gqa_flash_strided_f16kv.json declares scratch: [{ name: "rounded_kv", dtype: "fp32", shape: [2, "KV", "T", "D"], size_resolution: "required", lifetime: "kernel_call" }] with the note “K/V values rounded through FP16 once and retained as FP32” — the intermediate rounding point is part of the contract, and the planner sizes the region from the shape expression. Do not copy either map blindly: validate symbolic dims and interface names against the kernel you actually wrote.

Every section is schema-checked. The schemas live under version/v8/schemas/: kernel_call_abi.schema.json, kernel_provider_selection.schema.json, kernel_physical_layout.schema.json, numerical_kernel_capability.schema.json, kernel_codegen_capability.schema.json (all five verified present at this revision). The enforcing gates are tests/test_v8_provider_selection.py and tests/test_v8_kernel_call_abi.py, both run by make test-kernel-maps; see the kernel-maps contributor recipe for the full map-to-generated-C lane.

Failure is explicit, not silent. A misspelled port or an invented source is a hard error at build-IR time. Executed at the stated revision executed:

$ python3 -m unittest tests.test_v8_kernel_call_abi.V8KernelCallABITests.test_unknown_call_source_is_a_hard_failure \
    tests.test_v8_kernel_call_abi.V8KernelCallABITests.test_all_contract_governed_maps_own_valid_call_abi
Ran 2 tests in 0.053s - OK
# the first test feeds source "guessed:model_default" and requires RuntimeError: "unsupported source"

An unsupported dtype fails the same way at selection: if the map's quant block does not declare the circuit's required activation/weight dtypes, resolution finds no candidate and the build stops with the rejected-candidate reasons (the selection trace records them).

Registry generation. version/v8/kernel_maps/KERNEL_REGISTRY.json and version/v8/src/ck_kernel_dispatch_policy_v8.inc are generated artifacts. Never hand-edit them. The workflow, exactly as make v8-kernel-registry-freshness implements it:

python3 -c "import sys; sys.path.insert(0, 'version/v8/scripts'); \
  import ck_run_v8; ck_run_v8.step_regenerate_kernel_registry(force=True)"
python3 version/v8/scripts/generate_kernel_dispatch_policy_v8.py
git diff --exit-code -- version/v8/kernel_maps/KERNEL_REGISTRY.json   # must be clean in CI

Worked example: trace one C argument end to end

This is the actual decode-step RMSNorm from the verified Qwen2 0.5B run at this revision. One argument — gamma — traced through every layer of the stack:

7. Define the circuit

A circuit declares model composition, not a kernel name list: operation instances and ordering, tensor connections, weight bindings, repeated blocks, parameters and conditional features, state producers/consumers, numerical requirements, and modality bridges. Read version/v8/circuits/*.json before inventing structure — the compiler supports exactly what those files use. The complete Qwen2 circuit (abridged to structure; full file is 200 lines):

{
  "version": 2, "name": "qwen2", "family": "llama",
  "projection_inputs": { "q_proj": {"x": "main_stream_q8"}, ... },   // tensor connections
  "flags": { "use_qkv_bias": "from_weights", "activation": "swiglu", "rope": "rope", "tokenizer": "bpe" },
  "contract": {
    "tokenizer_contract": { "tokenizer_type": "bpe", "bos_policy": "model_defined", ... },
    "chat_contract": { "name": "qwen2", "turn_prefix": "<|im_start|>{role}\n", ... },
    "attention_contract": { "rope_layout": "split", "qk_norm": false,
                            "kv_layout": "layer_major_kv_cache", "attn_variant": "dense" },
    "block_contract": { "norm_type": "rmsnorm", "mlp_formula": "gate_up -> silu_mul -> down" },
    "logits_contract": { "final_norm": "rmsnorm", "lm_head": "weight_tying" },
    "quant_contract": { "kernel_select": "weight_dtype_registry", "per_tensor_mixed_quant": true },
    "runtime_invariants": { "required_call_args": { "kv_cache_batch_copy": [...] } }
  },
  "kernels": { "rope_qk": "rope_forward_qk" },
  "required_contracts":       { "decoder.attention":  { "op": "attention", "phases": {
      "prefill": { "requires": { "numerics.attention_reduction": "fp32_online", ... }, "validation": "validated" },
      "decode":  { "requires": { ... }, "validation": "validated" } } } },
  "required_numerical_contracts": { "decoder.swiglu": { "operation_interface": "swiglu.fp32.v1",
      "phases": { "prefill": { "contract_id": "swiglu_fp32_ggml_vector_exp_fp32_output" }, ... },
      "checkpoint": { "id": "decoder.layer.{layer}.swiglu.output", "producer": "silu_mul",
                      "logical_layout": "token_major", "axis_names": ["token", "channel"] } } },
  "sequence": ["decoder"],
  "block_types": { "decoder": {
      "sequence": ["header", "body", "footer"],
      "header": ["bpe_tokenizer", "dense_embedding_lookup"],
      "body": { "type": "dense", "ops": ["rmsnorm", "qkv_proj", "rope_qk", "attn", "out_proj",
                "residual_add", "rmsnorm", "mlp_gate_up", "silu_mul", "mlp_down", "residual_add"] },
      "footer": ["rmsnorm", "weight_tying", "logits"] } }
}

The minimal normalize → projection → activation → projection → residual chain is the body tail: rmsnorm, mlp_gate_up, silu_mul, mlp_down, residual_add. Extensions are declared, not coded: attention/KV is the attn op plus attention_contract + runtime_invariants; shared-KV is a named attention variant (attention_forward_causal_head_major_shared_kv_gemma4.json exists for Gemma4); recurrent state, MoE routing, and cross-attention have their own circuits (laguna.json, qwen35.json, cohere_transcribe.json); vision/audio bridges compose circuits (qwen3vl.json + qwen3_vl_vision.json, parakeet_tdt.json).

Nearest-neighbor diff — Qwen2 vs Qwen3, the two real files (this is the entire structural difference in the decoder body):

--- version/v8/circuits/qwen2.json
+++ version/v8/circuits/qwen3.json
 flags:  - "use_qkv_bias": "from_weights"            + "use_qkv_bias": false, "has_qk_norm": true
 body:   - ["rmsnorm", "qkv_proj", "rope_qk", ...]   + ["rmsnorm", "qkv_proj", "qk_norm", "rope_qk", ...]

One flag and one inserted op — Qwen3's per-head QK norm — is what separates two model families, because both reuse the same providers. The circuit genuinely controls execution: section 9 shows the op names appearing verbatim in generated C comments.

8. Build the import adapter

Model-specific logic is legitimate here — this is the one component whose job is knowing that GGUF calls it blk.0.attn_q.weight while CKE calls it layer.0.wq. Start from the real files: version/v8/model_maps/gguf_ck_map.json and safetensors_ck_map.json (declarative tensor-name/metadata contracts per architecture), and the converters version/v8/scripts/convert_gguf_to_bump_v8.py and convert_safetensors_to_bump_v8.py.

Inventory the external package first. Run against the tutorial model executed:

python3 version/v8/scripts/convert_gguf_to_bump_v8.py \
  --gguf ~/.cache/ck-engine-v8/models/Qwen--Qwen2-0.5B-Instruct-GGUF/qwen2-0_5b-instruct-q4_k_m.gguf \
  --list

[gguf] version=3 source_arch=qwen2 circuit=qwen2 tensors=290 kv=26 alignment=32
[gguf] tensor types: Q5_0: 132 | F32: 121 | Q8_0: 13 | Q4_K: 12 | Q6_K: 12
  - token_embd.weight: Q8_0 dims=(896, 151936)
  - blk.0.attn_q.weight: Q5_0 dims=(896, 896)
  - blk.0.attn_v.weight: Q8_0 dims=(896, 128)
  - blk.0.ffn_down.weight: Q6_K dims=(4864, 896)
  - output_norm.weight: F32 dims=(896,)

The inventory covers: external tensor names, shapes/orientation, packed or fused projections, per-tensor quantization (this file mixes five types — “Q4_K_M” in the filename is a recipe name, not a per-tensor truth), tied weights (Qwen2 ties the LM head to token_embd.weight), missing optionals, tokenizer/special tokens, position/cache config, and modality preprocessing metadata when present.

The mapping table is the adapter's contract — external tensor → canonical weight role → transformation → stored dtype. From the verified run's weights_manifest.map:

Conversion must not: silently reinterpret quantized storage (Q5_0 stays Q5_0 bytes; dequantization happens in the kernel under a declared contract), change precision, infer topology from filenames (architecture comes from GGUF metadata: source_arch=qwen2 selects the circuit), drop tensors without an explicit recorded reason (the Parakeet conversion excluded 24 BatchNorm training counters explicitly, 699 mapped + 24 excluded + 0 unaccounted), or introduce a runtime dependence on the original format — the deployment reads BUMP, never GGUF. Each transformation gets a small conversion test: bytes/shape/numerical interpretation, e.g. --verify checks parity between GGUF and bump after conversion, and --plan-only validates the full plan without writing. Export = weights (weights.bump, BUMPWGT5) + sidecars (weights_manifest.json, weights_manifest.map, config.json) + tokenizer/frontend assets (tokenizer.json embedded or referenced). See GGUF conversion and the BUMP format for format-level detail.

9. Run lowering and inspect the generated C

The verified entry point is the wrapper version/v8/scripts/cks-v8-run (it selects a Python, checks requirements, and drives ck_run_v8.py). The full pipeline, exactly as run for this page executed:

version/v8/scripts/cks-v8-run run \
  hf://Qwen/Qwen2-0.5B-Instruct-GGUF/qwen2-0_5b-instruct-q4_k_m.gguf \
  --context-len 512 --prompt "Say hello in one short sentence." \
  --max-tokens 16 --force-compile

[CK parallel prefill] Prepared 120 model-owned prefill weights at load time
Loading model from /home/antshiv/.cache/ck-engine-v8/models/Qwen--Qwen2-0.5B-Instruct-GGUF...
Using built-in C tokenizer (bpe)
Model loaded! Vocab: 151936, Context: 512

Prompt: Say hello in one short sentence.
Response: Hello! How can I assist you today?

prompt eval:    61.04 ms /   26 tokens (   2.35 ms/tok,  425.97 tok/s)
      decode:   128.68 ms /    9 runs   (  14.30 ms/tok,   69.94 tok/s)
stop: eos token 151645

The command does not pin --temperature, so the exact response wording and timings vary between runs; the re-run at the documentation revision produced the same 26-token prefill and 9-token decode shape with a slightly different sentence. The structural facts — vocab, context, token counts, eos stop — are the stable evidence.

Useful wrapper flags (read from --help at the stated revision source-verified): --force-convert re-runs the adapter, --force-compile regenerates and recompiles, --generate-only stops after codegen, --plan-only certifies prefill/decode memory plans without codegen or execution, --generate-visualizer opens the IR visualizer on the run. The run directory then contains the complete inspection surface:

The annotated call from section 6 is the pattern for reading any generated op: which input buffer (A_EMBEDDED_INPUT, the region the previous op wrote), why this offset (planner-assigned, compiled as a #define), why this stride (declared layout in the map), which provider (map impl.function, here rmsnorm_forward_parallel_dispatch), where each dim came from (dim:seq_len → 1 in decode, dim:embed_dim → 896 from GGUF metadata), which workspace region (map scratch → planner region), which state is read/modified (KV cache ops declare their append semantics).

The IR visualizer renders this interactively — operations, selected providers, memory offsets — including the “Explain this operation” panel (merged in #502) that renders why a provider was considered or rejected. Launch it with --generate-visualizer or python3 version/v8/tools/open_ir_visualizer_v8.py --list. If compilation needs a semantic feature the IR cannot express, the fix is extending the operation/lowering contract (circuit + schema + resolver + tests) — never a Python model loop as a production fallback.

10. Establish oracle parity — two separate paths

Certification compares the generated runtime against an independently executed reference. CKE maintains two oracle families, and they are not interchangeable at every boundary: PyTorch/Transformers and llama.cpp can embody different numerical contracts (llama.cpp's vector-exp SwiGLU vs libm, different reduction orders, different quantization dequant paths). Pick the oracle whose contract your circuit declares — Qwen2's SwiGLU contract names llama.cpp arithmetic explicitly.

PyTorch/Transformers path. Pin the model revision and the framework revision (the Parakeet page pins both: model 541d1f99..., Transformers 66799f45...). Match preprocessing, tokens, positions, masks, state, and precision; run eval mode; hook the actually-executed modules (functional/fused ops need other hook points than module outputs); capture immediately and detach().clone() mutable tensors; write explicit semantic mappings between oracle tensors and CKE checkpoint IDs. The real adapter is version/v8/scripts/xray_decoder_pytorch_v8.py — it registers register_forward_pre_hook/register_forward_hook pairs per layer, handles tuple outputs (hidden-state tuples are unpacked at the hook), and clones before the next module mutates in place. A generic sketch is fine for orientation but is labeled illustrative; copy the discipline from the real script.

llama.cpp/GGUF path. The oracle is the pinned in-repo submodule (llama.cpp at commit f3e182816421c648188b5eab269853bf1531d950 at this revision). Match quant formats and attention/reduction settings; use the existing capture infrastructure first — export_llama_hidden_v8.py exports final hidden states for explicit token IDs, compare_first_token_logits_v8.py runs tokenizer-free first-token logits parity between the CK run dir and the llama.cpp runtime, and certify_text_prompt_parity_v8.py certifies prompt sets. Record the actual loaded oracle library in evidence, map raw node identities explicitly, and reject ambiguous mappings. Add a narrowly scoped hook/patch only where the existing capture cannot see the boundary you need.

Commands for both oracle paths, read from source and --help output at the stated revision but not executed for this page (the llama.cpp submodule is pinned in-repo; these were not re-run here) source-verified:

# first-token logits parity: CK generated runtime vs pinned llama.cpp
python3 version/v8/scripts/compare_first_token_logits_v8.py \
  --model-dir ~/.cache/ck-engine-v8/models/Qwen--Qwen2-0.5B-Instruct-GGUF \
  --gguf ~/.cache/ck-engine-v8/models/Qwen--Qwen2-0.5B-Instruct-GGUF/qwen2-0_5b-instruct-q4_k_m.gguf \
  --tokens 9707,11,1879,330,13339  --require-top1-match --json-out build/parity_first_token.json

# hidden-state capture from the pinned llama.cpp for X-Ray alignment
python3 version/v8/scripts/export_llama_hidden_v8.py \
  --gguf <model.gguf> --tokens-before 151644,8946,198 --tokens 9707,11 --out build/llama_hidden.json

11. Diagnose divergence with X-Ray

When parity fails, do not poke at tolerances. Walk the decision tree — each branch names an owner, and the fix lands in that owner's artifact (see the X-Ray page for the full evidence model and capture-neutrality gates):

The evidence chain you must be able to produce for any verdict: semantic op → selected provider → generated call → actual capture → oracle mapping. The trace in section 6 is the first three links; X-Ray's capture manifest and the oracle adapter are the last two. Capture is not free: fused intermediates are unobservable without changing execution, which is why the neutrality gate requires captured runs to reproduce uncaptured runs bit-for-bit before any cross-engine comparison is accepted. Honest limitation: declarative checkpoint coverage is not yet uniform across modalities and checkpoints — text decoder boundaries are the best covered; when a boundary you need has no declared checkpoint, that is an implementation requirement to record, not a gap to paper over. Preserve failing inputs as regression fixtures where licensing permits (section 14).

12. Optimize only after the scalar contract works

The ladder is strict: scalar → vectorize → block/pack → thread → fuse (where justified) → measure at production shapes. A correctness gate stands before each stage — the map carries all variants, and the contract says which orders are legal:

Methodology, measurement discipline, and tuning workflow live on the kernel tuning, profiling, and SIMD architecture pages; the v8 runbook records per-family performance evidence. The numeric contract machinery is on numerical contracts.

13. Prove standalone deployment

The final practical exercise, executed for this page executed — with an explicit scope limit. What the evidence shows: the compiled artifact set was copied to a bare directory outside the repository checkout and executed there, with no Python in the execution path and only the recorded libraries loaded. What it does not show: a true isolated host — the copy ran on the same machine, where the repo, the model caches, and the network still existed, even though the binary demonstrably read none of them. Treat this as bundle-only execution outside the checkout, and see the open follow-up below:

# bundle = the compiled artifact set (389 MB total for Qwen2 0.5B q4_k_m):
ck-cli-v8  libmodel.so  libckernel_engine.so  libckernel_tokenizer.so
weights.bump  config.json  tokenizer.json  init.json  init_call.json
weights_manifest.json  weights_manifest.map

# in a bare directory outside the checkout (same host):
LD_LIBRARY_PATH=. ./ck-cli-v8 ./libmodel.so ./weights.bump

[Runtime] ABI: v1 | Role: decoder | Capabilities: decode text-encode token-decode
          chat-format stop-tokens mixed-prefill named-activations
[Hardware] AVX2+FMA (Intel Haswell+) | Vector: 256-bit | Kernel: gemm_avx2
You: What is 2+2? Answer with just the number.
Assistant: 4
prefill 32 tok 49.2 ms | decode 1 tok 24.4 ms

The binary demonstrably loaded only the recorded libraries: ldd libmodel.so resolves to the two CKE native libraries (libckernel_engine.so, libckernel_tokenizer.so) plus the system libraries libgomp, libc, libm — five in all — and ldd ck-cli-v8 resolves to libckernel_tokenizer.so, libm, and libc. The CLI then dlopens the model library given on its command line (./ck-cli-v8 <model.so> <weights.bump>), so the dependency contract of a deployment is the union of both link records plus that explicit load — ldd on the model library alone does not describe the CLI. The bundle declares its native dependencies (the two CKE libraries above), target features (the ISA the artifact was compiled for, reported at load: AVX2+FMA here), and memory requirements (the arena total, 411,400,448 bytes for this configuration). .ck_runtime_bundle.json in the run directory records provenance: compiler path + sha256 and the sha256 of every engine source file that went into the build.

Honest limitations. Two follow-ups remain open. First, a true isolated-environment run — a host or container with no checkout or cache mounts, no network, and a recorded loaded-library list — has not been performed and is the real proof this section points at; the /tmp copy demonstrates only that the bundle is self-contained as a file set. proposed Second, there is no single cke export command at this revision: the bundle is assembled by copying the run directory's artifact set, as above. A one-command exporter that validates completeness (all declared files present, hashes recorded, dependency closure checked) is an implementation requirement, not an existing feature. Note also that the deployment artifact is ISA-specific — an AVX2-compiled libmodel.so requires an AVX2 host; cross-ISA portability means compiling per target.

14. Register regression coverage

A bring-up is done when its evidence is registered: wired into targets another engineer (and CI) can run. All targets below were confirmed present in the Makefile at the stated revision (read from source; only the doc-policy subset was executed for this page) source-verified:

Model-family registration lives in version/v8/regression/families.json: each family pins the model source, context length, runtime args, response contract, and structural expectations (e.g. Qwen2 asserts the lowered ops contain rope_forward_qk_with_rotary_dim) — add your family there so run_regression_v8.py --family <id> exercises it. Then the vocabulary, because reviewers will ask: registered means a target or manifest entry names your evidence; executed means it ran somewhere and produced a report; certified means the run passed its declared gates on the pinned configuration. Only the last is a claim.

Contributor acceptance checklist

• Existing providers reused where compatible (inventory table in the PR, report_model_novelty_v8.py output attached)
• New scalar math independently tested (hand cases + fixed-seed randomized + boundary/aliasing/invalid-input cases)
• Maps and ABI validated (make test-kernel-maps; schemas pass; no hand-edited registry)
• Workspace/state ownership explicit (map-declared scratch, planner-sized, no kernel-side allocation)
• Conversion transformations tested (bytes/shape/numerical interpretation per transform; every external tensor accounted for)
• Circuit genuinely controls generated execution (op names appear in model_v8.c; changing the circuit changes the generated program)
• Generated model tested, not just C kernels (end-to-end run through cks-v8-run / ck-cli-v8)
• Oracle/capture provenance retained (pinned revisions, loaded-library record, evidence JSON)
• Regression cases registered (family manifest and/or make target)
• Standalone deployment demonstrated for the claimed scope (bundle-only execution outside the checkout with recorded loaded libraries; a true isolated-host run is either attached or listed as open)
• Unsupported features clearly listed (the honest-limitation sentence is part of the deliverable)

Further reading

• Kernel maps and provider selection — the resolver, selection traces, and the map-level contributor recipe
• Kernel library and quant formats — the provider inventory you are reusing
• Codegen and IR pipeline — the lowering stages this guide inspects
• v8 runbook — per-family certified commands and evidence
• X-Ray — capture neutrality, evidence schemas, divergence attribution
• Parakeet TDT bring-up — the worked audio example whose inventory discipline this guide generalizes
• Whisper tensor trace — deep dive: one encoder memory tensor into every decoder layer (producers, consumers, explicit edges)
• Developer guide and contributing — repo workflow around everything above

Diagram − + ×