Model + Kernel Matrix

A visual map of what the C Kernel Engine supports: IR templates, kernel families, quant contracts, and pipeline coverage (inference-only vs. inference + v7 training). Grounded in the actual templates, IR lowering passes, and GGUF test corpus run locally.

Recommended runner track: this model page points to the latest hardened v8 command path for hands-on smoke tests. Version pages remain historical snapshots of each engine generation; the cards below show the current practical command to convert, compile, run, and optionally emit the IR visualizer for that family. Large cards such as Gemma4, Nemotron, GLM4, and vision models assume a high-memory CPU host.

One Pipeline, Every Model

Every supported family enters through the same deterministic compile path. The artifact (GGUF) carries the numbers, the template carries the structure, and lowering turns both into a concrete C runtime with a fixed memory plan. Vision families add a second lane that produces an encoder prefix and bridges it into the decoder. Audio families arrive as safetensors → BUMP and run an encoder–decoder pair: the audio encoder's output becomes cross-attention memory that the decoder reads on every generated token.

click / tap the diagram to expand

Certification Summary

Evidence is not one bucket. Each family below is labeled with the strongest evidence class it has actually earned — an eight-token replay agreement and a 4,096-token bit-exact trajectory are different claims, and the dots on the cards use the same classes.

None of these labels alone means production certification. CKE requires complete executed-graph and deterministic decode evidence for full-model numerical certification, then separately gates advertised context/media capacity, quality, ISA coverage, memory safety, repeatability, and performance. See the v8 certification contract, the model-family provider ownership rule, and X-Ray workflow.

Trajectory exact — full token trajectory bit-exact against the oracle Numerical sampled — oracle agreement at sampled checkpoints (first-token / top-k / cosine / prefix replay) Coherent E2E — coherent end-to-end generation; smoke-level evidence Contract-only — circuit and contract tests; no model-weight run claimed Planner-only — memory-plan certification; runtime not claimed at that scale

Model	Runtime status	Strongest evidence	Max tested context	Oracle	Remaining gap
Qwen2 0.5B	Coherent E2E	nightly regression smoke	1,024	v8 nightly regression	long-context and perf sweeps
Qwen3 0.6B	Coherent E2E	nightly regression smoke	1,024	v8 nightly regression	long-context and perf sweeps
Qwen3.5 0.8B	Coherent E2E	nightly regression smoke (hybrid DeltaNet)	1,034	v8 nightly regression	long-generation parity monitored
Qwen3.6 27B	Numerical sampled	kernel-exact head-parallel DeltaNet prefill + Q4_K×Q8_K VNNI x16 providers	—	kernel parity gates	1K logit parity open; batched prefill opt-in
Qwen3.8 27B	Trajectory exact Planner-only	4,096-token trajectory bit-exact (4,000/4,000 vocab rows); 262K memory planner certified	4,096 runtime / 262,144 planner	llama.cpp	128K/262K runtime claims deferred; ~1.49× slower than llama.cpp at 8K
Qwen3-VL 8B	Numerical sampled	BF16 and llama.cpp corpus certification lanes; promoted vision baseline	1,024 smoke	PyTorch + llama.cpp corpus	semantic vision parity ongoing
Gemma3 270M	Coherent E2E	nightly regression Q5_K_M	1,024	v8 nightly regression	—
Gemma4 E4B	Numerical sampled	Q4_K_M coherent smoke + BF16 safetensors PyTorch parity lane	2,048	PyTorch	semantic vision parity active
GLM4 9B	Coherent E2E	GGUF Q4_K_M coherent generation; safetensors comparator lane	1,024	PyTorch comparator	long-trajectory sweeps
Nemotron Nano 9B v2	Coherent E2E	Q4_K_M coherent smoke; Mamba2 state-shape contracts	1,024	PyTorch parity lane	long-trajectory sweeps
Kimi-VL A3B (text)	Numerical sampled	first-token + top-20 ranking match, cosine 0.997696; bit-identical repeat runs	2,048	PyTorch	MoonViT vision bridge; internal BF16 drift near layer 7
Cohere2 Command R7B	Numerical sampled	tokenizer-free replay matches llama.cpp for the first 8 greedy positions	2,048	llama.cpp	long-trajectory parity; performance sweeps
Cohere North Mini Code 30B-A3B	Coherent E2E	two repeatable 131,072-token CKE capacity runs with identical first-logit and output-token hashes	131,072	CKE repeatability	matched full-model oracle parity; practical long-code quality
Cohere North Micro Vision 2.4B	Coherent E2E	real-image encoder/bridge/decoder run; official PyTorch layer X-Ray remains follow-up work	real-image smoke	PyTorch required	complete encoder/bridge/decoder parity; 40-image OCR quality
Cohere Transcribe 2B	Contract-only	2,104-tensor inventory plus 52 independent oracle checkpoints; CKE conversion fails closed	—	CrispASR	assemble and certify CKE encoder, cross-attention, decoder, and tokenizer
Cohere Command A+ 218B/25B-active	Coherent E2E Numerical sampled	675/675 required weights mapped; native packed NVFP4 leaf parity; coherent short text	126 prompt + 32 decode	compressed-tensors formula + pinned leaf oracle	full-model/long-context parity, vision, quality, and matched performance
Command A Vision / Aya Vision / Translate	Not implemented	official artifacts identified; no CKE real-weight trajectory claim	—	PyTorch required	circuit/provider gap audit and complete bring-up
Laguna-XS 2.1	Coherent E2E	bit-exact embedding + layer-0 RMSNorm; router replay within one ULP	2,048	llama.cpp (diagnostic)	long-trajectory parity not claimed (near-tied MoE route flips)
Instella-MoE 16B-A3B	Numerical sampled	full-logit cosine 0.99998 + top-1 match at a 32-token checkpoint	32-token checkpoint	PyTorch	quantized GGUF and long trajectories uncertified
Llama / Nanbeige 3B	Coherent E2E	nightly regression smoke	1,024	v8 nightly regression	—
Whisper Tiny/Base/Small	Trajectory exact	JFK fixtures match Hugging Face token for token (all three sizes)	JFK sample; 33 s Base long-form	Hugging Face	broader corpus certification; hour-scale recordings
GPT-2	Coherent E2E	GGUF baseline	—	v8 nightly regression	legacy lane

Qwen2

RMSNorm · RoPE split-half · SwiGLU · GQA
QKV bias from weights · BPE tokenizer

Quantized inference — GGUF, GEMM/GEMV dispatch

v7 training — IR backprop, AdamW, parity gates

Commands & runnable lanes

Latest hardened v8 smoke

version/v8/scripts/cks-v8-run run \
  hf://Qwen/Qwen2-0.5B-Instruct-GGUF/qwen2-0_5b-instruct-q4_k_m.gguf \
  --context-len 1024 --force-compile --force-convert --chat-template=auto \
  --generate-visualizer

Q4_K_M ★ Q5_K Q6_K Q8_0 FP32

Qwen3

RMSNorm · RoPE split-half · QK‑Norm · SwiGLU
No QKV bias · BPE tokenizer · learned QK‑Norm

Quantized inference — GGUF, GEMM/GEMV dispatch

v7 training — IR backprop, AdamW, parity gates

Commands & runnable lanes

Latest hardened v8 smoke

version/v8/scripts/cks-v8-run run \
  hf://Qwen/Qwen3-0.6B-GGUF/Qwen3-0.6B-Q8_0.gguf \
  --context-len 1024 --force-compile --force-convert --chat-template=auto \
  --generate-visualizer

Q4_K_M ★ Q5_K Q6_K Q8_0 FP32

Qwen3.5

Hybrid recurrent + gated attention · RMSNorm · RoPE
QK‑Norm (full‑attn only) · SwiGLU · Gated DeltaNet · BPE tokenizer
Block pattern: 3×recurrent → 1×full_attention

Quantized inference — hybrid recurrent-attention decode, DeltaNet kernel parity

v7 training — IR backprop, DeltaNet backward kernels, AdamW, parity gates

Bring-up fix: preserved <think> / </think> markers on the built-in C tokenizer path and moved visible vs. suppressed thinking into the exported chat contract, instead of relying on Python-tokenizer fallback.

hybrid_recurrent_attention gated_deltanet 0.8B dense compatible

Commands & runnable lanes

Latest hardened v8 smoke

version/v8/scripts/cks-v8-run run \
  hf://unsloth/Qwen3.5-0.8B-GGUF/Qwen3.5-0.8B-Q4_K_M.gguf \
  --context-len 1034 --force-compile --force-convert --chat-template=qwen35 \
  --generate-visualizer

Q4_K_M ★ Q8_0 FP32

qwen35-family hybrid circuit

Qwen3.6 27B

Hybrid recurrent + gated attention · Gated DeltaNet
Q4_K_M quantized inference · optimized batched prefill (opt-in)

Exact head-parallel DeltaNet prefill — value heads partitioned across the threadpool once per prompt

Q4_K × Q8_K AVX-512 VNNI x16 prefill — AVX2 / reference fallback on other CPUs

Sequential prefill remains the default while long-context parity is unresolved

Prefill coverage (PR #277): the measured Qwen3.6-27B Q4_K_M projection shapes dispatch to the AVX-512 VNNI x16 provider with packed weights prepared at load time, reaching 11.20 tok/s prompt rate (about 2.6x the previous CKE batched baseline) on an ICX AVX-512/VNNI host. 1K-context logit parity is not certified yet, so the lane stays behind CK_V8_FORCE_BATCHED_PREFILL=1. Operator steps: v8 runbook; performance context and remaining targets: prefill performance roadmap.

exact_head_parallel_deltanet q4k_q8k_vnni_x16_prefill opt-in until 1K parity

Q4_K_M ★

Qwen3.8 27B

Hybrid recurrent + gated attention · Gated DeltaNet (3×recurrent → 1×full)
Text M-RoPE split · QK‑Norm (full‑attn only) · SwiGLU · tied LM head · BPE / ChatML

Q4_K_M 27B text runtime — 4,096-token trajectory bit-exact to llama.cpp (4,000/4,000 full-vocabulary rows)

262,144-token context memory planner certified (capacity-aware nightly lane)

Six 131,072-token artifact runs completed at 32.37–32.71 tok/s prefill and 1.36–1.41 tok/s decode

128K bit-exact parity and 262K runtime execution remain separate open certification boundaries

Identity: upstream GGUFs advertise general.architecture=qwen35; the converter selects qwen38 via artifact metadata (basename / size label). Circuit carries validated contract selectors for q5_k, q6_k, q8_0 and BF16 recurrent dtypes. Evidence: PRs #387, #388, #390. Inspect the generated 128K engineering artifacts.

Commands & runnable lanes

Qwen3.8 contract + certification lane

.venv/bin/python -m unittest tests.test_v8_qwen38_contract
QWEN38_MODEL_DIR=/path/to/qwen38-27b QWEN38_GGUF=/path/to/qwen38.gguf \
  make certify-qwen38-svg-parity

hybrid_recurrent_attention mrope_text_split 262K planner certified

Q4_K_M ★ Q5_K Q6_K Q8_0 BF16 contracts

Gemma3

RMSNorm · RoPE split-half · QK‑Norm · GeGLU · Sliding Window
Post-norms · embed scale √dim · SentencePiece

Quantized inference — GGUF, sliding-window attn kernel, split-half RoPE parity restored

v7 training — IR backprop, AdamW, parity gates

Fix detail: Gemma was applying RoPE with the wrong dimension pairing. Gemma expects split-half RoPE, where the first half of Q/K is paired with the second half during the rotation step. The runtime could still choose the other common layout, so attention drifted from the reference path. The fix was to read exported rope_layout metadata and select the matching rope_qk kernel automatically.

first-divergence @ rope_qk rope_layout contract honored

Commands & runnable lanes

Latest hardened v8 smoke

version/v8/scripts/cks-v8-run run \
  hf://unsloth/gemma-3-270m-it-GGUF/gemma-3-270m-it-Q5_K_M.gguf \
  --context-len 1024 --force-compile --force-convert --chat-template=auto \
  --generate-visualizer

Q8_0 ★ Q4_K BF16 FP32

Qwen3-VL 8B Instruct

Vision encoder bridge · multimodal token stitching · Qwen3 decoder
GGUF decoder + mmproj runtime · thinking-mode control

v8 multimodal inference — image preprocessing, bridge, mixed prompt prefill

Nightly vision smoke — promoted baseline before adding more vision families

Bring-up scope: Qwen3-VL is the current promoted multimodal baseline. It validates that the encoder bridge, image marker contract, and decoder mixed-prefill path work together before Gemma4 vision and later audio lanes are treated as production-ready.

Commands & runnable lanes

Qwen3-VL vision smoke

version/v8/scripts/cks-v8-run run \
  hf://Qwen/Qwen3-VL-8B-Instruct-GGUF/Qwen3VL-8B-Instruct-Q4_K_M.gguf \
  --mmproj hf://Qwen/Qwen3-VL-8B-Instruct-GGUF/mmproj-Qwen3VL-8B-Instruct-Q8_0.gguf \
  --image-path version/v8/test_assets/v8_vision_doc_card_72.png \
  --prompt 'Explain this image.' \
  --context-len 1024 --force-compile --force-convert \
  --thinking-mode suppressed

multimodal_bridge mixed_prefill vision nightly smoke

Q4_K_M decoder ★ Q8_0 mmproj image bridge

circuits: audio frontend + encoder + decoder

Whisper Tiny / Base / Small

First encoder–decoder family · generated frontend (7 ops → ck_model_run_audio_wav)
LayerNorm · learned absolute position · GELU (erf) · dense bidirectional encoder

v8 audio inference — WAV to transcript via cks-v8-run audio, cached cross-attn K/V (≈79× decode)

Artifact-gated E2E — Tiny, Base, and Small JFK fixtures match Hugging Face token for token

Bring-up scope: Whisper Tiny, Base, and Small FP32 — English text and timestamp decoding certified on the JFK fixture; Base also has a 33-second long-form parity fixture. PCM16 WAV input; longer recordings use sequential source windows but arbitrary hour-scale recordings are not yet corpus-certified. A single safetensors checkpoint builds a generated encoder + decoder run-directory pair.

Commands & runnable lanes

Whisper Base end to end

version/v8/scripts/cks-v8-run audio hf://openai/whisper-base \
  --wav /path/to/audio.wav --language en --task transcribe

encoder_decoder cross_attention artifact-gated e2e

FP32 ★ safetensors → BUMP enc + dec pair

GLM4

RMSNorm · partial RoPE · GQA · SwiGLU
BPE tokenizer · attention bias · GGUF + safetensors conversion lane

v8 template — GLM4 chat contract and partial-RoPE metadata

Conversion/parity harness — GGUF fixes, safetensors declarative map, PyTorch-vs-CK comparator

Bring-up scope: GLM4 now has the real GGUF runtime lane plus the safetensors/PyTorch parity lane. The key hardening was making partial pairwise RoPE and projection producer/consumer wiring explicit enough that Q4_K_M GGUF generation stays coherent instead of collapsing at the first token.

declarative safetensors map partial_rope contract synthetic conversion tested

Commands & runnable lanes

Latest hardened v8 smoke

version/v8/scripts/cks-v8-run run \
  hf://unsloth/GLM-4-9B-0414-GGUF/GLM-4-9B-0414-Q4_K_M.gguf \
  --context-len 1024 --force-compile --force-convert --chat-template=glm4 \
  --prompt 'Give me a detailed example of C, Python and SQL code.' \
  --max-tokens 256 --temperature 0.0 --generate-visualizer

BF16 safetensors GGUF Q4_K Q8_0

Cohere2 Command R

Dense GQA · LayerNorm (shared pre-block norm, synthesized beta) · SwiGLU
Parallel attention+MLP residual · 3:1 sliding:full attention
Pairwise RoPE on sliding layers only · logit_scale footer · tied embeddings · BPE

GGUF discovery via model map — standard cks-v8-run path, no family branch

Command R7B Q4_K_M — tokenizer-free replay matches llama.cpp for the first 8 greedy positions

Long-trajectory parity and matched performance sweeps remain future certification work

Bring-up note (PR #401): lowering honors the model-declared logit_scale via a final_logit_scale_f32 footer op, and RoPE applies only to sliding layers per the exported contract. Contract test: tests.test_v8_cohere2_contract. Provider-level story: Cohere Kernel Story.

Commands & runnable lanes

Cohere2 smoke

version/v8/scripts/cks-v8-run run \
  /path/to/command-r7b-q4_k_m.gguf \
  --context-len 2048 --force-compile --force-convert --chat-template=auto \
  --max-tokens 64 --temperature 0.0

parallel_attn_mlp logit_scale contract rope sliding-layers only

Q4_K_M ★ Q6_K GGUF

Laguna-XS 2.1

Hybrid attention MoE · gated GQA (softplus per-head gate) · QK‑Norm
YaRN on global layers / full-rotary on sliding · sigmoid router + correction bias
Top‑8 routed + shared SwiGLU experts · RMSNorm · untied output.weight · BPE

Q4_K_M text runtime (PR #404) — mixed Q4/Q6 expert storage, map-owned routed/shared SwiGLU providers

Embedding and layer-0 RMSNorm bit-exact; identical-input router replay preserves selected experts within one ULP

Long-trajectory parity not claimed — near-tied MoE route flips vs llama.cpp

Bring-up note (PR #404): ~20 prefill / 14 decode tok/s on the Ryzen node (bring-up measurement, not a certified benchmark). Known oracle gap: the external llama.cpp Laguna loader omits GGUF YaRN beta_fast=64 (defaults 32). Softplus gate formula is observed, not yet oracle-validated. Contract test: tests.test_v8_laguna_contract. Expert kernels: MoE Expert Kernels.

Commands & runnable lanes

Laguna smoke

.venv/bin/python version/v8/scripts/ck_run_v8.py run \
  /path/to/Laguna-XS-2.1-Q4_K_M.gguf \
  --run /srv/cke/profiles/laguna-xs-2.1 \
  --context-len 2048 --force-convert --force-compile \
  --prompt 'Write a complete C program that prints the first ten Fibonacci numbers.' \
  --chat-template auto --thinking-mode suppressed --max-tokens 512 --temperature 0.0

gated_gqa softplus yarn_global / rotary_sliding mixed Q4/Q6 experts

Q4_K_M ★ Q6_K experts GGUF

Instella-MoE 16B-A3B

MoE + gated MLA · partial interleaved YaRN RoPE · RMSNorm · SwiGLU
Sigmoid group-limited top-k router + shared experts (64 experts/layer)
FarSkip two-stream residuals · BPE

BF16 safetensors runtime certified (PR #389) — coherent chat, top-1 matches PyTorch at the tested checkpoint

Full-logit cosine 0.99998 vs PyTorch at a 32-token X-Ray checkpoint (30 GiB real runtime)

Quantized GGUF and long trajectories not yet certified

Bring-up history: fail-closed MoE bring-up (#292), mechanical FarSkip circuit stitching (#294), nightly circuit contracts (#295), BF16 execution certification (#389). A high-memory Instella E2E lane remains follow-up work. Contract lane: tests.test_v8_instella_moe_bringup (runs in the nightly demo-readiness profile).

Commands & runnable lanes

Instella contract lane

.venv/bin/python -m unittest tests.test_v8_instella_moe_bringup
# nightly: registered as v8_instella_moe_circuit_contracts

gated_mla farskip_residuals BF16 certified

BF16 ★ safetensors → BUMP MoE 64 experts

Kimi / MLA Text Decoder

Multi-Head Latent Attention + MoE · partial pairwise-concat RoPE · RMSNorm
SwiGLU dense + sigmoid group-limited top-k routed/shared experts · tiktoken · untied LM head

Text decoder runtime certified (PR #403) — Kimi-VL-A3B-Instruct BF16, coherent on AVX2 and AVX-512

Repeatable: three Ryzen runs bit-identical (32-token SHA-256); PyTorch first-token + top-20 match, full-logit cosine 0.9977

MoonViT vision bridge / multimodal path — outside the certified lane

Certification note (PRs #399, #403): tokenizer source assets + chat protocol (kimi_vl) are now staged by the runner; decode inserts mla_kv_cache_store and switches mla_attention to deepseek_mla_attention_decode_f32; prefill uses mla_kv_cache_batch_store. Internal BF16 drift becomes material near layer 7 — do not claim bit-exact internal parity.

explicit MLA decode cache BF16 kv_lora path template contract tested

Commands & runnable lanes

Laptop-safe contract check

.venv/bin/python -m py_compile version/v8/scripts/build_ir_v8.py
make build/libckernel_engine.so
.venv/bin/python unittest/test_deepseek_reference_kernels.py
.venv/bin/python -m unittest tests.test_v8_kimi_template

Commands & runnable lanes

Kimi high-memory lane

V8_KIMI_MODEL=/path/to/Kimi-VL-A3B-Instruct make test-v8-kimi-highmem
# requires ≥ 40 GiB RAM host; context 2048, 64 tokens

BF16 FP32 reference MLA cache MoE

Gemma4 E4B IT

RMSNorm · split-half direct RoPE · QK‑Norm · GeGLU
Hybrid full/sliding attention · per-layer theta · SentencePiece

Quantized inference — GGUF Q4_K_M coherent long-answer smoke

BF16 safetensors → BUMP — PyTorch parity bring-up path

Vision bridge smoke — Gemma4 mmproj-F16 encoder → mixed decoder prefill

Bring-up fix: Gemma4 keeps the split-half RoPE channel layout, but it cannot use one global cached split-half contract. The template exports per-layer/direct RoPE metadata, and lowering selects rope_forward_qk_split_direct_f32 from IR data rather than hard-coding a Gemma-only kernel.

rope_param_mode: per_layer_direct BF16 PyTorch parity lane GGUF Q4_K_M smoke-tested vision bridge smoke-tested

Commands & runnable lanes

Latest hardened v8 text smoke

version/v8/scripts/cks-v8-run run \
  hf://unsloth/gemma-4-E4B-it-GGUF/gemma-4-E4B-it-Q4_K_M.gguf \
  --context-len 2048 --force-compile --force-convert --chat-template=gemma4 \
  --prompt 'Give me a detailed example of C code.' \
  --max-tokens 256 --temperature 0.0 --generate-visualizer

Commands & runnable lanes

Gemma4 vision bridge smoke

version/v8/scripts/cks-v8-run run \
  hf://unsloth/gemma-4-E4B-it-GGUF/gemma-4-E4B-it-Q4_K_M.gguf \
  --mmproj hf://unsloth/gemma-4-E4B-it-GGUF/mmproj-F16.gguf \
  --image-path version/v8/test_assets/v8_vision_doc_card_72.ppm \
  --prompt 'Explain this image in one short paragraph.' \
  --context-len 1024 --chat-template=gemma4 \
  --max-tokens 8 --temperature 0.0

Q4_K_M ★ BF16 safetensors Q4_K Q6_K

Nemotron Nano 9B v2

Nemotron-H · Mamba2 recurrent blocks · sparse attention layers
ReLU2 MLP · Q5_0/Q4_K/Q8_0 GGUF · BPE tokenizer

Quantized inference — GGUF Q4_K_M coherent C-code smoke

Safetensors/PyTorch parity lane — Mamba2 stitching and state-shape guardrails

Bring-up fix: Nemotron-H uses Mamba2 state shaped as [heads, head_dim, state_dim], not a square DeltaNet-style recurrent matrix. The template/lowering path now carries that contract explicitly, stores no-RoPE attention KV after v_proj, and generated BPE runtimes can encode raw prompts end-to-end by default.

Mamba2 state shape explicit native BPE encode default-on GGUF Q4_K_M smoke-tested

Commands & runnable lanes

Latest hardened v8 smoke

version/v8/scripts/cks-v8-run run \
  hf://bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q4_K_M.gguf \
  --context-len 1024 --force-compile --force-convert --chat-template=auto \
  --prompt 'Give me a detailed example of C, Python and SQL code.' \
  --max-tokens 256 --temperature 0.0 --generate-visualizer

Q4_K_M ★ Q5_0 Q8_0 BF16 safetensors

Llama / Nanbeige

RMSNorm · RoPE pairwise · SwiGLU · GQA
Untied LM head supported · SentencePiece · ChatML markers via GGUF

Quantized inference — active GGUF bring-up lane on Nanbeige 4.1 3B

v7 training — not yet a documented preset; use template-audit + parity gates first

Current bring-up lane keeps SentencePiece + ChatML on the C-tokenizer path, preserves untied output.weight, and treats Nanbeige long coherent think traces as model behavior instead of a kernel/parity failure.

untied output.weight ChatML auto template template-audit first

Commands & runnable lanes

Latest hardened v8 smoke

version/v8/scripts/cks-v8-run run \
  hf://mradermacher/Nanbeige4.1-3B-GGUF/Nanbeige4.1-3B.Q4_K_M.gguf \
  --context-len 1024 --force-compile --force-convert --chat-template=auto \
  --generate-visualizer

Q4_K_M Q8_0 FP32

GPT‑2

LayerNorm · Learned PosEmb · GELU
QKV bias · tied embeddings · BPE tokenizer

Quantized inference — GGUF baseline

v7 training — not yet in scope

Q4_0 Q4_1 Q5_0 Q8_0 FP32

Gemma4 Vision

SigLIP-style ViT encoder · XY position embeddings · average-pool merge
projector prep → Gemma4 decoder bridge · deterministic patch frontend

v8 multimodal inference — patch frontend, encoder graph, projector bridge

Bring-up lane — staged behind the promoted Qwen3-VL baseline

Vision lane scope: shares the same lowering + memory-planning stack as Qwen3-VL. Uses position_embeddings_add_gemma4v_xy and spatial_average_pool_contiguous for the Gemma4 merge path instead of Qwen3-VL's 2×2 spatial merge.

gemma4v_xy_posemb avgpool_merge projector_bridge

Q4_K_M decoder ★ Q8_0 mmproj FP32 encoder image bridge

SigLIP ViT (encoder)

Standalone vision tower · im2patch → patch embed · LayerNorm · full attention
learned position embeddings · GELU MLP · reusable encoder contract

v8 encoder graph — patch frontend, transformer body, projector footer

Encoder-only — pairs with a decoder template via the bridge contract

Why it exists: SigLIP is the shared ViT backbone behind several vision-language families. Templating it on its own keeps the encoder reusable, so a new VLM can be brought up by swapping the decoder + projector instead of rewriting the vision tower.

reusable_vit im2patch encoder_contract

FP32 BF16 encoder prefix

Kernel Families by Model

Model

Tokenizer

QKV Bias

QK‑Norm

RoPE / PosEmb

Attention

MLP

Post‑Norms

Logits

GPT‑2

BPE

Yes

Learned

Causal

GELU

Tied

Qwen2

BPE

From weights

RoPE split-half

Causal

SwiGLU

Tied

Qwen3

BPE

Yes

RoPE split-half

Causal

SwiGLU

LM head

Qwen3.5

BPE

Full‑attn only

RoPE

Hybrid DeltaNet + Gated

SwiGLU

Tied

Qwen3-VL

BPE + image markers

Yes

RoPE + vision pos

Vision bridge + causal

SwiGLU

Bridge norms

LM head

GLM4

BPE

Attention bias

Partial pairwise RoPE

Causal GQA

SwiGLU

LM head

Gemma3

SentencePiece

From weights

Yes

RoPE

Sliding

GeGLU

Yes

Tied

Gemma4 E4B

SentencePiece

From weights

Yes

Split-half direct

Full + sliding

GeGLU

Yes

Tied

Nemotron-H

BPE

From weights

No-RoPE KV + sparse attn

Mamba2 + sparse attn

ReLU2

LM head

Qwen3.8

BPE

Full‑attn only

M-RoPE text split

Hybrid DeltaNet + gated

SwiGLU

Tied

Kimi (MLA)

tiktoken

Partial pairwise concat

MLA + MoE

SwiGLU + routed MoE

Untied

Cohere2

BPE

Pairwise, sliding only

3:1 sliding + full GQA

SwiGLU (parallel)

LayerNorm shared

Tied + logit_scale

Laguna

BPE

Yes

YaRN global / rotary sliding

Gated GQA (softplus)

SwiGLU MoE top‑8 + shared

Untied

Instella

BPE

Partial interleaved YaRN

Gated MLA

SwiGLU MoE + shared

Untied

Llama / Nanbeige

SentencePiece

From weights

RoPE pairwise

Causal GQA

SwiGLU

Untied head

Whisper Tiny / Base / Small

BPE + task tokens

From weights

Learned absolute

Self + cross (enc–dec)

GELU (erf)

Tied

Whisper Tiny, Base, and Small share this encoder–decoder row: the encoder runs full self-attention over 1500 audio frames, the decoder adds the cross-attention lane (K/V from encoder memory, computed once) on top of causal self-attention. Position is a learned absolute table on both sides — no RoPE anywhere. Kernel-level detail: Concepts — Audio Frontend & Cross-Attention and Whisper Audio End-to-End.

IR + Quant Contracts

Model

Embed Scale

Prefer Q8 Contract

Prefer FP32 Logits

Tokenizer I/O

Attention Kernel

MLP Kernel

Residual Save

LM Head Tie

GPT‑2

BPE + vocab

attn

gelu

Yes

Qwen2

BPE + vocab

attn

silu_mul

Yes

Qwen3

BPE + vocab

attn + qk_norm

silu_mul

Yes

Qwen3.5

BPE + vocab

hybrid_recurrent + gated_attn

silu_mul

Yes

Qwen3-VL

BPE + image markers

vision_bridge + attn

silu_mul

Yes

GLM4

BPE + vocab

attn + partial_rope

silu_mul

Yes

Gemma3

sqrt(dim)

Yes

SentencePiece

attn_sliding

geglu

Yes

Gemma4 E4B

embed + per-layer

Yes

SentencePiece

attn + attn_sliding

geglu

Yes

Nemotron-H

Yes

BPE + vocab

mamba2 + sparse_attn

relu2

Yes

Qwen3.8

BPE + vocab

hybrid_recurrent + gated_attn

silu_mul

Yes

Kimi (MLA)

tiktoken + sidecar

deepseek_mla_attention

silu_mul + moe

Yes

Cohere2

Yes (logit_scale)

BPE + vocab

attn + attn_sliding

silu_mul

Yes

Laguna

BPE + vocab

attn + attn_gate_softplus

silu_mul + moe

Yes

Instella

BPE + vocab

deepseek_mla_attention (gated)

silu_mul + moe

Yes

Llama / Nanbeige

SentencePiece + ChatML

attn

silu_mul

Yes

Supported Quant + DType Coverage

Weight-only quantized GEMM/GEMV kernels with BF16/FP32 activations and Q8_* activation contracts. Quantized kernels (Q4_K, Q5_K, Q6_K, Q8_0) are verified against both PyTorch and llama.cpp reference output — both references must agree before a kernel is considered validated.

Q4_0 Q4_1 Q4_K Q5_0 Q5_1 Q5_K Q6_K Q8_0 Q8_K BF16 FP32

GEMV: q4_k, q5_0, q5_1, q5_k, q6_k, q8_0, fp32

GEMM: q4_k, q5_0, q5_k, q6_k, q8_0, fp32

Activations: fp32, bf16, q8_0, q8_k

Fused: attention + outproj + MLP (prefill) variants

Recurrent: src/kernels/deltanet_kernels.c for FP32 Gated DeltaNet parity

Hybrid Architecture Support

Qwen3.5 / qwen3next — Full Template
Full qwen35.json template with hybrid block pattern: 3×recurrent → 1×full_attention. Recurrent blocks: x → [q,k,v,z] + [beta,alpha] → conv(q/k/v) → DeltaNet state update S_t → RMSNorm(h) * SiLU(z) → outproj. Full-attention blocks: q_gate_proj → split_q_gate → k/v_proj → qk_norm → rope → gated_attention → sigmoid_mul → outproj. Dedicated kernel parity in src/kernels/deltanet_kernels.c (forward + backward). Compatible with Qwen3.5-0.8B dense variant.

Gated attention
Treat it as a composed graph: causal attention + sigmoid gate multiply + output projection. Keep it stitched in CK unless profiling proves a fused epilogue is worth the extra maintenance.

Interpretation: DeltaNet earns a dedicated kernel because it mutates recurrent state; gated attention usually does not, because the gate is just an elementwise epilogue on top of otherwise standard attention.

Kernel Delta: What the New Families Required

Each newly supported family forced the kernel registry to grow. This map shows the kernel families added or hardened per architecture — every entry is a named provider in version/v8/kernel_maps with a source file you can read.

click / tap the diagram to expand

Counts and provider status live in version/v8/kernel_maps/KERNEL_REGISTRY.json (308 provider maps); the audit baseline is version/v8/contracts/kernel_interface_migration_baseline.json. For the math behind each kernel family see v8 Kernel Architecture; for what each new model variant changed see Architecture Variants. Provider-level deep dives: MoE Expert Kernels covers the Laguna mixed-quant routed/shared experts and the Qwen3.5-MoE bucketed prefill; Cohere Kernel Story covers the Cohere2 reuse bindings and the final_logit_scale_f32 footer.

Why Build C Kernel Engine?

What this teaches
This stack forces us to understand the invariants that actually matter: memory layout, kernel contracts, numerical stability, and graph scheduling. It turns the model into something we can inspect — dump activations, trace tensor ranges, validate gradients later — without guessing what’s happening inside a framework. That clarity is the foundation for training on CPU.

Ocean depth metaphor: surface chaos (models, agents, APIs, best practices changing weekly) vs stable bedrock (kernels, memory hierarchy, linear algebra, physics)

Click to open full‑size SVG

Sources: calculus history (17th century), matrix theory formalization (19th century), AI as a field (1956), backpropagation popularization (1986).

Current GGUF Test Corpus

gpt2 (baseline)

Qwen--Qwen2-0.5B-Instruct-GGUF

Qwen--Qwen3-0.6B-GGUF

unsloth--gemma-3-270m-it-GGUF

unsloth--gemma-4-E4B-it-GGUF / local Q4_K_M BUMP

mradermacher--Nanbeige4.1-3B-GGUF

Qwen--Qwen3.5-0.8B-GGUF

Qwen--Qwen3-VL-8B-Instruct-GGUF + mmproj

bartowski--nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF / Q4_K_M

unsloth--GLM-4-9B-0414-GGUF / Q4_K_M

Audio lane: openai/whisper-tiny and openai/whisper-base safetensors → BUMP as generated encoder + decoder runtime pairs, validated through opt-in transcript and PyTorch parity gates rather than the GGUF corpus above.

Bring-up note: if a Llama-family/Nanbeige first reply starts with <think> or echoes <|im_start|>assistant, the runtime is using the wrong chat wrapper or stop policy. Preserve GGUF-exported ChatML markers and keep output.weight untied. For Gemma, if first-token parity collapses after a clean qk_norm stage, check the exported rope_layout before suspecting tokenizer or stop-token bugs.

Recent Bring-up Fixes

Gemma 3: the bug was not in tokenization. We were applying RoPE with the wrong pairing style for Q/K. Gemma expects split-half RoPE, but the runtime could still use the other layout. Fix: read exported rope_layout and choose the matching RoPE kernel automatically.

Gemma 4: the BF16 safetensors lane proved the circuit against PyTorch first, then the GGUF lane reused the same generic split-direct RoPE op. A 1024-token Q4_K_M C-code prompt now stays coherent; remaining work is performance and tighter quantized parity, not basic stitching.

Qwen3.5: hybrid recurrent-attention decode and Gated DeltaNet parity are now on the normal C-tokenizer route. <think> markers are preserved as special tokens, and visible vs. suppressed thinking is now a generic chat-contract mode instead of ad hoc runtime logic.

Llama / Nanbeige: SentencePiece + ChatML bring-up stabilized with untied output.weight preserved. Nanbeige remains an active inference lane; long coherent think traces are treated as model style, not as evidence of kernel breakage.

Nemotron Nano 9B v2: the first real failure was stitching, not tokenizer or final head: Mamba2 state shape and no-RoPE KV placement had to be explicit in IR. Native BPE encode is now default-on so ck_run_v8.py raw prompts work end-to-end; token-id-only fallback remains available with CK_DISABLE_FULL_BPE_TOKENIZER=1.

GLM4: v8 now has a template, GLM4 GGUF metadata fixes, declarative safetensors mapping, PyTorch-vs-CK parity harnesses, and a coherent Q4_K_M GGUF runtime smoke. The important fix was separating semantic stream sources in the template from physical quantized activation buffers in the kernel ABI.

Qwen3-VL / Gemma4 vision: Qwen3-VL remains the promoted vision baseline. Gemma4 vision now has a matching CLIP/mmproj conversion path, a Gemma4 image marker contract, and a confirmed encoder-prefix-to-decoder mixed-prefill smoke; semantic parity and speed remain active work.

#399/#403 — Kimi text decoder runtime certified: tokenizer sidecar staging, kimi_vl chat protocol, BF16 KV-LoRA decompression binding; MoonViT vision bridge still outside the certified lane.

#401 — Cohere2 Command R: LayerNorm shared pre-block norm, parallel attention+MLP residual, sliding-only RoPE, model-declared logit_scale footer; 8-token llama.cpp replay agreement on Q4_K_M.

#404 — Laguna-XS 2.1 quantized text runtime: gated GQA softplus gate, YaRN-global/full-rotary-sliding RoPE split, mixed Q4/Q6 MoE expert storage.

Validation Commands (v7 inference matrix)

These commands are the current inference readiness gates for Qwen2, Qwen3, Qwen3.5, Gemma3, and the active Nanbeige bring-up lane.

python3 version/v7/scripts/validate_model_matrix_v7.py
python3 version/v7/scripts/validate_parity_matrix_v7.py
python3 version/v7/scripts/validate_long_decode_stability_v7.py

# Nanbeige-specific stitched call-IR regression
python3 version/v7/scripts/test_kv_cache_batch_copy_call_ir_v7.py \
  --run-dir ~/.cache/ck-engine-v7/models/mradermacher--Nanbeige4.1-3B-GGUF

Matrix validators honor cached GGUFs by default; add --allow-download when you want them to fetch missing models.

Growth path: inference coverage now spans Qwen2/Qwen3/Qwen3.5/Qwen3.6/Qwen3.8/Qwen3-VL/Gemma3/Gemma4 text, Gemma4 vision bridge smoke support, Nemotron-H, GLM4, Kimi (MLA text decoder), Cohere2, Laguna, and Instella, plus the Llama/Nanbeige bring-up lane. Qwen3.5 adds hybrid recurrent-attention coverage with Gated DeltaNet kernels (forward + backward), compatible with the 0.8B dense variant. Gemma now runs on the correct split-half RoPE parity path, Gemma4 adds per-layer direct RoPE plus a safetensors-to-BUMP parity lane, Nemotron-H adds Mamba2 recurrent state coverage, GLM4 adds partial-RoPE/BPE conversion contracts, and Nanbeige has a documented SentencePiece/ChatML bring-up lane on the C tokenizer. v7 still targets full training expansion: backward kernels, optimizer state, gradient reduction, and IR‑driven training schedules. This page will keep evolving into a single matrix that shows inference + training coverage per model family.

References: Britannica — Calculus, MacTutor — Matrices and determinants, Dartmouth AI 1956, Nature 1986 — Backpropagation