C-Kernel-Engine
High-Performance ML Kernels

Architecture Variants

Every family enters through the same pipeline — GGUF or safetensors artifact, template, lowering, memory plan, codegen. Most families differ only in contract values. This page documents the families that differ in contract structure: the ones that forced new kernel families into the registry. Each section links to the provider-level math in v8 Kernel Architecture.

Already covered by dedicated deep dives

Kimi / MLA. Latent KV cache contract, prefill/decode store ops, and the text-decoder certification evidence: v8 MLA / Kimi.
Qwen3.5 / 3.6 / 3.8 hybrid recurrent. Gated DeltaNet block math, forward and backward kernels: Gated DeltaNet Deep Dive. Qwen3.8's addition is identity, not structure: upstream GGUFs advertise architecture=qwen35 and the converter selects the qwen38 circuit from artifact metadata.
Whisper. The generated audio frontend and encoder–decoder pair: Audio Kernels Deep Dive and Whisper end to end.
Laguna / Qwen3.5-MoE experts. The provider-level walk of the bucketed expert prefill and the mixed-quant compact experts: MoE Expert Kernels.
Cohere family. Command R provider reuse, North MoE composition, Compass BF16 vision, and the Transcribe audio foundation: Cohere Kernel Story.

Laguna — gated GQA, split RoPE, mixed-quant MoE

Laguna-XS 2.1 (PR #404) combines three structures that previously existed only separately: a softplus per-head attention gate, two RoPE contracts in one model, and MoE layers on top of sliding attention with mixed Q4/Q6 expert storage.

Layer kinds (in order) dense_global_attention leading dense block · full attention · dense MLP moe_global_attention full attention + routed/shared MoE moe_sliding_attention sliding window + routed/shared MoE Attention contract Gated GQA — softplus head gate out[h,c] = attn[h,c] · softplus(gate[h]) attn_gate_softplus_mul_forward per-head scalar · linear branch > 20 Two RoPE contracts, one model global layers → YaRN (theta 8M family) sliding layers → full rotary per-layer head dims: 48 vs 64 · rotary 64 vs 128 MoE contract Sigmoid router + correction bias bias steers selection; weights stay raw top-8 routed experts · Q4 storage shared SwiGLU experts · Q6 storage mixed Q4/Q6 in one model: map-owned per-tensor providers qk_norm on · RMSNorm · untied head Evidence (PR #404): Q4_K_M coherent text E2E · embedding and layer-0 RMSNorm bit-exact · router replay within one ULP. Caveats: long-trajectory parity not claimed (near-tied MoE route flips) · softplus gate observed, not oracle-validated.
click / tap the diagram to expand

Provider math: kernel architecture — attention gates, MoE routers, RoPE. Operator lane: v8 runbook. Kernel-level walk of the mixed-quant routed/shared experts and the parallel workspace variants: MoE Expert Kernels.

Cohere2 — parallel branches, shared LayerNorm, logit scale

Command R (PR #401) is the first family where attention and MLP read the same normalized tensor and add both results back, and the first to declare a model-level logit scale.

Block structure x LayerNorm (shared) one norm · synthesized beta Attention (GQA) 3 sliding : 1 full · RoPE sliding-only MLP (SwiGLU) reads the same normed tensor x + attn + mlp single residual update Footer contract final_logit_scale_f32 logits ×= model-declared logit_scale (GGUF metadata, not a constant) Evidence (PR #401): Command R7B Q4_K_M — tokenizer-free replay matches llama.cpp for the first 8 greedy positions. Caveats: long-trajectory parity and performance sweeps pending.
click / tap the diagram to expand

The shared norm makes the parallel branches a contract, not a fusion decision: both branches must consume the identical normed tensor, which the circuit declares and the memory plan shares. Provider math: kernel architecture — norms, footer ops. The provider-level reuse story (one new kernel, six reused bindings): Cohere Kernel Story.

Instella — gated MLA with FarSkip residual streams

Instella-MoE 16B-A3B (PRs #292/#294/#295, certified BF16 in #389) combines Kimi-style latent attention with a gate, and replaces the single residual highway with two coordinated streams.

Block structure Gated MLA sigmoid(gate) · mla_attention latent KV cache · kv_lora decompress MoE sigmoid group-limited top-k 64 routed + shared SwiGLU experts FarSkip residual two streams: main + routed-free stitched mechanically, fail-closed Position contract Partial interleaved YaRN rotary subset · interleaved pairing · YaRN long-context scaling BF16 safetensors safetensors → BUMP · no GGUF lane yet Evidence (PR #389): coherent chat E2E · top-1 matches PyTorch at the tested checkpoint · full-logit cosine 0.999976927 (32-token X-Ray checkpoint, 30 GiB runtime). Caveats: quantized GGUF and long trajectories uncertified; high-memory E2E lane is follow-up work.
click / tap the diagram to expand

MLA provider math: kernel architecture — MLA and v8 MLA / Kimi. The FarSkip shared-combine provider (farskip_swiglu_shared_combine_bf16) is candidate status.

Nemotron-H — Mamba2 state, sparse attention, ReLU2 experts

Nemotron Nano 9B v2 mixes three block types. The contract that mattered most: Mamba2 state is [heads, head_dim, state_dim], not a square DeltaNet-style matrix — the template carries that shape explicitly instead of reusing the DeltaNet kernels.

Block kinds Attention (no RoPE) sparse attention layers KV stored after v_proj Mamba2 selective scan · dt softplus+clamp state [heads, head_dim, state_dim] MoE — ReLU2 experts group-limited top-k router moe_relu2_expert_forward/backward Why DeltaNet kernels were not reused DeltaNet: S is a square per-head state matrix updated by outer products. Mamba2: state is per-(head, head_dim, state_dim) with a diagonal exp(dt·a) decay — different shape, different update, different kernels. Evidence: GGUF Q4_K_M coherent text E2E; safetensors/PyTorch parity lane covers Mamba2 stitching and state-shape guardrails.
click / tap the diagram to expand

Mamba2 update math and the decode contract: concepts — Nemotron-H hybrid circuit; provider list: kernel architecture — recurrent/SSM.

Throughput follow-up (PR #412): recurrent prefill is no longer serial — mamba2_conv1d_f32_parallel_dispatch partitions the conv by independent channel ranges and mamba2_selective_scan_f32_parallel_dispatch chunks by independent head ranges, with map-owned shape constraints (impl.variants[].parallel_head_ranges: T≥8, H≥2), a named sequential fallback, and the CK_DISABLE_MAMBA2_PARALLEL_PREFILL kill switch. Measured 1.87× on 8K-token Nemotron prefill with exact output agreement; provider-level detail in MoE Expert Kernels — Throughput kernels beyond MoE.

Image
100% | |
Scroll to zoom | Drag to pan | W/H to fit | 0 to reset | ESC to close