Kernel Catalog

This document lists the high-performance C kernels available in the engine. These kernels are designed to be "micro-libraries"—you can copy src/kernels/rope_kernels.c into your own project without taking the rest of the engine.

Registry snapshot: the v8 provider registry (version/v8/kernel_maps/KERNEL_REGISTRY.json) currently holds 364 provider maps across 126 ops (commit f7aedf6e4, 2026-09-12). The tables below are a curated catalog, not the full census — for the registry organized by operation family with selection rules, see v8 Kernel Architecture and Kernel Maps and Provider Selection.

Naming Convention

1. Attention, RoPE & Recurrent State

Composed vs Fused Gates

Not every gated block needs its own standalone kernel file.

• Gated attention is still normal causal attention plus a gate epilogue: attention -> sigmoid -> mul -> outproj. CK can express that as stitched kernels without inventing a dedicated gated_attention_kernels.c.
• SwiGLU / GeGLU follow the same rule. We keep the atomic activation kernels and only use fused decode helpers where profiling justifies the extra complexity.
• Gated DeltaNet is different because the gate participates inside the recurrent state update itself. That makes src/kernels/deltanet_kernels.c a real kernel family, not just a wrapper around GEMM/GEMV.

Rule of thumb: keep CK graph-first and stitch atomic kernels by default; add fused wrappers only when measurements show the gate epilogue is a real hotspot.

Qwen3.5 DeltaNet Block Flow

The Qwen3.5 / qwen3next linear-attention block is easier to reason about if you separate the three different gating roles instead of collapsing them into one "state-space" idea.

1. Project from the token input x. The block produces [q, k, v, z] from ssm_in(x) and [beta, alpha] from ssm_beta_alpha(x).
2. Run the short convolution path. The local conv cache mixes the projected q/k/v lanes before the recurrent update. The z, beta, and alpha lanes do not go through that conv kernel.
3. Turn alpha into the recurrent decay gate. In llama.cpp this becomes a decay control g via softplus(alpha + dt) followed by a learned scale. That gate controls how much of the old recurrent state survives.
4. Update the per-head memory matrix S_t. DeltaNet keeps a recurrent state matrix [Head, Dim, Dim], not just a hidden state vector.
5. Normalize and gate the recurrent output. After the DeltaNet core produces h_t, the block applies gated normalization: RMSNorm(h_t) * SiLU(z).
6. Project back out. The gated normalized result goes through ssm_out to rejoin the residual stream.

For single-token decode, the recurrent update that CK matches in src/kernels/deltanet_kernels.c is:

q_hat   = norm(q) / sqrt(d)
k_hat   = norm(k)
beta_s  = sigmoid(beta)
gate    = exp(g)

S_decay = gate * S_prev
kv_mem  = S_decay^T * k_hat
delta   = beta_s * (v - kv_mem)
S_new   = S_decay + outer(k_hat, delta)
out     = S_new^T * q_hat

Interpretation:

• g controls forgetting / decay of old memory.
• beta controls how aggressively the block writes the correction term.
• z is the output gate applied after the recurrent core, during gated normalization.

This is why Qwen3.5 feels Mamba-like without being "just Mamba": it has a short convolution path and a recurrent cache, but the core memory update is a DeltaNet matrix-state rule, and Qwen3.5 alternates these DeltaNet blocks with separate full Gated Attention blocks.

2. Elementwise & Activation

3. Normalization

4. GEMM (Matrix Multiplication)

5. Loss & Training Kernels

Audio Frontend (Cohere Transcribe)

The Transcribe audio frontend turns a WAV file into encoder-ready features with the following providers; the full pipeline story is in the Cohere Kernel Story — Transcribe audio stack.

Memory Layouts

Most kernels assume Row-Major contiguous memory unless specified otherwise.

• Matrices: [Rows, Cols]
• Attention Tensors: [Heads, Tokens, Head_Dim] (Head-Major) helps with cache locality during the attention loop.

Catalog refreshed against version/v8/kernel_maps/KERNEL_REGISTRY.json (364 provider maps across 126 ops) at commit f7aedf6e4 (2026-09-12). Provider counts are a hand-maintained snapshot until the docs build regenerates them; the registry JSON is the source of truth.