Cohere Kernel Story: Reuse, BF16 Vision, and the Transcribe Audio Stack

Four Cohere bring-ups, four different relationships to the kernel registry: Command R mostly composed existing providers, North composed them entirely, North Micro Vision added a BF16 image frontend, and Transcribe added a seven-provider audio foundation whose encoder is still not promoted. This page walks the kernels each one touched — and is explicit about what is certified versus what is still a provider foundation.

1. Cohere2 Command R (PR #401): the reuse story

Command R is structurally novel — a single shared pre-block LayerNorm feeds the attention and MLP branches in parallel, layers alternate sliding and full attention 3:1, the LM head is weight-tied, and per-tensor quant is mixed through the weight_dtype_registry (the weight policy simply ignores the duplicate ln2_* tensors the checkpoint carries). Yet the circuit version/v8/circuits/cohere2.json binds almost entirely to the existing Llama-family provider stack:

cohere2.json kernel bindings (all pre-existing except the last): layernorm = layernorm_fp32_exact rope_init = rope_precompute_cache_llama_cpu rope_qk = rope_forward_qk_pairwise (+ _llama_cpu decode variant) attn = attention_forward_causal_head_major_gqa_flash_strided attn_sliding = attention_forward_causal_head_major_gqa_flash_strided_sliding attn_decode = attention_forward_decode_head_major_gqa_flash[_sliding] final_logit_scale = final_logit_scale_f32 <-- the one new kernel

The single new provider is final_logit_scale_f32 (src/kernels/logit_kernels.c:7): an in-place FP32 logits[i] *= logit_scale footer op, driven by GGUF metadata rather than a hardcoded constant. Everything else was already hardened for other families.

click / tap the diagram to expand

The block-structure diagram lives in Architecture Variants — Cohere2; provider math in kernel architecture — norms, footer ops.

2. Cohere North (MoE, PR #426): zero new C kernels

North (North Mini Code, cohere2_moe.json) is a Cohere2-family MoE: alternating full/sliding attention, a leading dense layer, sigmoid top-8 routing, tied embedding head. It required no new C kernels at all — the circuit composes rmsnorm_forward_llama_production, the pairwise RoPE pair, the flash attention family, group_limited_topk_router_sigmoid_f32, moe_swiglu_expert_forward_q4k_q5k (Q8/Q4_K/Q5_K expert providers from the manifest), and the final_logit_scale_f32 footer added for Command R. The expert kernels themselves are documented in MoE Expert Kernels; the declarative layer plan (no Cohere family branches in lowering or codegen) was the actual work of PR #426.

3. North Micro Vision / Compass (PR #434): the BF16 frontend

North Micro Vision keeps Cohere2 text semantics and adds a Qwen3-VL-derived vision tower, expressed as three circuits — cohere_compass.json (stitch), cohere_compass_text.json, cohere_compass_vision.json — with BF16 storage throughout. The kernel delta:

patch_projection_image_bf16_native_storage
src/kernels/gemm_kernels_bf16.c:1835. Thread-pooled image→patch embedding projection: temporal-2 weight pair, merge-aligned patch grid, BF16 native storage end to end (no fp32 round-trip between projection and encoder).

gelu_erf_bf16_storage
src/kernels/gelu_kernels.c:609. Exact-erf GELU (not the tanh approximation) on BF16-stored activations — the vision tower's activation contract.

BF16 GEMV/GEMM parallel dispatch.
gemv_bf16 gained gemv_bf16_parallel_dispatch (gemm_kernels_bf16.c:646) — thread-pool row-partitioned BF16 GEMV for decode — and gemm_nt_bf16_bf16_storage gained a parallel dispatch variant (gemm_kernels_bf16.c:1905, +211 lines) with exact BF16 round-trip storage semantics: outputs are rounded to BF16 exactly where the contract says they are.

Evidence (PR #434): the official 577-tensor checkpoint header maps with zero unowned tensors; real-manifest text prefill, text decode, and vision lowering emit zero call-ABI errors. Caveat: public compatibility is not claimed until real-weight image and text generation are certified.

4. Cohere Transcribe (PRs #433, #436, #441): the audio stack

Status honesty. The seven providers below are a provider foundation: each carries a kernel map, a map-owned call ABI, and a numerical execution contract, and each contract is test-validated. None of them is a selection-managed production provider, no circuit consumes them yet, and Cohere Transcribe end-to-end transcription is not certified. The Conformer encoder attention, the cross-attention decoder, and the tokenizer are still blocked from promotion.

All seven live in src/kernels/audio_kernels.c and form the Conformer frontend:

Provider	Source	Contract	Status
`audio_preemphasis_f32`	`audio_kernels.c:303`	pre-emphasis FIR filter, y[t] = x[t] - k*x[t-1]	provider foundation
`audio_stft_power_centered_window_f32`	`audio_kernels.c:498`	centered, windowed STFT power spectrum	provider foundation
`audio_log_mel_time_major_f32`	`audio_kernels.c:553`	log-mel filterbank energies, time-major layout	provider foundation
`audio_feature_normalize_per_feature_f32`	`audio_kernels.c:328`	per-mel-bin mean/variance normalization	provider foundation
`audio_conv2d_whc_grouped_f32`	`audio_kernels.c:1025`	grouped conv2d subsampling frontend (WHC layout)	provider foundation
`audio_glu_split_channel_major_f32`	`audio_kernels.c:1115`	GLU split-gating after the frontend conv	provider foundation
`audio_relative_shift_f32`	`audio_kernels.c:1173`	relative-position shift for Conformer rel-pos attention (attn.pos_bias_u/v); exactly maps raw[h,q,T-1+k-q] to scores[h,q,k]	provider foundation

The model contract (PR #436) validates the real checkpoint generically: a cohere-transcribe GGUF is a 48-layer Conformer encoder (rel-pos attention, depthwise conv + batchnorm, GLU; 39 tensors per block) plus an 8-layer cross-attention decoder (26 tensors per block), 2,104 tensors matched exactly — and conversion then emits an explicit HARD MODEL CONTRACT FAULT instead of entering a partial runtime. PR #441 added the oracle harness version/v8/scripts/certify_cohere_transcribe_oracle_v8.py, which captures stage-wise X-Ray dumps so each frontend stage can be certified against the reference before the encoder is promoted.

click / tap the diagram to expand

The certified audio lane (Whisper, production) is a different stack: Audio Kernels Deep Dive and Whisper Tiny End-to-End.

5. The pattern

Cohere is the cleanest demonstration of how the registry is supposed to absorb a new family: Command R needed exactly one kernel, North needed zero, Compass needed a BF16 frontend, and only Transcribe — a genuinely new modality with a Conformer frontend — needed a provider family. The cost of a new model is proportional to its contract novelty, not its parameter count, and the fail-closed boundary (provider foundation → certified runtime) is what keeps "we have the kernels" from being read as "we run the model".