Back to Quantization Deep Dive

Quantization Formats Visual Guide

Byte-level structure, headers, and dequantization math for the formats CKE executes

Where These Formats Run Today

v8 is the primary engine. All current model families — Gemma 3/4, Qwen 2 / 3 / 3.5 / 3.6 / 3.8, GLM-4, Nemotron, Kimi, Cohere2, Laguna, Instella-MoE, Nanbeige, GPT-2, Qwen3-VL / Qwen3.6-VL vision, and Whisper audio — run as v8 generated runtimes over the GGUF formats below plus first-class BF16 storage. v7 remains the training lane (FP32 forward/backward); its quantized inference path is superseded by v8. The byte layouts are identical in both engines because both are parity-gated against llama.cpp.

Dense GGUF lanes

Q4_K_M, Q5_K_M, and Q8_0 checkpoints across Gemma, Qwen, GLM-4, Nemotron, and Nanbeige. Mixed dispatch: Q4_K/Q5_K/Q6_K weights with Q8_K or Q8_0 quantized activations.

MoE mixed-quant dispatch

Qwen3.5 35B-A3B and Instella-MoE route experts through mixed pairs — Q4_K x Q4_K, Q4_K x Q5_K (bucketed), Q4_K x Q6_K, Q5_0 / Q5_1 with Q8_0 — selected per tensor by the kernel maps.

BF16 as a first-class dtype

Kimi, Instella, and Qwen3.6 lanes run BF16 weights/KV with FP32 accumulation (AVX-512 BF16 / AMX where available, portable rounding elsewhere). Not a GGUF quant — a storage contract.

Format support boundary

Supported: Q4_0, Q4_1, Q5_0, Q5_1, Q8_0, Q4_K, Q5_K, Q6_K, Q8_K, plus NVFP4 (MoE expert weights only, candidate status). One nuance: Q4_0 and Q4_1 have C kernels and dtype IDs but no v8 kernel maps — they remain reachable through legacy v7 bindings only, so no v8 circuit selects them. Q5_K's block struct lives privately in src/kernels/gemm_kernels_q5_k.c rather than in the public header. Not supported: Q2_K, Q3_K, and the IQ imatrix family — the runbook's model table lists tested artifacts per family.

Simple Formats (32 weights/block)

Q4_0

4.5 bits/weight
18 bytes = 2 (d) + 32x4/8

Q4_1

5.0 bits/weight
20 bytes = 2+2 (d,m) + 32x4/8

Q5_0

5.5 bits/weight
22 bytes = 2 (d) + 32x1/8 + 32x4/8

Q5_1

6.0 bits/weight
24 bytes = 4 (d,m) + 32x1/8 + 32x4/8

Q8_0

8.5 bits/weight
34 bytes = 2 (d) + 32x8/8
K-Quant Formats (256 weights/block)

Q4_K (Primary)

4.5 bits/weight
144 bytes = 4+12 (header) + 256x4/8

Q5_K

5.5 bits/weight
176 bytes = 4+12 (header) + 32 (qh) + 256x4/8

Q6_K

6.5625 bits/weight
210 bytes = 18 (header) + 256x6/8

Q8_K (Activations)

9.125 bits/weight
292 bytes = 36 (header) + 256x8/8
FP4 Formats (64 weights/block)

NVFP4 (MoE-only, candidate)

4.5 bits/weight
36 bytes = 4 (unsigned E4M3 sub-scales) + 64x4/8; the FP32 per-tensor/expert scale is a separate runtime operand, never stored in the block
E2M1 nibbles in split-half order (low nibble = sub-block lanes 0-7, high = lanes 8-15). Only two v8 kernel maps exist (routed + shared MoE SwiGLU experts), both candidate status — not a general GEMM format. Block anatomy and dequant pipeline: Quantization Deep Dive — NVFP4.

Complete Format Reference

C-Kernel-Engine Quantization Formats - Complete byte-level visualization

Click image to open in fullscreen viewer with zoom and pan controls

How CKE Executes These Formats

Pipeline from GGUF quantized weights through kernel-map provider selection to per-tile execution with FP32 accumulation

From GGUF bytes to provider-selected kernels: the same path for every model family. Click to open the fullscreen viewer.

The byte layouts above are the storage format; the diagram shows the execution contract. For the mechanism behind each step:

Key Concepts

Block Quantization

Weights are grouped into blocks (32 or 256). Each block has a shared scale factor, reducing overhead while maintaining accuracy. Larger blocks = better compression, smaller blocks = better accuracy.

Symmetric vs Asymmetric

Symmetric (Q4_0, Q5_0, Q8_0): Values centered at 0. Formula: weight = (q - center) x d
Asymmetric (Q4_1, Q5_1): Adds min offset. Formula: weight = q x d + m

Q4_K dmin: SUBTRACT not ADD!

Common Bug: Q4_K uses weight = q x (d x sc) - (dmin x mn)
The minus sign is critical! dmin encodes a positive offset that gets subtracted to shift the range down. Using + instead of - produces wrong outputs.

K-Quant Nested Scales

K-quant formats use 2-level scaling: a super-block FP16 scale multiplied by per-sub-block 6-bit or int8 scales. This gives fine-grained control with minimal overhead.

5-bit and 6-bit Packing

Q5 and Q6 formats split bits across multiple byte arrays. Q5: 4 low bits in qs, 1 high bit in qh. Q6: 4 low bits in ql, 2 high bits in qh.

Q8_K: FP32 Scale + bsums

Q8_K uses FP32 scale (not FP16!) for higher precision. The bsums field contains precomputed sums of 16 consecutive int8s for VNNI/AVX-512 dot product optimization.

Recent Fix: Q8_K SSE Parity

On 2026-03-09 we fixed a subtle Q8_K parity bug in quantize_row_q8_k_sse.c. The SSE path now preserves the same signed-max selection and bsums contract as llama.cpp/ref. See commit 224a4d30.

This class of bug is easy to miss: text generation can still look mostly normal, but mixed-quant boundaries like quantize_row_q8_k -> gemv_q4_k_q8_k accumulate small parity drift until a model family like Nanbeige exposes it.

Quant Summary (current v8 mixed-quant path)

These are the formats that matter in the v8 generated runtimes across Gemma, Qwen (dense, MoE, and hybrid DeltaNet), GLM-4, Nemotron, Kimi, Cohere2, Laguna, Instella-MoE, Nanbeige, and the vision and audio lanes. The practical rule is not just "pick a small dtype", but "pick the weight format and activation contract that match the runtime kernel actually selected by the kernel map." The older v7 quantized inference path used the same byte formats; v7 is now the training lane.

Q8_K 256-block
UseActivation-side K-quant path
StrengthMatches ggml mixed-quant kernels
v8 path for quantize_row_q8_k -> gemv_q4_k_q8_k and gemv_q6_k_q8_k; the same byte contract ran in v7.
Q8_0 32‑block
UseLegacy/simple high-quality paths
StrengthSimple block format
Still useful, but it is no longer the whole story for modern K-quant inference.
Q5_0 / Q5_1 32‑block
UseQ/K/O projections
StrengthGood speed/quality
Q5_1 adds an offset for asymmetric ranges; Q5_0 is symmetric.
Q5_K 256‑block
UseGemma MLP / proj
StrengthHigher compression
K‑quant with nested scales; improves size without wrecking MLP quality.
Q6_K 256‑block
UseSensitive MLP / output weights
StrengthHigher fidelity
Often selected for sensitive MLP paths where Q5_K is too lossy.
Q4_K 256‑block
UseCompression-first K-quant weights
StrengthSmallest footprint
Great for footprint, but easiest to destabilize if overused.
Hidden State
FP32
Activation Quantize
quantize_row_q8_k
Mixed K-Quant Matvec
Q4_K / Q6_K × Q8_K
Q8_K Activation-side block format for the current mixed K-quant runtime path.
Q4_K Compression-first weight blocks that rely on the correct q8_k activation contract.
Q6_K Higher-fidelity weight blocks for sensitive MLP/output paths.

Related Documentation

Image
100% | |
Scroll to zoom | Drag to pan | W/H to fit | 0 to reset | ESC to close