C-Kernel-Engine
High-Performance ML Kernels

Whisper Tiny End-to-End

Kernel Source
src/kernels/audio_kernels.c and include/ckernel_audio.h — WAV decode, resampling, pad/truncate, STFT/FFT-400, Slaney Mel, log-Mel, Conv1D stem, and layout primitives. The frontend is the circuit version/v8/circuits/whisper_audio_frontend.json; encoder and decoder topology lives in version/v8/circuits/audio_transformer_encoder.json and version/v8/circuits/audio_transformer_decoder.json; lifecycle orchestration in version/v8/scripts/ck_run_v8.py (unified cks-v8-run audio command). Per-kernel math and diagrams: Concepts — Audio Frontend & Cross-Attention; per-kernel deep dive: Audio Kernels Deep Dive.

CKE v8 runs a generated FP32 Whisper Tiny encoder and decoder from a PCM16 WAV file — and the frontend is generated too. WAV decode, windowed-sinc resampling, pad/truncate, STFT table construction, FFT400 power, Slaney Mel filters, and log-Mel normalization are seven explicit circuit operations in version/v8/circuits/whisper_audio_frontend.json, each with an exact numerical contract and a map-owned call ABI. The resolved operation sequence lowers through call IR into generated C exporting ck_model_run_audio_wav. Python coordinates artifact lifecycles and process isolation; it performs no frontend arithmetic. Codegen contains no Whisper model-name branch; every arithmetic boundary resolves through a kernel-map contract.

How the Audio Pipeline Works

The diagram follows one WAV file through the full system: contract kernels prepare the log-Mel grid, the generated encoder writes an immutable encoder memory once, and the generated decoder reads it on every cross-attention step while its own KV-cache grows token by token.

AUDIO PIPELINE · GENERATED FP32 WHISPER TINY One WAV in, tokens out — every stage a kernel-map contract Python orchestrates; C kernels and generated model libraries compute. No Whisper model-name branch anywhere in codegen. 01 · WAV IN PCM16 · mono · 16 kHz Any format converted with ffmpeg: ffmpeg -ac 1 -ar 16000 -c:a pcm_s16le CIRCUIT OPS → MAP-OWNED KERNELS audio_wav_decode_memory_pcm16_mono_f32 audio_resample_windowed_sinc_f32 audio_pad_or_truncate_f32 ≤ 30 s today — truncated, not chunked 02 · LOG-MEL FRONTEND STFT power → 80-band log-Mel n_fft 400 · hop 160 · Slaney mel CIRCUIT OPS → MAP-OWNED KERNELS audio_stft_precompute_tables_f32 audio_stft_power_fft400_f32 20×20 mixed-radix FFT + direct-DFT reference audio_whisper_mel_filters_slaney_f32 audio_whisper_log_mel_from_power_reference_f32 vs Hugging Face: 1.54e-4 max|Δ| · 1.37e-6 RMSE 03 · CONV STEM + LAYOUT Conv1D ×2 + GELU → tokens Mel frames become the encoder token grid CIRCUIT OPS → MAP-OWNED KERNELS audio_conv1d_channel_major_f32 gelu_pytorch_erf_f32_inplace audio_transpose_channel_to_token_f32 unequal-length primitives, no model branch 04 · ENCODER — GENERATED C LIBRARY Transformer blocks · bidirectional self-attention whisper-tiny safetensors → BUMP → call IR → generated C 67 source tensors → 71 BUMP entries · 100% weight coverage PARITY EVIDENCE 79-edge PyTorch X-ray, checkpoint by checkpoint max_abs ≈ 5.05e-4 · RMSE ≈ 6.7e-6 (FP32) runs in an isolated child process — encoder and decoder share one ABI, so one process could bind a call to the wrong artifact ENCODER MEMORY immutable n_audio rows written once, read every decode step 05 · DECODER — PERSISTENT KV-CACHE forced prefix → greedy tokens → EOS self-attn: decode provider, pos + 1 valid KV rows cross-attn: K/V from encoder memory (immutable length), Q from active decoder-token count REFERENCE EVIDENCE JFK sample: same 23 greedy tokens as Hugging Face Whisper Tiny, followed by EOS English · no-timestamp decoding certified THE THREE CONTRACT FINDINGS THAT MADE E2E WORK 1 · CROSS-ATTN K/V EXTENT K/V projections keep the immutable encoder-memory length — not the active decoder-token count. 2 · CROSS-ATTN Q EXTENT Query length uses the active decoder-token count — not the decoder’s physical capacity. 3 · INCREMENTAL SELF-ATTN Decode provider attends model->pos + 1 valid KV rows — a one-token prefill read only cache row zero. NIGHTLY log-Mel frontend vs PyTorch · audio transformer primitives · v8 audio circuit/codegen regression OPT-IN full model-and-WAV transcription — artifact gate: full reference token sequence + EOS, no tolerance relaxation

Generated Frontend: Circuit Ops to Generated C

The frontend began as direct C calls selected by the Python runner — useful correctness bring-up, but provider selection sat outside the v8 architecture. It is now seven explicit circuit operations (audio_wav_decode, audio_resample, audio_pad_or_truncate, audio_stft_tables, audio_stft, audio_mel_filters, audio_log_mel), each carrying an exact numerical contract and a map-owned call ABI. Lowering plans the frontend buffers and lowers the runtime WAV metadata into call IR; the resolved sequence generates C exporting ck_model_run_audio_wav.

Whisper audio frontend lowering from circuit operations through kernel maps and call IR to generated C

What Nightly Proves, and What Stays Opt-In

Coverage is split deliberately, because GitHub runners do not carry the generated Whisper artifacts:

Transcribe Arbitrary Audio

Convert any source format to mono 16 kHz PCM16 WAV first: 

ffmpeg -i your-audio.mp3 \
  -ac 1 -ar 16000 -c:a pcm_s16le \
  /tmp/test-audio.wav

Then run the generated pipeline through the unified v8 command surface:

version/v8/scripts/cks-v8-run audio \
  --encoder-run-dir /path/to/whisper-tiny-encoder \
  --decoder-run-dir /path/to/whisper-tiny-decoder \
  --wav /tmp/test-audio.wav \
  --language en \
  --task transcribe \
  --max-tokens 128 \
  --output build/whisper-test-report.json

Encoder and decoder execution occurs in isolated child processes. Both generated artifacts export the same model ABI and shared-library symbol names, so loading both into one process can bind a call to the wrong artifact.

Current Limitations

Correctness Findings

  1. Cross-attention K/V projections must use the immutable encoder-memory length. Generic prefill code previously replaced their matrix row count with the active decoder-token count.
  2. Cross-attention query length must use the active decoder-token count. The generated prefill call previously retained the decoder's physical capacity.
  3. Incremental self-attention must use the decode provider and attend model->pos + 1 valid KV rows. A causal prefill provider invoked with one active token attended only cache row zero.
  4. Encoder-side cross-attention K/V projections are immutable for an audio segment. The circuit declares a prefill-populated, layer-major FP32 cache; decode consumes the cached head-major slices instead of projecting and transposing all 1,500 encoder rows for every generated token. Binding new encoder memory invalidates the cache, and decode fails closed until prefill repopulates it.

These dimensions are now represented in call IR and phase-specific circuit contracts. Codegen consumes the resolved semantics and contains no Whisper model-name branch.

Reference Evidence

On the public JFK Whisper sample, CKE emitted the same 23 greedy text tokens as Hugging Face Whisper Tiny, followed by EOS: 

And so my fellow Americans ask not what your country can do for you
ask what you can do for your country.

The optimized C FFT/log-Mel frontend differed from Hugging Face by approximately 1.54e-4 maximum absolute error and 1.37e-6 RMSE, without changing any generated token. Before persistent cross-attention caching, one measured run used 0.16 seconds for the frontend, 11.17 seconds for the encoder, 0.75 seconds for decoder prefill, and 16.57 seconds for 23 decode tokens. With the cache, the same exact token sequence used 0.12 seconds for the frontend, 11.15 seconds for the encoder, 0.73 seconds for prefill, and 0.21 seconds for decode. Decoder time improved by approximately 79x, and total time fell from 28.64 seconds to approximately 12.22 seconds.

Regression Gates

make test-audio

CK_WHISPER_ENCODER_RUN_DIR=/path/to/encoder \
CK_WHISPER_DECODER_RUN_DIR=/path/to/decoder \
CK_WHISPER_WAV=/path/to/jfk.wav \
make test-whisper-e2e-auto

Nightly publishes separate PyTorch-backed audio frontend and transformer primitive rows, plus a portable v8 audio circuit/codegen row. Together they fix the Slaney filter identity, forced decoder prefix, projection extents, phase-specific attention providers, and generated call arguments. The artifact gate requires the full reference token sequence and EOS without tolerance relaxation.

Image
100% | |
Scroll to zoom | Drag to pan | W/H to fit | 0 to reset | ESC to close