Parakeet TDT Native CPU Bring-up

CKE now runs the pinned Parakeet TDT 0.6B v3 checkpoint end to end through native CPU kernels and BUMP weights. The exact short trajectory passes, deterministic five-minute comparisons pass, and the bounded overlapping-window policy covers a complete 42-minute recording. Diarization remains a separate milestone.

Current claim boundary
One 7.435-second English trajectory matches the pinned Transformers reference exactly for all 50 token and duration decisions. Two complete five-minute full-attention runs and two deterministic overlapping-window runs pass. A separate 2,542-second recording completes all 17 bounded windows with exact source coverage, finite outputs, monotonic word timestamps, and unchanged output after resume. The runtime still uses Python orchestration around native kernels; generated-circuit execution, multilingual quality, concurrent-session isolation, and diarization remain open.

Frozen reference

Checkpoint
nvidia/parakeet-tdt-0.6b-v3
541d1f99...defd82b
CC BY 4.0

Reference code
Transformers commit 66799f45...
FP32 eager CPU

Model shape
24-layer FastConformer, width 1024
2-layer LSTM, width 640
723 tensors, 2,508,311,120 bytes

Decoder
8,193 token logits including blank 8192
durations [0, 1, 2, 3, 4]
0.08-second encoder frames

The checkpoint metadata says 5.6.0.dev0. The released v5.6.0 source does not contain ParakeetForTDT, so certification pins the executable source commit directly.

First reference fixture

The 7.435-second mono 16 kHz utterance 2086-149220-0033.wav is the sample linked by NVIDIA's model card and comes from the CC BY 4.0 LibriSpeech corpus. On Ryzen with PyTorch 2.13.0+cpu, 16 threads, and the pinned Transformers commit, the reference completed in 0.39 seconds and produced 50 decode steps, 44 nonblank emissions, durations, and token timestamps.

Well, I don't wish to see it any more, observed Phoebe, turning away her eyes. It is certainly very like the old portrait.

The compressed tensor fixture retains audio, normalized frontend features and mask, subsampling, encoder layers 0 and 23, projected encoder output, initial decoder output, all 8,198 first-step joint logits, generated token IDs, and durations. Every recorded array is finite and has an independent content hash.

Native execution evidence

The short native path converts all 699 inference tensors to BUMPWGT5 and excludes 24 BatchNorm training counters explicitly. On Ryzen, the frontend, all 24 FastConformer blocks, prediction network, joint network, greedy TDT decoder, tokenizer, and token timestamps complete without importing PyTorch or Transformers. The retained short result matches 50 of 50 token IDs, 50 of 50 durations, the decoded transcript, and all 44 timestamp entries.

The five-minute fixture consumes all 4,800,000 source samples and produces a complete transcript and monotonic token timestamps. Two runs produced identical token IDs, durations, transcript, and 1,296 timestamp entries. The retained repeat took 369.19 seconds (RTF 1.23) and 2.87 GB peak RSS on Ryzen. Its WER is 10.43% against the curated 690-word transcript.

Whisper Base on the identical PCM completed in 10.19 seconds (RTF 0.034) with 5.65% WER using its per-window fallback. Whisper's default persistent-worker path failed before inference with a dynamic-module pickling error, which is retained as a separate runtime limitation. Full Parakeet attention uses one caller-owned score row per head and schedules heads independently; the provider does not allocate heap memory. The complete selected call chain still needs repeated-call allocation instrumentation before an allocation-free runtime claim.

A synthetic 48 kHz stereo transform of the short fixture also reproduces the 16 kHz mono trajectory after CKE windowed-sinc resampling. This validates the wiring and deterministic fixture transform; broader real-world resampling quality is not yet certified.

The bounded long-audio policy uses 180-second full-attention windows with 30 seconds of overlap. Encoder and decoder state reset for each window. Each word belongs to the window whose ownership region contains its timestamp midpoint; ownership regions meet at the midpoint of each overlap. Two five-minute runs produced identical 700-word trajectories, exact 0–300 second coverage, and 7.25% WER. The retained repeat took 231.19 seconds (RTF 0.771) with 3.24 GB peak RSS.

The encoder's scaled and unscaled residual additions and both TDT argmax decisions now use declared native providers. On the clean Ryzen build at commit d3a6a8e55, this path preserved all 50 reference token IDs and all 50 duration decisions exactly. The machine-readable host-math inventory distinguishes the remaining model, layout, session, and planner ownership work.

Run Parakeet through the production entry point

The audio front door recognizes the checkpoint architecture, creates the FP32 BUMP bundle, builds the current native libraries, and selects deterministic bounded long-audio execution. The output report can resume only at completed window boundaries.

version/v8/scripts/cks-v8-run audio \
  hf://nvidia/parakeet-tdt-0.6b-v3 \
  --wav recording.wav \
  --output build/parakeet-transcript.json \
  --resume

Original operation-reuse inventory

This table records the pre-implementation inventory from PR #503. Its missing-provider and composition labels describe that starting point; the native evidence above is the current status. Candidate reuse meant that an interface fit still required comparison at the named fixture boundary.

Stage	Required operation	Disposition	CKE foundation and exact gap
frontend	PCM decode and mono conversion	Reusable	`audio_wav_decode_memory_pcm16_mono_window_f32; audio_pcm_s16_to_mono_f32` 16-bit WAV is directly covered; FLAC remains host/frontend decoding before native session input.
frontend	sample-rate validation and resampling	Contract extension	`audio_resample_windowed_sinc_f32` The pinned processor rejects non-16 kHz input. CKE may expose explicit resampling, but must record this policy and certify its filter separately.
frontend	pre-emphasis x[t]-0.97*x[t-1]	Candidate reuse	`audio_preemphasis_f32` Arithmetic and first-sample rule match; real-fixture comparison remains required.
frontend	centered STFT power, n_fft=512, win=400, hop=160	Candidate reuse	`audio_stft_power_centered_window_f32` Use an explicit non-periodic Hann window and zero padding. Existing provider supports the shape; compare direct-DFT accumulation with torch.stft.
frontend	Slaney mel filter bank, 128 bins	Candidate reuse	`audio_whisper_mel_filters_slaney_f32` Provider is dimension-general despite its legacy name. Compare every filter coefficient with the pinned librosa FP32 bank.
frontend	natural-log mel projection	Candidate reuse	`audio_log_mel_time_major_f32` Time-major layout and ln(sum+epsilon) match structurally; freeze the pinned epsilon and accumulation tolerance.
frontend	per-feature normalization over live frames	Candidate reuse	`audio_feature_normalize_per_feature_f32` Sample variance uses N-1 and epsilon after sqrt. Batched padding masks need composition around the provider.
encoder	three-stage grouped Conv2D subsampling	Candidate reuse	`audio_conv2d_whc_grouped_f32` Covers regular, depthwise, and 1x1 stages with bias. Circuit must reproduce masking after every Conv2D and CHW/time reshapes.
encoder	subsampling ReLU	Map promotion	`relu_forward` Kernel exists but lacks a current v8 inference kernel map; promote a narrow FP32 map or fuse only after parity.
encoder	subsampling flatten and linear projection	Reusable	`gemm_nt_fp32_exact` Optional bias is supported. Layout conversion from [C,H,W] to time-major C*W must be declared.
encoder	relative sinusoidal position encoding	Missing provider	`none` Needs FP32 descending relative positions [T-1..-(T-1)], sin/cos interleave, and exact cast boundaries.
encoder	LayerNorm at five block sites	Candidate reuse	`layernorm_fp32_exact` Shape fits, but existing exact contract targets ggml reduction order. PyTorch FP32 CPU parity must be measured before selection.
encoder	Macaron FFN linear-SiLU-linear	Composition gap	`gemm_nt_fp32_exact; recurrent_silu_forward` Linear provider is reusable. Standalone SiLU needs an architecture-neutral map and PyTorch FP32 numerical certification.
encoder	half-scaled FFN residual	Composition gap	`add_stream_inplace` DSL needs explicit 0.5 branch scaling at both FFN residual sites with preserved operation order.
encoder	Transformer-XL relative self-attention	Composition gap	`gemm_nt_fp32_exact; audio_relative_shift_f32` Q/K/V/O and relative-K projections reuse GEMM. Bias-u/v, content/position score composition, non-causal mask, FP32 softmax, value product, and layout contracts need a declared composite/provider.
encoder	Conformer pointwise/depthwise Conv1D	Contract extension	`audio_conv1d_channel_major_f32` Kernel covers stride/padding. Parakeet weights omit convolution bias, while the map currently requires one; bind a prepared zero bias or add an optional-bias contract.
encoder	channel-split GLU	Candidate reuse	`audio_glu_split_channel_major_f32` Layout and value*sigmoid(gate) match; compare libm/SLEEF behavior against pinned PyTorch.
encoder	inference BatchNorm1D	Missing provider	`none` Needs running mean/variance, affine weight/bias, epsilon, FP32 order, no training-state updates, and no per-call allocation.
encoder	padding-mask zeroing before convolution	Composition gap	`none` Declare masked fill over channel-major activations; must use post-subsampling live lengths.
transducer	encoder 1024-to-640 projector	Reusable	`gemm_nt_fp32_exact` Bias is present and provider dimensions fit.
prediction	8193-by-640 token embedding	Candidate reuse	`embedding_forward_fp32` Vocabulary includes blank id 8192. Select the FP32 variant without positional embeddings.
prediction	two-layer 640-wide LSTM prediction network	Missing provider	`none` Needs prepared/session-owned hidden and cell state, PyTorch gate order and nonlinearities, explicit reset/reuse semantics, and no compute-time heap allocation.
prediction	decoder 640-to-640 projector	Reusable	`gemm_nt_fp32_exact` Bias is present.
joint	joint encoder-plus-decoder add and ReLU	Map promotion	`add_stream_inplace; relu_forward` Elementwise kernels exist; ReLU needs a v8 inference map and broadcast/add shapes must be declared.
joint	joint 640-to-8198 token-duration head	Reusable	`gemm_nt_fp32_exact` Output splits into 8193 token logits and five duration logits.
decode	greedy TDT token-duration state machine	Missing runtime	`none` Argmax token and duration independently; blank with predicted duration zero advances by one; nonblank duration zero stays on frame; max output is bounded by encoder traversal.
decode	Metaspace BPE decode	Contract extension	`existing tokenizer infrastructure` Tokenizer JSON has 8192 BPE entries with Metaspace replacement U+2581. Verify repeated tokens, blank/pad suppression, punctuation, and UTF-8 languages.
decode	token timestamps	Missing runtime	`none` Cumulative duration gives frame index; frame rate is 0.08 s. Punctuation is collapsed to the previous token endpoint by the pinned processor.
conversion	safetensors to BUMP plus sidecar conversion	Missing composition	`convert_safetensors_to_bump_v8.py foundation` Add strict Parakeet name/shape rules and require all 723 tensors to be consumed or explicitly classified; do not load Transformers during inference.
runtime	Parakeet circuit and native session	Missing composition	`v8 circuit/lowering/runtime foundations` Add declared frontend, encoder, prediction, joint, and decode components. No model-specific branches in generic emitters and no per-call heap allocation in selected compute paths.

Implementation milestones

#	Milestone	Status and evidence
1	frontend and subsampling X-Ray	Passed on the retained short fixture under declared tolerances.
2	one FastConformer block	Passed at layer 0; BatchNorm and relative-attention contracts are declared.
3	full encoder	Passed across all 24 retained boundaries and the 1024-to-640 projection.
4	prediction, joint, and TDT decode	Passed: token IDs, durations, timestamps, and transcript match the pinned short reference.
5	BUMP conversion and native diagnostic runner	Passed: 723 tensors are accounted for and inference has no reference-framework dependency. Generated-circuit execution remains open.
6	long audio and multilingual coverage	In progress: deterministic five-minute full-attention and overlapping-window evidence passes. A matched Whisper Base control passes. The complete 42-minute input passes the bounded 17-window policy at RTF 0.847 with 3.29 GiB peak RSS. Multilingual quality remains open.

New production compute paths must use planner-declared scratch or session-owned state. They may not allocate or free heap memory per call, hide allocation in helpers, substitute large stack buffers, or share mutable scratch across sessions.

Reproduce the reference capture

Download the exact Hugging Face revision once into the shared model cache, then run the fixture tool. Transformers, PyTorch, librosa, and soundfile belong to this certification environment; the eventual CKE inference path will not depend on them.

export CKE_MODEL_CACHE="$HOME/.cache/cke/models"
hf download nvidia/parakeet-tdt-0.6b-v3 \
  --revision 541d1f99c6b0c3cd0b11a95167540bb8edefd82b \
  --local-dir "$CKE_MODEL_CACHE/parakeet-tdt-0.6b-v3-541d1f99"

PYTHONPATH="$HOME/.cache/cke/reference/transformers-66799f45" \
python version/v8/scripts/capture_parakeet_reference_v8.py \
  --model-dir "$CKE_MODEL_CACHE/parakeet-tdt-0.6b-v3-541d1f99" \
  --audio version/v8/test_assets/parakeet_tdt_0_6b_v3/2086-149220-0033.wav \
  --output-json /tmp/parakeet-reference.json \
  --output-npz /tmp/parakeet-reference.npz \
  --model-revision 541d1f99c6b0c3cd0b11a95167540bb8edefd82b \
  --transformers-revision 66799f45cea7513712580c6170cdaa4438df702a

Next work

The complete 42-minute retained recording now passes the overlapping-window policy with coverage ending at exactly 2,542.0 seconds. Its 7,225-word output has no aligned human transcript, so this result establishes execution, coverage, timestamp, and resource behavior rather than WER. The full-attention diagnostic runner fails closed above the checkpoint's 5,000-position limit. Model arithmetic still performed by NumPy orchestration must continue moving into declared CKE operations and a planned generated circuit. Multilingual coverage and diarization follow as separate evidence lanes.