C-Kernel-Engine / TTS bring-up

Kokoro TTS bring-up

This page records the first bounded phoneme-to-waveform groundwork. Generated Kokoro waveform, native text preparation, listening, playback, and P3/Ryzen performance remain NOT_TESTED.

Implementation plan

Status: architecture and interface proposal. No CKE TTS model, speech session, quality result, or CPU speed is certified by this document.

Scope and ordering

The first implementation milestone is phoneme IDs plus one fixed voice to a generated FP32 waveform. This isolates model arithmetic from text normalization and G2P; it is phoneme-to-waveform, not complete native text-to-speech. The first product milestone is one repeatable path from a local assistant response to audible PCM. Keep Kokoro as the first generated-C model and fallback candidate; evaluate Chatterbox Turbo for interaction and Qwen3-TTS 1.7B for capability only after their exact upstream revisions, preprocessing, and numerical contracts are recorded. The accessibility application may use an existing local speech runtime until native Kokoro passes the complete gate. Kokoro implementation can proceed in parallel with bounded generated-audio optimization and Gemma vision work; reserve P3 or Ryzen for uncontended numerical and performance certification.

The accessibility design is in docs/notes/ACCESSIBLE_AGENT_MEMORY_PALACE.md in the active accessibility checkout. Its application owns speech queuing, playback, policy, emergency stop, and sentence navigation. Generated model code owns only mathematical execution. The host owns text preparation, model state, scheduling, PCM conversion, and cancellation.

Current CKE boundary

include/ck_session_v8.h describes a text token session. The current version/v8/src/ck_cli_v8.c implementation requires autoregressive decode, text encode, and token decode capabilities at open. It uses process-global generation variables and generated-model entry points without a session handle. Its cancellation flag is checked between text generation steps. Reusing that ABI unchanged would fail the requirements for independent speech requests, PCM metadata, explicit model scratch, and bounded output. A speech ABI should be separate and versioned; it can reuse descriptor/error conventions.

The v8 audio circuit and kernel maps primarily implement inbound audio and ASR: PCM decode, resampling, STFT, Mel, Conv1D, Conformer, and Whisper encoder/decoder work. This is useful for input conversion and selected forward primitives, but it does not constitute a TTS decoder or a waveform oracle. Any shared kernel must retain the target model's exact layout, padding, rounding, and operation order. The current tree also has an audio_lstm_step_f32 map; Kokoro's bidirectional and stacked LSTM paths still need shape and numerical checks before reuse. CKE's C implementations belong in src/kernels; v8 circuits and kernel maps describe selection and lowering. TTS reference capture lives separately from deployed arithmetic.

DSL fit and compiler boundary

CKE already supports declarative sequence, block_types with header/body/footer, component circuits with explicit stitch edges, graph_slots, weight_refs, required numerical contracts, and circuit-owned activation buffers. Kokoro's fixed arithmetic graph should be decomposed into phoneme embedding/ALBERT, duration and alignment, prosody/text encoding, decoder/generator, and inverse-STFT components. Repeated ALBERT and residual blocks belong in circuit bodies; fixed voice selection and duration policy belong in declared operations and the native host, not a kokoro branch in the lowerer. New mathematical providers belong in src/kernels with kernel-map contracts and oracle tests.

Current build_ir_v8.py still uses static OP_DATAFLOW, TEMPLATE_TO_KERNEL_OP, and TEMPLATE_OP_WEIGHTS tables. Its circuit activation-buffer expressions resolve from configuration at build time; they cannot by themselves bind a live predicted duration sum. A fixed oracle utterance can specialize those extents for numerical bring-up. General speech requests need a generic checked runtime-extent/buffer-binding capability, or bounded maximum buffers with live lengths passed through ordinary operation parameters. The compiler must reject an unknown TTS operation or unresolved extent; adding family-specific branches would hide missing DSL capability. Ownership of shared codegen and registry changes must be agreed with agents working in those files before implementation.

Target architecture inventory

These are candidate upstream pins for an import audit, not compatible CKE artifacts. Recheck licenses for every packaged dependency and voice asset independently of the model repository's top-level license. The Kokoro oracle environment still needs resolved wheel, spaCy model, and optional eSpeak data versions and hashes; the selected voice ID and asset hash are also open.

Target Candidate model snapshot and license Preparation, generation, output State and missing CKE contracts
Kokoro v1.0 hexgrad/Kokoro-82M@e8a90b41091c3c5b70375c47cc959799920fa4d6, Apache 2.0 Normalization and Misaki G2P at fba1236 produce phoneme IDs. Kokoro code at dfb907a applies ALBERT, duration LSTM and length regulation, prosody and text encoding, then AdaIN/ISTFTNet waveform decoding. One exported voice table can supply fixed style conditioning. Deterministic whole-utterance synthesis; cap symbols and predicted frames. Need exact LSTM, duration expansion, adaptive/instance norm, upsampling, inverse STFT, waveform and native G2P contracts. Lock resolved G2P packages and data before oracle capture.
Chatterbox Turbo ResembleAI/chatterbox-turbo@749d1c1a46eb10492095d68fbcf55691ccf137cd, MIT model/repo; review reused component notices Code at 5de7a54 normalizes text, uses the tokenizer files in the model snapshot and stored conditioning, generates T3 speech tokens autoregressively, then S3Gen converts tokens to waveform via conditional flow and HiFTGAN/F0 stages. Sampling seed, EOS/max-token caps, flow state and vocoder scratch must be explicit. Need model-specific token, flow, F0 and vocoder operations. Reference returns a whole waveform; audio streaming is unverified.
Qwen3-TTS 1.7B CustomVoice, 12 Hz Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice@0a272c2df2ee9be6b850d1df75bcf673541e523a, Apache 2.0 Code at 022e286 formats text using the model snapshot's tokenizer files and built-in speaker/language condition, generates codec tokens with talker and code predictor, then calls the separate neural speech tokenizer decoder for waveform. Sampling, EOS, token and frame caps, code-predictor cache and codec state must be explicit. Need exact codec decoder operations and numerical contracts. Voice cloning applies to a different variant. Upstream input simulation does not establish waveform streaming.

Existing CKE GEMM, embedding, attention, normalization, and selected audio Conv1D maps are candidates for reuse after shape and numerical checks. The inbound FFT-400 power map is not an inverse STFT. Model size does not determine porting effort or CPU speed; each target must be measured as a full text-to-PCM path.

The first isolated C additions are src/kernels/kokoro_shape_bounds.c and src/kernels/audio_duration_expand.c, declared in include/ckernel_tts.h. The planner validates duration sums, alignment extent, decoder and generator upsampling, inverse-STFT extent, and a conservative output-frame bound before writing. The duration-expansion primitive accepts caller-owned output storage. tests/test_v8_kokoro_shape_bounds.py compiles and exercises both. Neither file is a generated Kokoro graph or waveform evidence.

Proposed native speech session contract

Open a read-only model object once with pinned model artifact and manifest hashes. Each session has its own generation state and a predeclared workspace. A model descriptor reports supported voice IDs, language IDs, PCM sample rate, channels, format, maximum UTF-8 bytes, maximum normalized symbols, maximum audio frames, maximum synthesis step duration, and whether genuine incremental waveform output is supported. No Python process participates after export.

The request includes UTF-8 text and byte length, voice ID, supported model-specific settings, a reproducible seed if sampling exists, and explicit maximum input/output limits. Unsupported settings fail before generation. Empty text has a defined completed-with-zero-frames result. Oversized input fails before allocating a request workspace. Speech rate belongs in the model request only if the pinned model defines it; playback rate belongs to the application.

The output format for the first Kokoro path is interleaved signed PCM16, one channel, with the sample rate read from the pinned artifact manifest. The model math may produce FP32 waveform internally. Conversion to PCM16 is an explicit native host operation with clipping and nonfinite-sample rejection. Each audio callback receives a borrowed buffer, frame count, sample rate, channel count, format, monotonic utterance/frame offsets, and a request ID. The buffer remains valid only until the callback returns. The application copies or consumes it before returning; ownership never crosses the ABI silently.

Events are STARTED, AUDIO, and exactly one terminal event: COMPLETED, CANCELLED, or ERROR. STARTED occurs only after validation and workspace reservation. COMPLETED reports the exact frame count and whether output was sentence-batched or genuinely incremental. An error includes a stable code and message. A callback that cannot accept audio returns a backpressure result; generation blocks only while its bounded queue has capacity, or the request returns a typed backpressure error. No unbounded buffering or silent dropping is allowed. The first implementation may choose synchronous callbacks and no host output queue; application queue limits still apply.

Cancellation is callable from another thread, sets a session-private atomic flag, and wakes any backpressure wait. Generated execution checks it at bounded model operation boundaries, including long acoustic and vocoder stages; a single large noninterruptible forward call must be split before responsiveness is claimed. The terminal event follows cancellation, and no further audio events follow it. Playback stop and queued-text removal are separate application actions and must occur immediately even while generation unwinds. Do not claim a cancellation latency until it is measured on P3 and Ryzen.

Start with one session and one active request. Before enabling concurrent sessions, verify generated-model globals and the global thread pool permit concurrent calls; private scratch alone is insufficient. A later concurrent implementation may share immutable weights but must never share mutable caches, scratch, random state, or output buffers. Predeclare and cap every model workspace, tokenizer buffer, vocoder workspace, and output chunk. No allocator calls are allowed in generated hot paths. The host may allocate the full bounded workspace at session creation.

First Kokoro proof

  1. Pin a complete upstream revision set: model weights, config, voices, model code, phonemizer and lexicon, tokenizer if any, and dependency licenses. Save checksums and a fixture manifest. Include a phoneme fixture so text-front-end drift cannot be confused with model arithmetic drift.
  2. Export one voice and one short fixed utterance. Save oracle inputs and named intermediate tensors at every new numerical operation. Record waveform FP32 and PCM16 reference hashes, tolerances, and listening sample.
  3. Build the phoneme-to-waveform path through a circuit, operation contracts, kernel maps, generated C, a native phoneme host, and explicit scratch. Python is only an import and oracle tool. Native text preparation is a separate work package, and its uncertainty does not block this arithmetic gate.
  4. Run the same fixed utterance twice in fresh and persistent sessions. Require deterministic outputs under a fixed seed or document model nondeterminism. Inspect silence, clipping, nonfinite samples, truncation, and repetitions.
  5. After a native frontend is certified, connect assistant text -> bounded sentence queue -> native speech session -> PCM player with immediate queue clear and playback interruption on a new request. Buffer incomplete assistant fragments until a sentence boundary or explicit final flush; avoid speaking the same fragment twice.

Sentence-by-sentence synthesis is useful for application responsiveness but is not model streaming. Publish time to first playable audio and chunk-gap results before labeling any path streaming.

Work packages and acceptance gates

  1. Artifact and oracle lock: select one Kokoro voice and utterance, lock the model/code/G2P environment and licenses, publish phoneme IDs, style vector, named tensors, waveform, and manifest hashes. This gate is currently missing evidence until captures exist. The style row depends on the raw phoneme count including the reference's pad-wrap behavior; pin and test that selection, rather than treating the voice as one static vector.
  2. Primitive and circuit bring-up: implement only the missing operations reached by the pinned Kokoro graph, each with shape, stride, padding, precision, scratch, and oracle tests. Lower the complete graph to generated C and verify every exported weight and operation map. This gate is unsupported until a generated waveform path runs.
  3. Native text frontend: determine whether native preprocessing reproduces the pinned reference phoneme IDs. A native eSpeak-backed alternative needs its own license and pronunciation contract; differing phonemes cannot count as Misaki parity. This gate is missing evidence until tested.
  4. Speech session and host: add a versioned native speech ABI with independent state, bounded PCM callbacks, and responsive cancellation. Test failed open, empty input, buffer limits, backpressure, and repeated requests in one session. Verify global model and thread-pool safety before adding concurrent sessions. Measure the longest noninterruptible operation; a descriptor cannot promise a cancellation bound across CPUs. This gate is unsupported until the ABI has an implementation and tests.
  5. Accessibility workflow: connect bounded sentence buffering, native synthesis, playback, interruption, and the working fallback. Confirm the application can stop output immediately while model cancellation unwinds. This gate is unsupported until the full text-to-speaker path is exercised.
  6. Evidence and expansion: run quality and speed certification on P3 and Ryzen, then apply the same ABI and reporting to Chatterbox Turbo and Qwen3-TTS. Neither model is claimed compatible today.

Certification and reports

Keep numerical, intelligibility, and listening gates separate. Numerical gates compare pinned preprocessing, intermediate tensors, FP32 waveform, and PCM16 conversion. Speech fixtures cover ordinary and long text, numbers, dates, abbreviations, technical terms, CKE/kernel names, file paths, code explanations, empty and punctuation-only input, unsupported symbols, oversized requests, cancellation, repeated sessions, and interrupted playback. Listening review checks pronunciation, omissions, repetitions, and naturalness; ASR transcripts are diagnostic evidence only.

On both P3 and Ryzen, publish cold load, warm execution, first playable audio, whole-utterance latency, audio seconds per wall second, chunk gaps, cancellation latency, CPU use, and peak memory. Mark every result passed, failed, unsupported, or missing evidence. PR CI should run cheap primitive oracle, PCM conversion, and session state tests. Nightly should run bounded generated speech; idle-host certification should run long text, listening fixtures, and performance. Reuse existing report and visualizer conventions rather than inferring support from a circuit file alone.

Training lane

Inference does not wait for TTS backward coverage. V8 has a bounded FP32 generated-training certification and training -> checkpoint -> v8 inference export workflow that selectively reuses v7 training IR/codegen. This does not certify TTS-specific backward or composition-level training. Once the exact TTS graph is pinned, inventory every new backward operation. Later work can certify a small frozen-vocoder acoustic predictor, then generated training -> checkpoint -> inference export; voice adaptation requires voice consent and dataset provenance.

Kokoro graph inventory

Kokoro v1 phoneme-to-waveform graph inventory

Status: source-traced design inventory, not generated-waveform certification. Scope is batch one, pinned af_heart, speed 1, and the fixed phoneme fixture in version/v8/tts/reference/kokoro_v1_reference.json. The model/code pins and asset hashes belong to its capture manifest. The deployed graph starts with already prepared phoneme IDs; G2P is a separate contract.

Pinned execution and symbols

Sources: Kokoro model.py at dfb907a, modules.py, istftnet.py, and the pinned model config. Default disable_complex=False selects TorchSTFT, not CustomSTFT (istftnet.py:293-297).

Let T = phoneme_count + 2 include the two zero IDs; A = sum(round(sum(sigmoid(duration_logits), last_dim) / speed).clamp(min=1)); batch B=1. Input IDs are [1,T], int64 in PyTorch. The upstream model drops unknown phoneme characters in model.py:128, which the native phoneme fixture must never silently emulate: validate every ID. The voice row is af_heart[len(phonemes)-1], [1,256] FP32, selected before model execution (capture_kokoro_v1.py:128-137). Decoder style is [:,0:128]; predictor style is [:,128:256]. Pin the exact row and its hash, not just the voice name.

The config gives vocabulary 178, ALBERT hidden 768, 12 nominal transformer layers and 12 heads, projection/encoder width 512, 3 duration-encoder pairs, max-duration projection width 50, and ISTFTNet upsample factors [10,6] with FFT 20/hop 5. Verify ALBERT parameter sharing and grouping from the resolved Transformers version and checkpoint; twelve layer executions do not imply twelve independent parameter sets. All named activations and learned weights are FP32 in the intended first path; record actual imported dtypes and layouts.

Stage and source Reached math and logical shape Required contract / state
ALBERT, model.py:100-103, modules.py:180-183 Token/position/type embedding, layer norm, shared/grouped attention and FFN, attention softmax/mask: [1,T,768]; dense projection and transpose to [1,512,T]. Exact Transformers ALBERT parameter sharing, embedding projection, GELU variant, layer-norm epsilon, mask additive semantics, softmax and GEMM order. No autoregressive KV state. Existing CKE attention/GEMM/embedding are candidates only after this contract check.
Duration encoder, modules.py:137-176 Append predictor style to every token ([1,T,640]), mask, three bidirectional LSTM + adaptive layer-norm pairs. AdaLN computes LN(x, eps=1e-5)*(1+gamma)+beta, with [gamma,beta]=linear(style). Each pair reappends style. Output [1,T,640]. PyTorch IFGO LSTM, forward/backward scans, zero initial hidden/cell, style broadcast, masks. Eval disables dropout. The reusable audio_lstm_bidirectional_scan_f32 provider has independent primitive evidence; its Kokoro weight binding and circuit composition remain uncertified.
Duration predictor, model.py:106-114 Bidirectional LSTM [1,T,640]→[1,T,512]; linear [1,T,50], elementwise sigmoid, ascending sum over 50, divide by speed, PyTorch round-to-even, clamp minimum 1, cast to integer. Repeat token indices and one-hot alignment [1,T,A]; multiply duration-encoder features [1,640,T] @ [1,T,A] → [1,640,A]. Keep reduction/rounding explicit. Check each finite duration, T, A, and every downstream capacity before expansion. One-hot matrix need not be materialized if a verified duration expansion exactly matches the matmul's copy semantics.
Prosody, modules.py:124-134 Shared bidirectional LSTM on expanded duration features [1,A,640]→[1,A,512]. Independent F0/N branches, each three AdaIN residual blocks, middle block nearest-upsample ×2, then 1×1 conv: F0,N=[1,2A]. AdaIN is InstanceNorm1d(affine=True, eps=1e-5) followed by style (1+gamma)*norm+beta; preserve stored affine parameters. Residual block has nearest upsample, optional depthwise transposed conv and shortcut, two weighted convs, leaky ReLU, and 1/sqrt(2) (istftnet.py:340-381).
Text encoder, modules.py:35-69 Embedding [1,T,512], three weight-normalized Conv1D(k5,p2) + layer norm + leaky ReLU, then bidirectional LSTM, output [1,512,T]; expand by alignment to ASR features [1,512,A]. PyTorch convolution padding and weight normalization; dropout disabled. Pack/pad is identity on the single full-length fixture but must be declared if batching is added. audio_conv1d_channel_major_f32 is a candidate only after exact bias, stride, dilation, reduction and layout checks.
Decoder, istftnet.py:384-421 Stride-2 Conv1D of F0/N: [1,A]; concatenate with ASR, AdaIN residual encode to [1,1024,A]. Three decoder residual blocks consume concatenated [x, asr_res, F0, N]; fourth also concatenates, then upsamples ×2 to [1,512,2A]. Explicit multi-producer edges; first three stages use A, last emits 2A. Preserve concat order, reflection/nearest/depthwise transposed conv contracts, and style split.
Source and waveform generator, istftnet.py:108-325 Upsample F0 from 2A by 10*6*5=300 to 600A samples; make 9 harmonic sines plus stochastic noise, voiced/unvoiced mask, linear+tanh excitation, STFT magnitude/phase concatenation. Two stages: ConvTranspose1D strides 10 and 6; noise conv and AdaIN residual branch; sum three Snake residual blocks per stage. Post conv yields 22 channels; magnitude exp(first 11), phase sin(last 11); inverse STFT emits waveform approximately 600A samples (derive exact edge length from PyTorch). Stochastic source is reached even with fixed text/voice: torch.rand initial harmonic phases and torch.randn_like excitation (istftnet.py:149-209,241-254). Pin RNG implementation/state or capture source tensors as oracle inputs to isolate deterministic arithmetic; report which mode is compared. Snake is x+sin(alpha*x)^2/alpha, not SiLU. TorchSTFT uses periodic Hann, centered STFT, complex magnitude/angle, and torch.istft overlap-add/window normalization (istftnet.py:80-100). The added audio_istft_mag_phase.c has isolated PyTorch primitive evidence; it is still a candidate for generated Kokoro composition.

Convolution summation order, BLAS/oneDNN selection, transcendental implementation, normalization statistics, and STFT accumulation order must be captured in numerical contracts. A provider with the right operation name is not automatically bitwise equivalent to PyTorch; compare at intermediate checkpoints with stated tolerances before waveform comparison. Instance normalization reduces over time per batch/channel; layer normalization reduces over channels per token. Weight-normalized convs should be exported as effective FP32 weights with the transformation and hash recorded, or have a separately verified runtime operation.

Buffer and state boundaries

T is known at request validation. A becomes known only after the duration head. Validate finite logits, round/cast, checked sum, and a configured A_max before writing alignment/expanded features. Derived maximums include 2A prosody/decoder frames, 20A then 120A generator frames, 600A source samples, STFT frame capacity, inverse-STFT overlap workspace, and final waveform capacity. Exact STFT frame and waveform formulas must come from the pinned PyTorch center/padding behavior and be tested at short and boundary lengths; do not infer them from nominal audio duration. Return a typed capacity error rather than truncate or allocate. The scratch planner must account for forward/backward LSTM hidden+cell/gates, norm reductions, all branch lifetimes, convolution workspaces, RNG state/source, STFT complex buffers, and overlap/window envelope. Session initialization can reserve the maximum; compute kernels take explicit pointers, strides, capacities, and valid lengths. No variable-length stack arrays or hidden heap allocations.

The first data-dependent tensor is indices in model.py:110, a frame-to-token map with valid length A, produced by repeat-interleaving the T rounded durations. The next is dense alignment [T,A] (:111-113), then en=[640,A] and asr=[512,A] (:114-117). These feed the prosody shared LSTM and text-to-decoder path. If alignment is elided, the two expansion outputs are still the first data-dependent feature tensors. A generic tensor descriptor must distinguish allocated capacity=[T,A_max] or [C,A_max], per-request valid_extent=[T,A] or [C,A], and physical strides, e.g. [A_max,1] for a channel-major aligned buffer. Passing A as both loop extent and row stride would overlap channels when the backing buffer is padded to A_max. The producer publishes A after a checked extent barrier; downstream nodes consume the same validated symbol. Kernel maps must specify which ABI arguments receive each capacity, valid extent, and stride; planner computes physical bytes from capacity and liveness, while runtime loops use valid extents. Buffer views may alias only under declared nonoverlap/lifetime rules.

State is per request: LSTM states reset to zero for each scan, source RNG state is explicit, and all activations/scratch are private. Check global generated-model and threadpool behavior before concurrent sessions. For the first deterministic arithmetic oracle, captured source excitation (and optionally STFT output) may be an explicit input at a declared graph edge; a complete production waveform graph must generate it natively from a documented seeded RNG contract.

Circuit decomposition and provider fit

Use the v8 sequence plus component stitch and named graph_slots contracts (version/v8/circuits/README.md:200-245; PIPELINE.md:100-159). Proposed components, not separate Python-driven C entry points:

  1. phoneme_encoder: header validates IDs/style, ALBERT embedding; body expands declared ALBERT shared-layer schedule; footer projects [T,512]. A parallel declared text_encoder path consumes the same IDs and emits [512,T].
  2. duration: header concatenates style; body is three explicitly ordered bidirectional-scan/AdaLN pairs; footer runs predictor LSTM/head, validates A, and emits the duration vector and expansion map. The runtime shape-validation boundary is here.
  3. prosody: header expands duration features to valid A; body runs shared bidirectional scan and independent F0/N branches; footer emits F0,N=[2A]. Stitch also sends the expansion map to the text-encoder output path.
  4. waveform_decoder: header expands text features and combines ASR/F0/N/style; body declares four decoder residual stages, two generator stages, and each branch/merge; footer post-conv, magnitude/phase, inverse STFT, and FP32 waveform valid length.

Do not encode the entire schedule as one provider or a model-name branch. Nonrepeated operations and branch fan-in must be explicit circuit operations with declared ports. A single generated native entry point must execute the connected graph. PCM16 conversion and playback remain host/application steps.

CKE's audio_conv1d_channel_major_f32 map has channel-major contiguous FP32 input, bias, stride, padding and ascending accumulation; it lacks dilation/group semantics in that ABI. audio_lstm_step_f32 is a stateful IFGO single step with caller-owned gates scratch; scan and backward direction are not provided by the map. Existing STFT power maps are forward spectrogram power operations and do not provide Kokoro complex STFT, phase, or inverse. New src/kernels/audio_duration_expand.c and audio_istft_mag_phase.c have registered maps and isolated primitive oracles, but are not evidence of a complete generated graph. Likely missing provider contracts include adaptive instance/layer normalization, nearest interpolation, grouped/dilated/transpose convolution, Snake, exact phase/STFT/ISTFT, stochastic excitation, checked duration/extent, and bidirectional scan. Place reusable C math in src/kernels/; maps declare ABI, layouts, ordered arguments, scratch size/alignment/lifetime/aliasing, state reset, numeric order, sources, tests, and unsupported backward status.

The Kokoro inventory identified a compiler gap: circuit activation_buffers extents were resolved at build time, while A=sum(pred_dur) becomes known during execution. PR #560 adds a bounded, checked runtime valid extent to resolved call IR while keeping allocated capacity and physical row stride distinct. Static operation registration remains legitimate; the generic emitter has no Kokoro-specific branch. Complete Kokoro composition is still unimplemented.

Tested compiler fixture: a small v8 circuit passes through IR1, provider resolution, memory planning, lowering, and call IR. Its duration producer publishes a checked length; expansion and a consumer use that valid length and the planned physical stride. Generated C rejects invalid extent, undersized or misaligned caller-owned arena, and provider failure before downstream writes. Repeated requests exercise changing lengths, and X-Ray compares valid tensor regions rather than padding. The circuit declares its host native_entry; lowering carries that entry into call IR, and the normal codegen_v8.py command emits the checked native function. The separately supplied inverse-STFT input proves one generated provider call; it is not a complete Kokoro acoustic graph or generated speech.

Normal-codegen boundary

emit_checked_calls() consumes resolved IR Lower 3 calls and canonical kernel maps as a reusable backend component. codegen_v8.main() selects it for a circuit-declared checked native entry with a runtime_extent_contract; other model artifacts continue through the existing core backend. The command-level fixture compiles and executes the emitted function. This establishes ordinary command integration for the bounded synthetic graph, not complete Kokoro compilation, weight loading, or a production speech session.

The checked entry requires a caller-owned arena aligned to at least 64 bytes, or the strongest selected provider alignment when greater. Generated C checks the base alignment and planned byte capacity before invoking providers. X-Ray records artifact_library as an on-disk path and hash. Its optional runtime_library record uses dladdr at capture time to identify the library containing the resolved symbol, with that path and file hash. These are distinct provenance claims. Symbol resolution and a file hash do not constitute an embedded runtime build-identity check.

Generated duration segment: a second circuit takes the pinned [36,50] PyTorch duration logits as an external FP32 activation. Normal v8 lowering binds the reusable logits-to-frames provider, its checked 103-frame result, feature expansion, and a downstream consumer into one generated native entry. The fixture checks distinct physical input strides, padded output rows, repeated requests, exact capacity, and failure propagation. Its feature values are synthetic. Phoneme IDs do not yet produce these logits inside CKE, and the generated entry does not produce a waveform.

Evidence still required

The pinned capture now records 63 named tensors, including six duration-encoder LSTM/AdaLN operation boundaries, duration logits, the duration vector, and the final FP32 waveform. Finer prosody/decoder checkpoints, source RNG inputs, imported-weight bindings, and a generated graph comparison remain required. Verify the exact generated output length, repeated calls, undersized workspace, malformed duration/state, finite outputs, clipping/silence, and a human listening fixture. Keep numerical parity, candidate-versus-baseline equality, and listening results separate. Until the generated C entry point produces and verifies the waveform, status remains unsupported / missing evidence.

Operation checklist for the first generated utterance

Scope: the pinned phoneme IDs and af_heart[len(phonemes)-1] row, batch one, FP32, and the short reference utterance. “Primitive oracle passed” describes an isolated operation under its own contract; it does not certify Kokoro's parameters, tensor connections, or waveform. “Candidate” means CKE has related math whose exact Kokoro contract still needs comparison. All TTS backward and complete waveform claims remain NOT_TESTED.

Reached stageEvidence available nowMissing work for the pinned pathNext independent proof
Phoneme IDs and voice conditioningPinned IDs and upstream captures exist in version/v8/tts/reference/. The offline importer now writes 599 effective FP32 model tensors and the selected voice row/slices into BUMPWGT5; an asset-backed test compares every tensor and reproduces the pinned upstream waveform.Bind the canonical weights and voice tensors to declared circuit edges. The bundle is import evidence only: no generated Kokoro model consumes it yet. Native G2P is a later milestone.Compare generated phoneme-encoder, duration and decoder checkpoints against the pinned PyTorch capture, then the final waveform.
ALBERT phoneme encoderCKE has embedding, GEMM, attention, softmax, normalization and activation providers. Their Kokoro-specific fit is a candidate, not certified.Declare embedding projection, shared/grouped ALBERT layers, masks, GELU variant, layer-norm epsilon and the final 768→512 projection in circuits. Resolve exact provider ABI and numerical contracts.Compare embeddings, one shared ALBERT layer, final encoder output and imported effective weights against pinned PyTorch checkpoints.
Duration and text-encoder recurrent blocksaudio_lstm_step_f32 has a single-step oracle; audio_lstm_bidirectional_scan_f32 has independent PyTorch scan fixtures. The pinned predictor capture now records each packed LSTM input/output, batch-size metadata, and final hidden/cell state.Connect forward and reverse scans in Kokoro circuits with exact weights, valid lengths and per-request state reset. The new checkpoint captures are upstream oracle data, not generated-C parity.Compare each generated recurrent stage and repeated-request reset against those pinned checkpoints.
Style-conditioned adaptive layer normaudio_adaptive_layer_norm_f32 has PyTorch primitive oracles, explicit strides/capacities and caller-owned scratch. The pinned reference now captures input features, predictor style and output for all three occurrences.Bind Kokoro's effective projection weights and predictor style in the circuit. The complete predictor graph remains NOT_TESTED.Compare each generated adaptive-layer-norm stage against its pinned input/output checkpoint.
Duration head and checked frame countaudio_duration_logits_to_frames_f32 passes the pinned 36×50 PyTorch fixture and native bounds tests; the generated duration subgraph binds its checked A to downstream calls.Generate the 50 logits from the pinned encoder and duration-predictor weights. The current subgraph supplies logits externally; request-time speed binding remains a separate generic entry contract.Compare generated predictor logits, per-token durations and A to the pinned reference; retain nonfinite, overflow and over-capacity rejection before consumers run.
Duration expansionaudio_duration_expand_channel_major_f32 passes committed and live PyTorch primitive oracles. The generated two-stream graph expands captured 640-channel duration features and 512-channel text features with one checked A, padded strides, repeated requests and X-Ray valid-region parity.Replace both supplied feature tensors with generated duration-encoder and text-encoder producers while preserving the same declared edges and physical layouts.Compare the generated producers and both expanded tensors against the captured alignment, then carry valid A into prosody and decoder stages.
Text-encoder convolutionsaudio_conv1d_channel_major_f32 and audio_conv1d_channel_major_grouped_f32 have PyTorch primitive tests. They cover bias, stride, padding and groups under their maps.Export effective weight-normalized weights; verify Kokoro's exact padding, layout and accumulation. Dilation and transposed convolution are not provided by those ABIs.Compare each text-encoder convolution, norm and recurrent output with pinned checkpoints at its real shape.
Prosody F0/N and decoder AdaIN blocksGrouped Conv1D, basic normalization and scaled residual add are reusable candidates only for matching suboperations.Implement InstanceNorm1d affine statistics plus style (1+gamma)*norm+beta, nearest upsampling, required dilated/depthwise/transposed convolutions and branch/concat edges. Keep each math operation in src/kernels/ and its ABI in a canonical map.Independent PyTorch oracles for each new primitive, then F0, N and decoder stage captures with valid A/2A extents.
Harmonic source and generatorNo certified complete Kokoro source or generator path exists.Implement nine harmonics, voiced mask, seeded random phase/noise or a declared captured-source input for deterministic arithmetic isolation, centered complex STFT, Snake, transposed-convolution stages and post-conv magnitude/phase. Preserve exact branch order.First compare with identical captured excitation; then certify native seeded source generation separately. Check all intermediate shapes and finite values.
Inverse STFT and FP32 waveformaudio_istft_mag_phase_f32 has bounded scratch, negative tests and independent PyTorch primitive fixtures. The generated fixture supplies its spectrum separately.Feed it the generator's actual magnitude/phase with exact periodic Hann, centering, length and overlap contract. Size final waveform and scratch from checked limits.Compare generator spectrum, output length and every finite waveform sample to the pinned reference, then save a native WAV for listening.
One generated native entry#566 proves circuit-declared entry → lowering → normal codegen → compiled execution for a synthetic bounded graph.Compose all phoneme encoder, duration, prosody, text encoder and decoder edges in v8 circuits. Existing math providers that return no status need generic checked adapters or an equivalent resolved call contract without duplicating their arithmetic. Keep caller-owned workspace and persistent weights separate from per-request state.Run fixed IDs through one generated entry without Python model scheduling; test provider-failure propagation, repeated requests and workspace bounds.

Remaining order: capture operation-level Kokoro inputs/outputs, certify missing AdaIN/convolution/source/generator primitives, then compose the complete circuit and compare its native waveform. The BUMP exporter, bidirectional scan and AdaLayerNorm providers are independent groundwork. Native text preparation, PCM playback queue and cancellation come after the fixed-phoneme waveform works.

Canonical Kokoro weight and voice export

version/v8/tts/export_kokoro_bump.py is offline import tooling. It verifies the pinned config, checkpoint and af_heart SHA-256 hashes; requires the captured Kokoro and PyTorch package versions; materializes 89 weight-normalized modules into effective FP32 weights; and writes aligned BUMPWGT5 payloads, a manifest, and a config sidecar. All 548 raw checkpoint tensors are accounted for. The pinned model constructor supplies 140 missing InstanceNorm affine defaults; the exporter validates their one/zero values and labels them synthesized. Tensor names use semantic CKE prefixes. The manifest records each source name, transform, shape, offset, payload hash, selected voice-row index and observed package-source hashes. The package-source hashes do not independently prove a match to the cited upstream code commit.

The dependency-free import contract runs in PR checks. The asset-backed test runs when CKE_KOKORO_MODEL_DIR points to the pinned snapshot; otherwise nightly reports SKIP with a reason. On the pinned local reference, it compared all 599 effective tensors and three voice tensors byte-for-byte after BUMP roundtrip, confirmed weight-norm removal does not change upstream inference, and reproduced the captured 61,800-frame FP32 waveform hash. This establishes import fidelity, not generated-C speech or native playback.

CKE_KOKORO_MODEL_DIR=/path/to/pinned/kokoro \
  python3 tests/test_v8_kokoro_bump_export_live.py
python3 version/v8/tts/export_kokoro_bump.py \
  --model-dir /path/to/pinned/kokoro \
  --output-dir /path/to/kokoro-bump

Numerical coverage

TTS kernel numerical coverage, 2026-09-24

This report records isolated inference providers and a Kokoro-specific extent preflight. It does not certify a generated Kokoro waveform or native text-to-speech. The provider maps declare backward and training unsupported.

Provider Dtype / direction Independent oracle Executable test PR / nightly registration Observed result
audio_duration_logits_to_frames_f32 FP32 logits → int32 durations and valid extent / forward only Pinned PyTorch sigmoid → sum → speed divide → round → clamp at production [36,50] shape tests/test_v8_audio_duration_logits_oracle.py and separate live tests/test_v8_audio_duration_logits_live.py .github/workflows/tts-kernels.yml; scripts/nightly_runner.py keys tts_duration_logits_oracle and tts_duration_logits_live PASS locally: exact 36-token durations and 103-frame extent, ties-to-even, padded stride, invalid input and capacity rejection without output writes. Live PyTorch comparison PASS in the pinned local environment. Generated composition is covered separately below.
Generated duration logits → checked frames → feature consumer FP32 logits and features / int32 durations / forward Pinned Kokoro/PyTorch duration logits and durations; independent repeated-feature expectation tests/test_v8_duration_logits_runtime_graph.py .github/workflows/tts-kernels.yml; scripts/nightly_runner.py key tts_duration_logits_runtime_graph PASS locally: normal v8 lowering and generated C produce 103 frames, preserve physical stride/padding, and stop downstream calls on invalid extent. Logit generation from phonemes and waveform output remain NOT_TESTED.
Generated logits → checked extent → both Kokoro feature expansions FP32 logits and captured features / int32 durations / forward Pinned PyTorch Kokoro feature, duration and frame-to-token captures; committed expected tensors indexed by the captured alignment tests/test_v8_duration_two_stream_runtime_graph.py; fixture packaging command python3 version/v8/tts/reference/build_duration_two_stream_fixture.py --capture-dir <pinned-capture-dir> .github/workflows/tts-kernels.yml; scripts/nightly_runner.py key tts_duration_two_stream_runtime_graph PASS locally: normal v8 lowering and generated C produce the pinned 103-frame duration and text streams, with input stride 40, output stride 128, untouched padding, repeated 36-frame requests, and no downstream writes or published extent on rejected duration. Compiler-declared X-Ray checkpoints compare both valid regions to the captured alignment with exact FP32 equality. X-Ray also records the on-disk artifact hash and the library containing the executed symbol; the resolved numerical contract remains explicit as unresolved. Logits and features are oracle supplied. Phoneme-to-waveform execution remains NOT_TESTED.
audio_duration_expand_channel_major_f32 FP32 + int32 durations / forward only PyTorch torch.repeat_interleave; committed PyTorch fixture tests/test_v8_audio_duration_expand_oracle.py; invalid extent and stride checks in tests/test_v8_kokoro_shape_bounds.py .github/workflows/tts-kernels.yml; scripts/nightly_runner.py key tts_duration_expand_oracle PASS: exact values on the fixed fixture, untouched padding, repeated call, and capacity rejection. Live PyTorch fixture check PASS in the local CKE venv.
audio_istft_mag_phase_f32 FP32 / forward only PyTorch torch.istft with centered periodic Hann; pinned Kokoro magnitude/phase capture tests/test_v8_audio_istft_oracle.py; version/v8/scripts/compare_kokoro_istft_native_v8.py .github/workflows/tts-kernels.yml; scripts/nightly_runner.py key tts_istft_oracle PASS: synthetic PyTorch fixture, minimum-frame and alternate-geometry fixtures, silence, DC/Nyquist, portable rejection checks, and live oracle locally. Pinned Kokoro primitive capture: 61,800 samples, max absolute error 8.4564e-7, mean absolute error 7.2885e-8.
kokoro_shape_bounds host preflight Size arithmetic / inference Independent mathematical boundary cases tests/test_v8_kokoro_shape_bounds.c via tests/test_v8_kokoro_shape_bounds.py .github/workflows/tts-kernels.yml; scripts/nightly_runner.py key tts_kokoro_shape_bounds PASS: checked limits and overflow cases locally.
Generated producer → length validation → consumer FP32 / forward Independent synthetic duration-expansion reference; committed PyTorch inverse-STFT fixture tests/test_v8_runtime_extent_lowering.py; tests/test_v8_checked_call_codegen.py; tests/test_v8_xray_numerical_parity.py .github/workflows/tts-kernels.yml; scripts/nightly_runner.py keys tts_runtime_extent_lowering, tts_checked_call_codegen PASS: #560/#566 generate, compile, and execute a bounded synthetic graph through normal v8 codegen. Tests cover valid extents, physical padding, repeated requests, capacity/alignment rejection, and stopping downstream calls on provider failure. This is not a Kokoro acoustic graph.
Generated Kokoro phoneme → waveform FP32 / forward Pinned Kokoro reference capture Pending PR B NOT_TESTED No generated waveform exists yet.
TTS backward FP32 / backward PyTorch autograd and finite differences Pending training work NOT_TESTED Inference-only provider maps.

The committed fixtures make cheap PR tests independent of a PyTorch installation. The nightly runner runs live PyTorch checks when available; if the dependency is missing, those individual checks are SKIP with a reason. The nightly JSON and dashboard expose the live-oracle suite as SKIP with a dependency reason while the committed-fixture suite remains PASS. A skipped oracle is not evidence of parity. The scalar providers have no multithread execution mode, so single-versus-multithread numerical comparison is not applicable. Performance and P3/Ryzen certification have not been run.

Reproduce the cheap PR checks:

python3 -m unittest \
  tests.test_v8_kokoro_shape_bounds \
  tests.test_v8_audio_duration_expand_oracle.AudioDurationExpandOracleTest.test_committed_torch_fixture \
  tests.test_v8_audio_istft_oracle.AudioIstftOracleTest.test_committed_torch_fixture \
  tests.test_v8_audio_istft_oracle.AudioIstftSafetyTest

Reproduce live PyTorch checks in an environment with PyTorch and NumPy:

python3 -m unittest tests.test_v8_audio_duration_expand_oracle tests.test_v8_audio_istft_oracle

The pinned-capture comparison requires the reference artifact path described in this page, “Pinned reference capture”.

Pinned reference capture

Kokoro v1.0 reference capture

This directory contains a single-utterance PyTorch oracle for native CKE Kokoro bring-up. The pinned model snapshot, source revisions, voice, text, seed, and conditioning rule are in kokoro_v1_reference.json. fixture_manifest.json records observed assets, dependency versions, phonemes, token IDs, tensor shapes, and file hashes from the canonical capture. The full .npy and WAV artifacts are currently in /tmp/cke-kokoro-v1-af-heart-reference. The script refuses network access.

Run with an environment containing the pinned Kokoro and Misaki source checkouts and their resolved Python dependencies. Supply the three files from the pinned Hugging Face snapshot at the paths listed in the pin manifest:

python version/v8/tts/reference/capture_kokoro_v1.py \
  --model-dir /path/to/Kokoro-82M-snapshot \
  --output-dir /tmp/cke-kokoro-v1-af-heart-reference

--preprocess-only needs just config.json and captures segmentation, G2P, and token IDs. Full capture also emits selected style, ALBERT and duration stages, F0/noise projections, text encoder, decoder, frame-to-token indices, float32 waveform, final ISTFT magnitude and phase, and a 24 kHz mono PCM16 WAV for listening. The .npy waveform is the numerical oracle; the WAV is a playable derivative. Both are checked for finite samples. Keep the captured fixture beside its generated manifest and check the source/dependency pins before comparing CKE output.

The first full capture took 5.51 seconds wall time and peaked at 1,134,048 KiB RSS on this host with one PyTorch thread. A second process produced identical tensor and WAV hashes with seed 0. Reconstructing from captured ISTFT magnitude and phase with PyTorch istft (n_fft=20, hop=5, periodic Hann) reproduced the float32 waveform exactly. oracle-requirements.lock records the 89 installed package versions and source URLs. It does not contain wheel hashes; the asset SHA-256 hashes are in fixture_manifest.json.

The fixture is upstream reference evidence only. Its evidence object leaves native primitive parity, native full waveform parity, human listening, and application playback as NOT_TESTED. Missing dependencies or assets stop capture with an error; they never count as passing evidence. Each production kernel needs its own oracle comparison and CI registration outside this reference directory.

The predictor now has six required fine hooks: three packed bidirectional LSTM calls alternating with three AdaLayerNorm calls. The capture adds 39 tensors for packed data, sequence metadata, hidden/cell state, adaptive-norm input/style and output. For this batch-one fixture the 36 packed batch sizes are all one, and sorted/unsorted indices are both zero; the generated scan must still bind the declared valid length and reset its own state. Repeating the pinned capture preserved all 24 previous tensor hashes and the 61,800-frame waveform. The manifest shape test runs in PR CI; the full recapture runs in nightly when the pinned local assets are available and reports SKIP with a reason otherwise. These checkpoints give generated circuits comparison targets; no generated predictor stage has passed them yet.

The asset-backed nightly gate also compiles the standalone audio_adaptive_layer_norm_f32 kernel and compares its three real predictor calls against these PyTorch checkpoints. Their maximum absolute errors were 2.38e-6, 1.43e-6 and 1.61e-6, each below the declared 5e-6 elementwise limit. This is real-weight primitive parity, not circuit stitching or waveform parity.

The selected style row is af_heart[len(phonemes)-1]; the predictor receives its last 128 columns and the decoder receives its first 128. The generated manifest marks any missing hook explicitly.

Image
100% | |
Scroll to zoom | Drag to pan | W/H to fit | 0 to reset | ESC to close