v8 Inference and Certified Training Runbook

Training kernel capability ledger

The capability ledger lists public cke.nn authoring modules separately from exact v8 provider-map operations. It groups maps by FP32, BF16, quantized, and unspecified dtype tags and distinguishes declared backward providers from registered tests. These inventories do not establish that an authored module lowers to those providers or that generated training passed numerical certification. In particular, a BF16 or quantized forward map does not establish a trainable backward path or QLoRA support.

python3 version/v8/scripts/build_training_capability_ledger_v8.py
make v8-training-capability-ledger-check

For executed evidence, open the current run's training_workflow.json and identity-bound ir_report.html; the ledger does not turn a registered provider into a PASS.

Frozen SVG training fixture

The second notebook uses the same cke.v8.compile workflow with one complete SVG document in each split. It uses exact byte token IDs, a five-layer FP32 dense/GQA graph, generated-C AdamW, PyTorch parity, fresh-process resume, and independently generated v8 inference. The source XML is checked before weight initialization and appears beside token IDs and loss curves in the identity-bound IR report.

 .venv/bin/python version/v8/examples/python_authoring_tiny_lm_v8.py \
  --fixture svg --run-dir version/v8/.cache/python_authoring/svg_fixture --execute

.venv/bin/jupyter lab version/v8/notebooks/02_frozen_svg_training_fixture.ipynb

The corpus is pinned in version/v8/training/svg_single_document_v1.json. Its train and validation sources are separate, complete SVG files. Token limits that would truncate either tokenized document are rejected before weight initialization. The current target policy is cyclic causal next-token prediction within each document, including a last-to-first token transition. Validation repeats almost all training geometry and changes only colors, so falling validation loss tests the machinery rather than generalization. The report separately marks whether each 48-token generated sample is well-formed SVG and whether it exhausted the probe budget; that diagnostic does not change the numerical certification verdict. Broader SVG quality needs a separate evaluation set.

Each additional supported graph inherits optimizer, checkpoint, parity, and exported-inference gates. The Python surface builds certified CKE graphs; it does not yet compile arbitrary torch.nn programs. RWKV remains outside this workstream.

Artifacts and visual inspection

The full workflow writes its current run under version/v8/.cache/training_workflow/qwen3_4layer_fp32/.

RUN=version/v8/.cache/training_workflow/qwen3_4layer_fp32

# The workflow creates this report. Regenerate it after inspecting or moving artifacts.
.venv/bin/python version/v8/tools/open_ir_visualizer_v8.py \
  --generate --run "$RUN" --html-only --strict-run-artifacts \
  --output "$RUN/ir_report.html"

# Open locally, or serve the run directory on a headless host.
xdg-open "$RUN/ir_report.html"
python3 -m http.server 7021 --bind 127.0.0.1 --directory "$RUN"

Supported Host

• Linux host with a working C toolchain and Python 3.
• Repo-local .venv with version/v8/requirements.txt, or let cks-v8-run bootstrap it interactively.
• A local Qwen3-VL mmproj file available at a known path.

python3 -m venv .venv
. .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r version/v8/requirements.txt

On Linux ARM, verify that the v8 engine links without unresolved symbols and that mixed BUMP allocation maps complete weight pages read-only. The unaligned weights/activations boundary is copied into one writable anonymous page.

make test-v8-arm-runtime
make test-v8-arm-model-smoke CK_ARM_MODEL_DIR=/path/to/generated-v8-model

The model smoke expects an artifact generated at context length 100. It forces file-backed weights and token-ID input with the full tokenizer disabled. This checks native forward execution, not raw-text tokenization, output quality, or performance parity. regression-fast-arm remains the broader family gate.

For low-RAM deployment, set CK_BUMP_FORCE_MIXED=1: Linux overcommit can let an anonymous reservation succeed before the process is OOM-killed. Mapping weights from disk does not prove that the full model will fit or run at useful speed. Page faults, activations and tokenizer memory are separate costs. On TDA4VM, the 2.33 GiB Nanbeige BUMP mapped successfully, but end-to-end Nanbeige inference has not been certified. A full 262k-token Gemma SPM tokenizer was OOM-killed; tokenizer-disabled token-ID inference passed. Test raw-text chat separately.

Current Large-Model Evidence

Support levels are intentionally separate: coherent execution does not imply bit-exact numerical parity, multimodal support, long-trajectory certification, or competitive performance.

Text-Family Notes

• Gemma 3: use --chat-template auto for normal instruction/chat runs. --chat-template none is raw continuation mode now and needs --allow-raw-prompt if you intentionally want it.
• Gemma 4: use --chat-template gemma4. The template uses explicit per-layer/direct split-half RoPE metadata; do not replace it with the older global split-half RoPE path when debugging.
• Nemotron Nano 9B v2: use --chat-template auto. The BPE runtime should encode raw prompts by default; set CK_DISABLE_FULL_BPE_TOKENIZER=1 only for token-id fallback/debug runs.
• GLM4: the real Q4_K_M path executes coherently and carries partial-RoPE, persistent FP16-KV, and llama.cpp parity coverage. Keep it in long-context and ISA regression sweeps.
• Instella-MoE: BF16 text executes coherently with strong short-checkpoint PyTorch agreement. Do not describe quantized GGUF or long trajectories as certified yet.
• Kimi-VL: the BF16 text decoder is coherent and repeatable; use --chat-template kimi_vl. Vision remains unimplemented, and the current X-Ray evidence is ranking parity rather than bit-exact internal parity.
• Laguna-XS 2.1: Q4_K_M text executes coherently and the mixed global/sliding circuit is map-owned. CKE preserves the GGUF's YaRN beta_fast=64; the current external llama.cpp Laguna loader defaults that value to 32. Even with a metadata-aligned diagnostic oracle, small attention/projection drift can change a near-tied MoE route, so long-trajectory parity remains open.
• Cohere2: pairwise RoPE applies to sliding layers only; the model-declared logit_scale runs as a final_logit_scale_f32 footer op. Long-trajectory parity and performance sweeps are not certified yet.
• Qwen2 / Qwen3 / Qwen3.5: the v8 runner now reproduces the same public command shapes as v7 and clean short prompt smokes succeed on the promoted examples.
• NaanBeige / llama-family symptom: if the first reply echoes <|im_start|>assistant or starts with <think>, keep --chat-template auto, do not force none, and treat it as a prompt-wrapper/chat-contract symptom rather than the expected reply.

Planned Automatic Policy

CK_V8_FORCE_BATCHED_PREFILL is a temporary correctness guard, not a permanent user configuration requirement. The intended user experience is automatic selection:

model metadata + quantization + prompt shape + CPU ISA
                       |
                       v
              certified kernel map
                       |
                       v
       optimized provider or safe fallback

After long-context numerical parity is closed: Qwen3.6 should automatically use batched prefill, CK_V8_FORCE_BATCHED_PREFILL should no longer be required for normal use, a disable/debug escape hatch may remain for developers, and nightly tests should verify automatic selection without user flags. Remaining performance targets are tracked in the Prefill Performance Roadmap, and kernel/provider coverage is summarized in the Model + Kernel Matrix.

Synthetic Prefix Probe

For a seam-only smoke test without a real image file, keep the decoder pair but switch to a synthetic prefix:

version/v8/scripts/cks-v8-run run \
  hf://Qwen/Qwen3-VL-8B-Instruct-GGUF/Qwen3VL-8B-Instruct-Q4_K_M.gguf \
  --mmproj ./mmproj-Qwen3VL-8B-Instruct-Q8_0.gguf \
  --image-mode checker \
  --prompt "Describe the image." \
  --context-len 1024 \
  --force-convert --force-compile

Practical Local CPU Workflow: YouTube Or Teams

Export meeting media through the service's authorized controls, or use yt-dlp for public or otherwise authorized YouTube content. Normalize either source to the same PCM16 contract:

sudo apt install ffmpeg
python3 -m pip install --user yt-dlp

mkdir -p "$HOME/cke-audio/results"

yt-dlp -x --audio-format wav \
  -o "$HOME/cke-audio/source.%(ext)s" \
  "YOUTUBE_URL"

ffmpeg -y \
  -i "$HOME/cke-audio/source.wav" \
  -ac 1 -ar 16000 -c:a pcm_s16le \
  "$HOME/cke-audio/input-16k.wav"

For a Teams, Zoom, or local recording, replace $HOME/cke-audio/source.wav with the exported MP4, M4A, MP3, FLAC, or WAV file. Transcribe with Whisper Base:

RUN_DIR="$HOME/.cache/ck-engine-v8/models/whisper-base-local"

CK_NUM_THREADS=20 OMP_NUM_THREADS=20 \
version/v8/scripts/cks-v8-run audio hf://openai/whisper-base \
  --run "$RUN_DIR" \
  --wav "$HOME/cke-audio/input-16k.wav" \
  --language en \
  --task transcribe \
  --max-tokens 448 \
  --output "$HOME/cke-audio/results/transcript.json"

jq -r '.decoder.transcript_text' \
  "$HOME/cke-audio/results/transcript.json" \
  > "$HOME/cke-audio/results/transcript.txt"

Audio and transcript data stay local unless the operator separately uploads them. Recordings longer than 30 seconds use sequential source windows. The 33-second parity fixture is certified; arbitrary hour-scale recordings remain functional rather than corpus-certified. Add --timestamps when timestamp tokens are required.

Short Exact And Long Nightly Gates

Nightly already covers the log-Mel frontend against PyTorch, the audio transformer primitives, and the portable v8 audio circuit/codegen regression. Run the short public checkpoint path through the same front door:

make test-audio

version/v8/scripts/cks-v8-run audio hf://openai/whisper-tiny \
  --wav recording.wav \
  --output build/whisper-short-report.json

The artifact gate requires the full reference token sequence and EOS without tolerance relaxation.

make test-whisper-long-audio-nightly \
  V8_WHISPER_LONG_AUDIO_MODEL=base

Scheduled CI uses a published five-minute MIC1 recording and a human-corrected transcript across Tiny, Base, Small, Medium, and Large-v3. Every model job is fail-closed: it verifies the lossless fixture, builds generated runtimes, requires complete sequential-window progress, bounds timestamps to the recording, and enforces its WER ceiling. The same manifest can add future audio model families without changing the evidence contract.

Encoder X-Ray Parity Probe

For checkpoint-level evidence across the encoder side of the bridge, compare_whisper_encoder_pytorch_v8.py walks the generated encoder stop by stop — stem conv1 → GELU → conv2 → token-major transpose → per-layer attention and MLP residuals — against a PyTorch reference, enforcing max-abs and relative-RMSE tolerances per checkpoint. It needs torch + transformers. The command below resolves the public Hugging Face checkpoint to its local snapshot before starting the comparison:

WHISPER_CHECKPOINT=$(hf download openai/whisper-tiny)

python version/v8/scripts/compare_whisper_encoder_pytorch_v8.py \
  --run-dir $CK_WHISPER_ENCODER_RUN_DIR \
  --checkpoint "$WHISPER_CHECKPOINT" \
  --stops key \
  --output build/whisper-encoder-xray.json

The stop table is derived from the architecture dimensions, not the model name, so the same probe carries to other audio transformer encoders. CI drives the same script through tests/test_v8_audio_encoder_contract.py in the nightly audio lane.

Native Server And FFI Boundary

See the circuit-derived serving coverage inventory for declared profiles and the artifact-level evidence still missing for each circuit.

External Python, Rust, or C hosts should bind include/ck_session_v8.h through build/libck_session_v8.so. The session loads the generated model once and exposes native chat formatting, encode, decode, generation, streaming token callbacks, cancellation, reset, timings, and a versioned capability descriptor.

make ck-session-v8
make test-native-session-v8

Independent requests reset KV and recurrent state by default. Continuation is explicit. A generated runtime missing the required tokenizer or chat capability fails closed instead of guessing from its filename.

Converted Jinja Serving Contract

GGUF conversion writes chat_template.jinja beside the generated runtime; safetensors conversion stages its tokenizer template assets. The live server loads the selected sidecar, renders the conversation with Jinja, and passes that raw prompt to the native tokenizer. A separate tool_protocol.json declares how tool output is parsed and binds the protocol to the selected template's SHA-256. An unreadable sidecar, empty render, or stale hash is an error, not a reason to silently switch prompt formats.

A cached GGUF conversion can recover a missing native sidecar by reading only GGUF metadata. It rejects a sidecar whose bytes differ from the source. Serving a complete converted bundle with --no-build needs no source GGUF, download, or conversion. Keep the generated libraries, BUMP weights, tokenizer, template sidecars, and manifest together when moving the bundle.

Normal serving requires a nonempty chat_template.jinja before opening the native session. To intentionally serve untemplated raw prompts, set both --no-chat-template and --allow-raw-prompt. Serving overrides use --chat-template-file PATH or --chat-template-inline SOURCE; the ambiguous --chat-template option is rejected by the serve command. An override replaces the native template and all tool variants. An existing tool_protocol.json must match that effective template or startup fails. String tool arguments preserve their payload whitespace; the native Qwen XML form removes only one paired framing newline. Unsupported tool-schema forms fail before generation, while supported arguments are checked against their declared schema.

The cached Qwen3.8 Q4_K_M runtime has a known native-template tokenizer mismatch: its tool delimiter is split into ordinary CKE tokens while the upstream tokenizer uses one special token. That numerical gap remains open. A separate Ryzen probe with the publisher Jinja and native qwen_xml output protocol nevertheless completed a Qwen Code read_file continuation and an HTML read_file → edit task using the harness's ordinary system prompt. The compatibility template installation path has therefore been retired; its bytes are retained only as a legacy protocol test fixture. The qwen38.json circuit owns an explicit serving.profile_ref to this reusable profile. Bundle construction resolves the selected variant together with imported publisher assets, then packages content-addressed templates and circuit/profile snapshots in the deployment bundle. The server reads only that bundle's serving.json and assets; it never searches repository profiles or server/templates/ at request time. Publisher templates are the shipped profile's only variant. Task success does not certify exact native token IDs, cancellation recovery, or compatibility with other artifacts.

A separate native-BPE input limit once reduced a 33,379-byte Qwen Code tool prompt to 304 CKE tokens instead of the independent GGUF tokenizer's 7,773 tokens. A rebuilt model, engine, and tokenizer matched all 7,773 IDs and completed a cached read_file and tool-result continuation. Rebuild those three libraries together; replacing only the tokenizer under an older generated model is unsafe. The declared qwen_code_xml_raw_v2 protocol accepts a short plain-text introduction followed by one terminal XML envelope. It still rejects inline examples, fenced XML, trailing text, and unadvertised tools. If a request sets parallel_tool_calls=false, multiple invokes fail atomically; the server does not forward the first call while dropping the rest. Its delimited parameter bodies preserve raw HTML and code for edit arguments; a literal closing </parameter> inside a value is outside this compatibility protocol. The older qwen_code_xml protocol keeps its strict whole-response XML/entity interpretation for existing bundles; opt in to v2 only for existing explicitly configured compatibility bundles. Legacy bundles without serving.json retain their hash-bound tool_protocol.json path. One cached Qwen Code run recovered from a read-before-edit rejection, read the file, and applied a bounded HTML edit in a disposable worktree. That does not certify general edit-and-test reliability, cancellation recovery, or 262K actual input.

RUN_DIR=/path/to/dedicated/qwen38-generated-runtime
# Explicit metadata upgrade of an existing, complete generated bundle.
# Publisher assets are the default; no repository template installation is needed.
python3 version/v8/scripts/resolve_serving_bundle_v8.py \
  --run "$RUN_DIR" --circuit version/v8/circuits/qwen38.json
version/v8/scripts/ck_serve_v8.py "$RUN_DIR" --run "$RUN_DIR" \
  --no-build --model-name qwen38-27b-q4km --host 127.0.0.1 --port 8080

For a new chat bundle, ck_run_v8.py run ... --generate-only --resolve-serving resolves the circuit profile after compilation. Normal chat serving requests this automatically. Raw/token-ID builds do not require publisher chat assets; omit --resolve-serving or build with --no-chat-template. Unsupported selections fail explicitly. The circuit's publisher variant is the default. Safetensors tokenizer configuration can supply embedded or named publisher templates; conflicts with colocated Jinja files fail instead of choosing one silently. Startup validates template, tokenizer, weights, configuration, layout, and generated-library hashes, so hashing large weights is part of setup time. Rebuilding these assets requires explicit re-resolution. Explicit upgrades verify the retained manifest → IR → generated C → runtime stamp chain against the current bundle bytes. A matching model name alone is insufficient. Missing build provenance requires a rebuild. Computational circuit changes are rejected; only changes confined to the serving section may be allowed with --allow-serving-update. A missing or invalid resolved asset never falls back to a legacy template. Raw serving skips formatting but still validates a present resolved bundle and actual loaded libraries. Runtime template overrides are rejected for resolved bundles. The sidecar marks certification NOT_TESTED; dependency resolution alone is not a model/tool certification. Freshly compiled bundles also bind selected kernel runtime dependencies to the compiled kernel registry. If a selected provider needs Sleef_expf16_u10, startup checks CK_SLEEF_LIBRARY (or an already loaded process symbol) before opening HTTP traffic. Rebuild and re-resolve older bundles to gain this check; changing an existing server process does not upgrade its loaded runtime.

Run make test-v8-serve-native-jinja for the deterministic sidecar-loader, Jinja-renderer, request/continuation, hash, and render-failure contract. The nightly v8 Serve Localhost E2E also requires the converted Jinja sidecar in a real Qwen3-0.6B generated-model request. The pinned Qwen3.8 contract uses scripted model output; its green result does not certify Qwen3.8 token IDs or generated tool calls. Cached Ryzen Qwen Code 0.24.6 runs completed server-issued read_file and bounded read_file → edit calls through this variant. A write_file request was rejected because that tool was absent from the client's advertised tool set. File creation and native-template prompt-token parity remain open.

The generated decoder now has an optional, serialized sequence-state switch ABI for layouts whose compiled activation lifetimes declare only KV as persistent mutable sequence state. It permits two additional generation-tagged state IDs backed by caller-owned, 64-byte-aligned KV arenas. The ABI reports the exact bytes required; the caller budgets and retains each arena through retirement. KV contents and positions survive switches. Recurrent, encoder, and multimodal layouts do not advertise this capability. The server does not use these handles yet. An additional generated ck_model_decode_batch2 entry now admits two one-token rows only when the KV-only compiled layout and kernel map declare compatible Q5_1/Q8_1 projections. The first layer's Q and K projections share one M=2 weight traversal; the remaining layers and attention execute with separate sequence state. This is actual shared generated projection work, but not a full-layer batched decoder, a server scheduler, mixed prefill/decode, or continuous batching. For a non-default handle, named KV pointer lookup resolves to that handle; a bump-arena offset is unavailable and returns -1. Run python3 version/v8/scripts/certify_sequence_state_v8.py --run RUN_DIR against a freshly generated eligible bundle. A 128-token Gemma3-270M Q5_K_M build passed eight exact finite-logit comparisons, including reset isolation and slot reuse; the retained JSON report records the library, weights, layout, and config hashes. This bounded result does not cover concurrent calls, recurrent models, cancellation, or throughput.

Run python3 version/v8/scripts/certify_batch_decode_v8.py --bundle BUNDLE --generated-source MODEL_C against a freshly generated eligible bundle. The retained Gemma3-270M Q5_K_M report compares full finite logits and complete KV bytes with isolated prefill/decode: seven continuations matched exactly across unequal prompts, swapped row order, one-row continuation, retired-slot reuse, pre-step cancellation, capacity rejection, and rejected buffer aliases, including a third live sequence arena. Admission now checks the split's crossing values and exact base-pointer expressions, and binds the M=2 provider to the selected decode kernel map. The report identifies actual loaded model, engine, and tokenizer mappings and checks the generated source against the loaded model's source digest. CI compiles and executes a tiny generated-entry fixture and tests rejection after a loaded library pathname is replaced. Its timings are diagnostic workloads, not a speedup measurement. Recurrent Qwen, larger artifacts, simultaneous API calls, and HTTP request scheduling remain uncertified for this batch entry.

The next generated step shares the first layer's MLP gate/up Q5_1 projection as a second two-row weight traversal after sequence-local attention and KV writes. The compiler derives both cutpoints' live values from lowered IR without depending on argument order. Only validated full decode-row writes retire earlier values; unknown or partial writes prevent the extension. It checks the gate/up provider and GeGLU consumption contract, and retains the Q/K-only entry for layouts that do not qualify. The separate batch_decode_gemma3_270m_two_row_layer_20261003.json report records seven exact full-logit and KV comparisons from a freshly rebuilt local bundle and requires both shared projection groups. A compiled fixture also compares the actual emitted split functions with isolated execution. In that earlier two-group build, V, output and down projections remained sequence-local; it made no full-layer or throughput claim.

The next bounded generated entry derives four projection groups and their crossing tensors from the first layer's lowered graph. Q/K/V, attention output, gate/up and MLP down use map-selected two-row providers; attention, KV writes, normalization and all later layers remain sequence-local. The planner rejects unsupported provider bindings, pointer offsets, new live values and persistent state outside the declared KV contract. The generated batch entry validates both handles, positions, capacity, cancellation and buffer ownership before work starts; a rejected step advances neither sequence. Once admitted, both rows finish the forward pass. Cancellation is observed at the next step boundary, not inside a running kernel. The retained batch_decode_gemma3_270m_full_layer_20261007.json report requires four groups and records seven exact full-vocabulary logit and complete-KV comparisons on a newly generated Gemma3-270M Q5_K_M bundle. On the local i7-14700T, three artifact-identical diagnostic repetitions of six matched decode tokens took 183.1–184.3 ms as isolated sequential calls and 57.8–65.5 ms in three two-row steps. The base report and its _run2 and _run3 siblings preserve each attempt. These tiny samples are not an overnight-task or generalized speedup claim. The ordinary server path still uses single-flight execution; later layers, recurrent Qwen state, dynamic admission and mixed prefill/decode remain separate milestones.

The subsequent two-row generated entry extends the checked graph-derived plan across consecutive complete decoder layers. On the identified Gemma3-270M Q5_K_M artifact, all 18 supported layers execute four shared map-selected projection groups per layer (72 total); attention, KV writes and positions remain sequence-local. The plan checks values live across every split against the complete remaining graph. It rejects an unsupported layer and leaves the rest of the decode path local rather than assuming an unproven buffer extent or provider binding. Compact row packing is required where a selected provider stride differs from the live activation extent.

Run certify_batch_decode_v8.py with --expected-shared-layers 18 --expected-projection-groups 72 against the identified local bundle and matching generated source. The retained batch_decode_gemma3_270m_multi_layer_20261008.json report compares complete finite logits, KV bytes and positions after each of seven continuations: unequal prefixes, swapped rows, retirement and slot reuse, cancellation, capacity rejection and recovery. A small compiled two-layer regression runs in the standard test suite and includes deliberately broken restoration and cache-ownership controls. On the local i7-14700T, three artifact-identical repetitions measured six isolated native decode calls at 59.4–62.1 ms versus three two-row native calls at 70.6–76.8 ms. Python validation and output copying are outside both timing boundaries. Thus this small-artifact batch path was slower on the matched workload; it is a correctness milestone, not a speedup claim. The reports record workspace and copy volume. An optional monotonic-clock copy profile measured 0.748 ms of copying across three instrumented steps; its instrumentation perturbs full-step timing. Ordinary HTTP serving remains single-flight; recurrent Qwen state, request scheduling and mixed prefill/decode still need their own contracts.

Experimental two-request HTTP decode

--batch-http-extra-bytes BYTES --batch-http-prompt-cap TOKENS --request-output-cap TOKENS opts into a separate two-slot Responses path for a generated KV-only batch-capable model. It uses the loaded bundle's selected Jinja, tokenizer and declared tool parser. One worker owns native prefill and decode while each request has separate cancellation, detokenization, events and output. A third active slot receives HTTP 429; the ordinary single-flight path remains the default.

--batch-http-decode-replay-chunk-tokens N additionally opts into staged prefill. The owner computes at most N prompt tokens through generated single-token decode, then gives ready requests a decode turn before the next chunk. Cancellation has a ticket before prompt work starts. This prevents a whole prompt from monopolizing the owner, but one slow token operation still sets the cancellation/latency granularity. Decode replay can differ numerically from the artifact's ordinary batched prefill and requires artifact-specific parity testing. The effective serving record names the selected prefill strategy and chunk budget. Add --batch-http-mixed-one-row to allow one ready decode token and one staged prompt token to enter the same generated two-row decode call. This is a two-slot, one-prompt-row bridge, not variable-size chunk execution or a general batch-N contract. Both options remain opt-in; the mixed flag requires staged prefill. The local Gemma3-270M Q5_K_M generated-runtime check compares finite full logits at every step, complete KV bytes and positions against serialized decode replay for either prefill slot. It covers consecutive mixed steps, continuation after prompt completion and a 32-token session-capacity boundary. A separate optional comparison against the ordinary generated prefill entry fails on the pinned artifact: 9 tokens match exactly, but 10 tokens change both full logits and KV bytes; at 28 tokens, the top predicted token also changes. The --compare-ordinary flag records this as a failing, artifact-bound diagnostic in version/v8/reports/batch_prefill_gemma3_270m_20261011.json. The cause and which path matches the publisher reference remain unverified. Mixed-step parity against decode replay does not certify ordinary-prefill parity or a resolved serving deployment.

The optional --compare-ordinary --oracle-gguf GGUF_PATH diagnostic compares complete logits using the same token IDs. The retained result is version/v8/reports/batch_prefill_gemma3_oracle_20261012.json. It verifies the helper's actual loaded llama.cpp library mappings, records their backing-file hashes and effective settings, and repeats each llama.cpp mode twice. Seven selected GGUF/BUMP tensors match at beginning, middle and end samples; nine model metadata fields match. Full conversion and build provenance remain unverified.

The comparison has an earlier boundary than the CKE prefill split: at one token, CKE and llama.cpp select different top tokens. At 10 tokens, llama.cpp batched and sequential evaluation themselves select different top tokens (107 versus 1018), with 1.248 full-logit RMSE; repeated runs of each mode are exact. The one-token llama top token is unchanged with one or four threads and with repacking on or off. The helper requested 64 context tokens but used 256 after runtime adjustment, while CKE's session capacity was 32. These differences prevent treating either llama.cpp mode as a numerical arbiter. CKE ordinary and staged prefill first diverge at 10 tokens; an internal hidden-tensor inspection places that CKE-only difference after matching layer-16 Q/K/V inputs. The cross-runtime one-token mismatch is not explained by that later difference. No numerical path is promoted, and batching remains experimental on this artifact.

A separate resolved Gemma3-270M Q5_K_M chat-only HTTP acceptance is available through server/batch_http_acceptance.py. It checks expected serving and loaded session hashes before generation, compares two simultaneous greedy streams with their isolated outputs, requires mixed-step evidence attributed to both response IDs, then cancels one request while its peer continues and serves a follow-up on the same instance. The bounded /v1/cke/batch-observation/{response_id} endpoint identifies the exact peer sharing each generated step; global counters alone are diagnostic. The probe requires an idle owner, records synthetic prompts, settings, output hashes and terminal reasons, and cancels outstanding requests on failure. The retained local run observed 27 attributed mixed steps with matching outputs, peer continuation and recovery. The tested artifact exhausted its output limit, so model-generated normal EOS is explicitly not certified; deterministic stream tests cover normal EOS and duplicate-terminal rejection. This certifies the tested HTTP/native boundary only; Gemma3's profile declares chat without tools, so it is not a completed agent harness task or an ordinary-prefill parity result. Keep a full tokenizer.json beside this GGUF-derived artifact when its publisher template requires bos_token; the resolved bundle hashes that asset and otherwise fails closed at rendering.

Native slot budgeting does not bound HTTP delivery by itself. The opt-in path also limits undelivered response consumers (default 4), output token events (default 4096), decoded output bytes per response (default 1 MiB), and time for each stream write (default 30 seconds). Configure these with --batch-http-delivery-consumers, --batch-http-output-token-cap, --batch-http-output-byte-cap and --batch-http-delivery-timeout. A stalled stream is cancelled without blocking the shared native owner; an exceeded output-byte cap fails that response. Token event queues are bounded and terminal failure notification cannot wait behind a full queue. An owner failure poisons the owner, wakes active and queued consumers, and rejects new submissions rather than leaving a request waiting forever.

This is a bounded experimental integration, not certified overnight concurrency. Without the staged flag, prefill remains synchronous and cannot be interrupted inside one slow native operation; a prompt-token cap is not a time bound. Native per-request phase timing is reported unavailable. A local generated Gemma3-270M diagnostic matched isolated and concurrent output with 11 shared decode steps, but it did not use a resolved deployment bundle or an external harness. Certify two real harness tasks and certify staged prefill on the deployed artifact before promoting this path. Recurrent Qwen state is outside this KV-only capability.

Qwen3.5 publisher templates and request ownership

The Qwen3.5 circuit references the shared qwen-tools-v1 serving profile. Conversion packages the publisher Jinja and tokenizer assets into the resolved BUMP serving bundle. The declared qwen_xml parser handles native tool envelopes; it does not constrain the model's next-token sampling. Grammar-constrained generation remains a separate capability. Template rendering, token-ID agreement and actual tool completion each require their own evidence.

Reasoning starts from the generation suffix isolated by rendering the selected template with and without add_generation_prompt. Literal reasoning tags in user messages or tool results cannot choose the initial parser state. Tool-bearing streams expose reasoning while withholding tool calls until validation. Over-limit calls are rejected together; the server never silently executes a retained subset.

The Responses API's usage.output_tokens_details.reasoning_tokens is currently an estimate: the server divides visible reasoning text length by four and rounds down. It is not an exact count of generated reasoning token IDs and should not be used for billing or context admission. reasoning.effort=none selects the server's suppressed reasoning mode; actual model behavior remains artifact-specific. Other accepted effort labels do not select different compute budgets. Accepted service_tier values are echoed in the response but do not change native scheduling priority or model speed.

Typed images, files, audio and video are rejected before rendering or native execution in this text-only session ABI, including media returned by tools. Publisher image-marker rendering tests are render-only evidence. Plain paths remain text. Enabling vision requires a connected preprocessing → generated encoder → bridge → decoder path, not positive layout dimensions or template words.

429 Session busy means the single-flight native slot is occupied. Native worker completion now releases its request-owned slot independently of HTTP stream consumption. Disconnects request cancellation but retain ownership until native work finishes; an old response cannot cancel a newer request. A legitimate active request can still produce 429. Long-prefill cancellation may exceed its acknowledgement deadline: a cancellation timeout must not release buffers still in use or be described as successful bounded cancellation.

Newly generated text runtimes can accept a request-owned cancellation flag and poll it between lowered prefill operations or sequential prompt tokens. The native session reports cancellation before sampling and resets partial model state before reuse. Older generated libraries do not have this hook. A slow individual kernel can still delay acknowledgement, so a real-model prefill cancellation result is required before claiming bounded latency for a deployed bundle.

Black-box HTTP lifecycle acceptance

Run this probe against the intended live endpoint after deployment. It uses only HTTP: cancel a long-input request after its in-progress event and a decoding request after observing an output delta, close a stream, close a client after its read deadline, then serve a follow-up request after each case. It checks the loaded bundle, native session library and server instance before and after the run. A mismatched identity fails before generation; a missing run leaves no PASS report. The JSON records whether follow-up generation completed or exhausted its output allowance. The long-input cancellation is a timing inference, not native prefill-phase proof; the report labels its trigger and phase evidence explicitly.

python3 server/http_lifecycle_acceptance.py \
  --endpoint http://127.0.0.1:8080/v1 \
  --model <advertised-model-id> \
  --expected-serving-identity <serving.json identity> \
  --expected-session-library-sha256 <loaded session-library SHA-256> \
  --output /path/to/new-http-lifecycle-report.json

The probe is a transport and ownership check, not a tool-use or answer-quality certification. It requires a generated runtime with the cancellation hook and a local plain-HTTP endpoint; use the separate Qwen Code acceptance runner to test model-generated tool calls and task completion. Schedule the two on a local runner at separate times so the single-flight endpoint is not deliberately contended; retain each dated report and show a missing run as NOT_TESTED, never PASS.

The versioned native session ABI borrows request buffers and callback data only while a synchronous call is active. Cancellation requests a stop; generation returning acknowledges that native work has stopped. The current generated-model state is not certified for concurrent sessions in one process, so another HTTP host must preserve single-flight ownership until a separate sequence-state contract is proven. See include/ck_session_v8.h for the buffer and thread rules.

In an isolated Ryzen native-session probe on 2026-09-30, regenerated Qwen3.8-27B Q4_K_M code acknowledged a long-prefill cancellation in 0.55 seconds and completed a one-token follow-up in 3.24 seconds without reopening the session. Its uncancelled 64-token logits matched the previous generated library byte-for-byte. The retained report is /srv/cke/profiles/qwen38-prefill-cancel-20260930/probe.json. This probe does not certify HTTP disconnect timing, all prompt lengths, or other model/provider combinations.

The 2026-09-28 Qwen3.8 overnight documentation trial failed after approximately five hours and eighteen successful inspection calls with repeated busy responses. It produced no HTML/SVG outputs. This is retained failure evidence; short read/edit acceptance does not establish unattended reliability. Cheap lifecycle, media rejection and pinned-template tests run in the existing serving suite; real-model acceptance is recorded separately.

A separately rebuilt Qwen3.5-0.8B Q4_K_M bundle exposed an imported tokenizer-type mismatch: its tool delimiters were split into ordinary BPE pieces. Registering the publisher's CONTROL and USER_DEFINED token types produced exact generated-runtime agreement with the reference tokenizer for the tested tool prompts (327/327 and 325/325 token IDs). That bundle completed a bounded read_file request and tool-result continuation on both Responses and Chat Completions routes, with thinking suppressed and visible. A real Qwen Code 0.24.6 run received the correct file contents but repeated the read until its three-call budget stopped it; this is not a completed harness-task certification. These results apply to the tested 0.8B artifact, not every Qwen3.5 variant.

Qwen Code Interactive And Overnight Profiles

On ARM, an explicit CK_ARM_ARCH used to build the engine is retained by ordinary cks-v8-run run and serve invocations in the same build tree. The runner checks compiler support, rebuilds with that profile, and records it with the bundled library hashes. Set a new CK_ARM_ARCH to change the target, or set CK_ARM_NATIVE=0 explicitly to request baseline ARM. If an existing engine has no valid build-flag record, choose a profile explicitly before rebuilding; do not infer its ISA from its filename.

A cached Qwen Code 0.24.6 read_file task passed through the Responses server using the circuit-resolved publisher Jinja bundle. The earlier 0.21.5 pilot used the Chat Completions compatibility route. Both use the pinned Qwen3.8 27B Q4_K_M artifact and a generated runtime with at least 16K context. Repository-owned settings separate short interactive work from large overnight artifact generation without replacing the user's ~/.qwen/settings.json.

CK_NUM_THREADS=16 OMP_NUM_THREADS=1 \
version/v8/scripts/cks-v8-run serve \
  hf://ggml-org/Qwen3.8-27B-GGUF/Qwen3.8-27B-Q4_K_M.gguf \
  --run /path/to/qwen38-agent-runtime \
  --context-len 16384 \
  --force-compile \
  --model-name qwen38-27b-q4km

OPENAI_API_KEY=cke-local-only \
OPENAI_BASE_URL=http://127.0.0.1:8080/v1 \
OPENAI_MODEL=qwen38-27b-q4km \
QWEN_CODE_MAX_OUTPUT_TOKENS=2048 \
QWEN_CODE_SYSTEM_SETTINGS_PATH="$PWD/server/qwen-code/interactive.settings.json" \
qwen --bare \
  --auth-type openai-responses \
  --system-prompt 'Use read_file exactly once when asked, then answer without another tool call.' \
  --allowed-tools read_file \
  --exclude-tools edit,notebook_edit,run_shell_command,get_goal,update_goal \
  --max-tool-calls 1 \
  --model qwen38-27b-q4km

OPENAI_API_KEY=cke-local-only \
OPENAI_BASE_URL=http://127.0.0.1:8080/v1 \
OPENAI_MODEL=qwen38-27b-q4km \
QWEN_CODE_MAX_OUTPUT_TOKENS=32768 \
QWEN_CODE_SYSTEM_SETTINGS_PATH="$PWD/server/qwen-code/overnight.settings.json" \
qwen --bare \
  --auth-type openai-responses \
  --allowed-tools read_file \
  --exclude-tools edit,notebook_edit,run_shell_command,get_goal,update_goal \
  --max-wall-time 18h \
  --model qwen38-27b-q4km \
  --output-format stream-json \
  --prompt "$(cat /path/to/reviewed-task.txt)" \
  > /path/to/task-events.jsonl

The circuit-linked serving profile resolves the imported publisher Jinja, tokenizer assets and declared tool-output protocol into the BUMP bundle. The server consumes that resolved contract and rejects stale hashes; it does not install a repository compatibility template at request time. Tool-bearing streams publish parsed calls only after validation. A complete bundle can serve with --no-build without the source GGUF or a server-owned template copy. Publisher special-token variables such as bos_token come from the bundle's hashed tokenizer metadata. Missing variables remain undefined and conflicting metadata fails; the renderer does not invent model delimiters. The standard strftime_now helper uses the host's current date when a publisher template requests it. This supports publisher text-chat rendering. Scripted Muse ATEM parsing and pinned prompt-token comparisons are covered separately; they do not certify Muse-generated tool calls or a completed harness task on a current bundle.

Gemma4 references gemma4-v1, with a chat-only default variant and an explicit tools variant. The default output protocol is none: ordinary text chat can resolve the publisher Jinja asset, while tool-bearing requests fail before generation. When HF token sidecars are absent, verified GGUF special-token IDs are matched to bundled tokenizer.json text; conflicting definitions fail at startup. GGUF configs may carry model_type and model_name without a model key; bundle resolution then checks the stamped manifest's circuit identity. This declaration alone does not certify a Gemma4 artifact. Keep rendered-byte parity, token-ID parity, generated chat, tool protocol and recovery as separate artifact-specific results.

A fresh isolated Gemma4 E4B GGUF build exposed two native BPE differences from the bundled publisher tokenizer: its Jinja prompt already contains <bos>, and its tokenizer declares add_space_prefix=false. Generated tokenization now honors both contracts. Exact IDs matched the bundled tokenizer.json for single-turn, multi-turn, and Unicode/multiline rendered prompts in the isolated Ryzen run. The same isolated artifact passed short generated chat through Chat Completions and streaming Responses. The repeatable HTTP probe also checks multi-turn recall, a deliberately exhausted one-token output budget, undeclared-tool rejection and a successful request afterward. The existing lifecycle probe passed cancellation, disconnect, client timeout and follow-up recovery against this 2,048-token build. The timing-only prefill trigger proves an in-progress request, not the exact native phase at cancellation. These are bounded text-serving checks, not a harness task.

The imported publisher template contains Gemma4's <|tool_call>, <tool_call|> and <|tool_response> syntax, as described in Google's Gemma4 prompt format. The optional gemma4_dsl_v1 variant selects a strict parser, adapts function declarations to the publisher's nested Jinja input, and declares <|tool_response> and <eos> output stops in the resolved bundle. One isolated Gemma4 E4B Q4_K_M artifact completed real generated tool-call/result continuations through both HTTP routes. The circuit default remains chat-only; selecting the tools variant and testing one artifact does not certify every Gemma4 build.

For clients such as the pinned Antigravity SDK 0.1.20 that send a very large max_completion_tokens upper bound, an operator may explicitly start this bounded Gemma server with --request-output-cap 256. The normal policy still rejects requests whose declared allowance exceeds the loaded context. With the cap enabled, the effective allowance is recorded in the response and /v1/cke/loaded-identity; exhausting it remains an incomplete response. The SDK's external OpenAI-compatible configuration must name this local endpoint and the loaded model. No cloud fallback or model-name substitution is part of this test.

# Run an already resolved, self-contained Gemma tools bundle on Ryzen.
# Verify serving.json selects the tools variant before starting; do not
# silently replace the publisher template at request time.
GEMMA_RUN_DIR=/path/to/identified/gemma4-tools-bundle
python3 version/v8/scripts/ck_serve_v8.py "$GEMMA_RUN_DIR" \
  --run "$GEMMA_RUN_DIR" --no-build --model-name gemma4-e4b-q4km-tools \
  --host 127.0.0.1 --port 18205 --request-output-cap 256 \
  --request-prompt-byte-cap 32768

# On the harness machine, tunnel the localhost-only endpoint from Ryzen.
ssh -N -L 127.0.0.1:18215:127.0.0.1:18205 ryzen
# GET http://127.0.0.1:18215/v1/cke/loaded-identity and confirm the expected
# model, gemma4_dsl_v1 protocol, effective cap and artifact hashes.

On the isolated 2,048-token tools bundle, Antigravity SDK 0.1.20 connected directly to CKE with --request-output-cap 256 and completed a file read, two-file answer, exact HTML edit plus validator, and retry after a transient tool error in two consecutive four-case runs. Both reports and disposable workspace archives are retained under /srv/cke/profiles/gemma4-tools-20261003/. Two earlier runs failed the single-file answer despite a correct tool call and result; those failed reports are retained too. The passing runs used a more specific task prompt. This establishes a bounded successful workflow, not repeated overnight reliability or larger-context capacity. The HTTP cancellation/disconnect suite passed on the same server instance; a fresh SDK read then succeeded without restart.

# Read both SHA-256 values from /v1/cke/loaded-identity first.
python3 server/http_lifecycle_acceptance.py \
  --suite plain-chat --endpoint http://127.0.0.1:18204/v1 \
  --model gemma4-e4b-q4km-test \
  --expected-serving-identity "$SERVING_SHA" \
  --expected-session-library-sha256 "$SESSION_SHA" \
  --output /path/to/new-plain-chat-report.json

# The same artifact can run the existing cancellation/recovery suite.
python3 server/http_lifecycle_acceptance.py \
  --suite lifecycle --prefill-lines 100 \
  --endpoint http://127.0.0.1:18204/v1 \
  --model gemma4-e4b-q4km-test \
  --expected-serving-identity "$SERVING_SHA" \
  --expected-session-library-sha256 "$SESSION_SHA" \
  --output /path/to/new-lifecycle-report.json

# Against the explicit Gemma tools variant, with the endpoint's output cap set to 256.
# Install google-antigravity==0.1.20 in an isolated environment on the client.
python3 server/gemma_antigravity_acceptance.py \
  --endpoint http://127.0.0.1:18215/v1 \
  --model gemma4-e4b-q4km-tools \
  --expected-serving-identity "$SERVING_SHA" \
  --expected-session-library-sha256 "$SESSION_SHA" \
  --case-timeout-seconds 300 \
  --workspace /path/to/new-disposable-workspace \
  --output /path/to/new-gemma-antigravity-report.json

Bounded local Qwen Code acceptance

Use the same Qwen Code client as a real contributor task in a new disposable directory. This command reads a small HTML file, edits one heading, runs its Python validator and asks for a summary. The runner checks tool IDs, arguments and results against both the event stream and retained Qwen Code conversation, requires terminal success after tool completion, and checks the exact file content. Extra reads of the two fixture files are allowed; unexpected file paths or validation commands fail the retrospective acceptance check. These checks do not sandbox tool execution. Use a restricted account or container when the harness must be isolated.

python3 server/qwen_harness_acceptance.py \
  --endpoint http://127.0.0.1:8080/v1 \
  --model qwen38-27b-q4km \
  --expected-serving-identity <serving.json identity> \
  --expected-session-library-sha256 <libck_session_v8.so SHA-256> \
  --settings server/qwen-code/interactive.settings.json \
  --run-dir /tmp/cke-qwen-acceptance-001

The runner requires an empty run directory and writes events.jsonl, stderr.log, the edited fixture and report.json. A PASS proves this bounded harness task on the named endpoint. For a resolved bundle, read the intended identity field from its serving.json and pass it with --expected-serving-identity. The runner checks GET /v1/cke/loaded-identity before launching Qwen Code and again after the task. That endpoint reports the verified bundle identity, asset hashes, native session-library hash and server-instance ID without filesystem paths. The session hash comes from the backing file of the loaded native session symbol; deleted, replaced or ambiguous mappings are rejected at startup. Supply the expected session-library hash to certify that runtime ABI as well. Every supplied hash is checked independently: a mismatch prevents task launch or fails certification, while an absent expectation leaves certification incomplete. The endpoint reports effective_serving separately from the bundle: configured raw or templated mode, selected output protocol, active context limit, default output allowance, and stop behavior. The runner requires that snapshot to remain stable across its task. A legacy endpoint with no loaded identity, or a run without both expected hashes, remains INCOMPLETE even if its bounded task passes. The acceptance command exits successfully only when both the task and artifact certification report PASS. The endpoint is a startup verification snapshot, not continuous file-integrity monitoring. A cached 191335eba Qwen3.8 27B Q4_K_M run completed read → edit → test → summary on Ryzen at 16K compiled capacity; it does not establish 256K-input execution or other model-family tool support.

Qwen Code bare mode ignores --core-tools, so these commands explicitly remove unused bare-mode tools. The interactive profile uses 16K context and a 2K output allowance. The overnight profile requires a 256K generated runtime, advertises that capacity to the client, and reserves up to 32K output tokens. It does not add mid-generation resume, automatic publication, commits, or deployment. On later --no-build starts, the server reads the compiled capacity from layout_decode.json. Qwen Code 0.24.6 needs the explicit QWEN_CODE_MAX_OUTPUT_TOKENS environment value in these examples; the older settings profile alone did not constrain its output request in a bounded local probe.

Repeated Qwen and Gemma harness acceptance

Run server/harness_repeatability.py from a client machine that can reach both endpoints through localhost tunnels. It invokes the existing black-box HTTP suites, Qwen Code runner, and Antigravity SDK runner; it does not select templates or execute model arithmetic. Pin each entry to the intended loaded serving and session-library SHA-256 values. Use a Python environment with the server test dependencies for the coordinator, and name a separate Python environment containing google-antigravity==0.1.20 for the Gemma client. The Qwen executable must report version 0.24.6.

{
  "schema": "cke.harness_repeatability.v1",
  "attempts": 3,
  "clients": [
    {
      "name": "qwen-control", "adapter": "qwen_code",
      "endpoint": "http://127.0.0.1:18103/v1",
      "model": "qwen38-27b-q4km-262k",
      "serving_identity": "<64 lowercase hex characters>",
      "session_library_sha256": "<64 lowercase hex characters>",
      "qwen_bin": "qwen", "qwen_version": "0.24.6",
      "task_timeout_seconds": 1800, "prefill_lines": 100
    },
    {
      "name": "gemma-control", "adapter": "antigravity_gemma",
      "endpoint": "http://127.0.0.1:18215/v1",
      "model": "gemma4-e4b-q4km-tools",
      "serving_identity": "<64 lowercase hex characters>",
      "session_library_sha256": "<64 lowercase hex characters>",
      "python": "/path/to/pinned-antigravity-venv/bin/python",
      "case_timeout_seconds": 300, "prefill_lines": 100
    }
  ]
}

# Save the JSON above as /path/to/local-serving-matrix.json and replace both
# placeholder hashes with the intended bundle/session identities.
python3 server/harness_repeatability.py \
  --config /path/to/local-serving-matrix.json \
  --output-dir /path/to/new-matrix-run

# If reporting logic changes, reassess completed retained reports without
# repeating any harness tool or native generation:
python3 server/harness_repeatability.py \
  --reassess-dir /path/to/completed-matrix-run

The output directory must be new. It retains the exact configuration, stdout/stderr, every individual HTTP and harness report, disposable workspaces, and an incrementally written summary. A failed task remains failed even when the loaded identity matched; a passed task does not inherit a missing protocol or recovery result. The matrix also rejects a changed server-instance ID between attempts, even if the same bundle is reloaded. The HTTP suites test multi-turn and output-limit behavior, bounded-input rejection with a successful follow-up, cancellation, disconnect, and timeout recovery. For an uncapped runtime, the context probe tests an oversized output reservation. The 2,048-token Gemma deployment also declares an explicit 32,768-byte rendered-input cap; its probe tests that separate byte limit and a successful follow-up. It does not infer a token count from the input's byte length. Native tokenization failures without a proven capacity cause remain structured server errors, while native busy responses remain HTTP 429. The probe records whether an exact token-count exceedance was proven. It records which boundary was exercised and does not claim that the Qwen probe consumed 262K input. After a runner timeout, the matrix stops submitting work to that endpoint because native completion is not yet proven. The suite records failures without silently retrying edits or commands. It checks outcomes after execution; a fresh directory alone does not sandbox the client. Use a restricted account or container when tool execution needs isolation. Reassessment writes a separate immutable reassessment.json, pins the original summary and each child report hash, and never replays a task. Older reports without child hashes remain integrity-unverified. The small Gemma artifact has only 2,048 tokens of compiled context and uses an explicit 256-token output cap; this matrix is bounded workflow evidence, not a larger-context or unattended-reliability claim. Repeat with a separately built and identified larger-context artifact before advertising practical prompt/tool-schema capacity.

GET /v1/models/{model} reports the loaded CKE context capacity. The server uses the native tokenizer to validate the fully rendered prompt plus requested output before generation and rejects an oversized request with the exact token counts. The selected Qwen profile must not advertise more context than the generated runtime.

Each non-streaming completion and terminal streaming chunk includes cke_performance. It reports prompt/output token counts, native prefill and decode time, complete server request time, and time outside native generation. Client tool execution occurs between requests and must be timed separately.

Profile The Native Token Loop

Use the C CLI directly so Python startup and orchestration do not contaminate the profile:

perf stat -d -- \
  ./build/ck-cli-v8 --lib "$MODEL_SO" --weights "$WEIGHTS" \
  --manifest "$MANIFEST" --require-generated-abi \
  --prompt "Hello" --max-tokens 64 --quiet-output --no-timing

perf record -g --call-graph dwarf -- \
  ./build/ck-cli-v8 --lib "$MODEL_SO" --weights "$WEIGHTS" \
  --manifest "$MANIFEST" --require-generated-abi \
  --prompt "Hello" --max-tokens 64 --quiet-output --no-timing

vtune -collect hotspots -result-dir build/vtune-v8-native -- \
  ./build/ck-cli-v8 --lib "$MODEL_SO" --weights "$WEIGHTS" \
  --manifest "$MANIFEST" --require-generated-abi \
  --prompt "Hello" --max-tokens 64 --quiet-output --no-timing

v8 has its own operator artifact surface. It is not only a decoder dump: text runs can emit an ir_report.html, the v8 hub indexes run directories, and multimodal runs carry encoder → bridge → decoder artifacts that the visualizer can render as a single circuit.

# Generate or refresh a v8 IR visualizer report for one run.
.venv/bin/python version/v8/tools/open_ir_visualizer_v8.py \
  --generate --run "$RUN" --html-only --strict-run-artifacts \
  --output "$RUN/ir_report.html"

# Refresh the v8 run hub.
.venv/bin/python version/v8/tools/open_ir_hub_v8.py \
  --models-root "${CK_CACHE_DIR:-$HOME/.cache/ck-engine-v8/models}" \
  --output "${CK_CACHE_DIR:-$HOME/.cache/ck-engine-v8/models}/ir_hub_v8.html"

# List local v8 runtimes.
./build/ck-cli-v8 --list

• If you omit --image-path, the bridge uses a synthetic image path. That is useful for seam probes, not for validating real-image captioning.
• If the bridge report is missing, check disk space first. The first compile path writes large intermediate artifacts.
• If output starts echoing raw chat markers or thinking-control strings, keep --chat-template auto and do not force a manual template override on this model.
• If captions are incoherent again, verify that the decoder GGUF and mmproj GGUF are the matching Qwen3-VL pair.