Local harness quickstart
Run a generated CKE model behind one OpenAI-compatible endpoint, then point a client at that
endpoint. The harness owns files, tools and permissions; CKE owns rendering, tokenization, the
native model session and response delivery. A model appearing in /v1/models proves
that it loaded, not that it can complete an agent task. The table below names dated, tested
artifact boundaries. Consult the
serving coverage inventory for circuit declarations and the
serving runbook for detailed acceptance commands.
The Qwen report's artifact_certification_status: PASS means the expected
loaded serving and native-library identities remained bound to the task; it is
not a full-model numerical-parity certificate. The dated reports are retained
results from their listed source revisions, not continuously current endpoint status.
Model and harness map
| CKE model artifact | Client to try | Current evidence | Tool-task status |
|---|---|---|---|
| Qwen3.8 27B Q4_K_M | Qwen Code | An identified 32K bundle built from merged main completed repeated Qwen Code 0.25.1 read → edit → test tasks on Ryzen on 3 November 2026. An artifact-bound cache-on/off pair processed 58,872 input tokens across five responses in each mode; exact-prefix reuse saved prefill work. HTTP cancellation, eviction, capacity rejection and recovery checks passed. See #1015 and the serving runbook. | Bounded repeated task PASS for that 32K artifact. The 32K capacity was not consumed by one turn. Its ordinary HTTP path remains single-flight; recurrent batching and other Qwen3.8 builds require their own certification. |
| Qwen3.8 Flash Next Q4_K_M | No coding harness is certified for the pinned bundle. | The retained 8K-capacity generated runtime matched 64 full-vocabulary logit comparisons across a 40-token batched prompt. The Flash circuit and provider contract suite was already in the default nightly; it now also runs alongside dense Qwen in the demo-readiness profile. See the Flash Next runbook. | Generated arithmetic evidence; no harness certification. The circuit now declares the existing publisher Qwen XML tool profile, with synthetic HTTP round-trip regressions. The older serving bundle still needs rebuilding and resolution; its tool protocol is not enabled retroactively. Long-context state continuity, recurrent prefix restore, repeated harness tasks and current-endpoint recovery remain uncertified. |
| Gemma4 E4B, explicit tools variant | Antigravity SDK | Four bounded local SDK tasks passed on the historical 2K bundle and on one identified 32K-capacity bundle on 26 October 2026. The fresh run took 154.8 seconds; see version/v8/reports/gemma4_antigravity_32k_20261026.json for per-case evidence. |
Bounded PASS for short tasks. The 32K build capacity is not a 32K-input result. The chat-only variant rejects tools; overnight reliability remains untested. |
| Muse Glimmer | Direct OpenAI-compatible HTTP is tested; Muse Code and Hermes Agent remain separate client gates. | On an identified 32K-capacity bundle, a direct HTTP file-read call and correlated result produced the exact final value. The continuation used 518 input and 328 output tokens; a 256-token cap failed mid-reasoning. See version/v8/reports/serving_family_http_sweep_20261026.json. |
Bounded HTTP tool PASS. The pinned Hermes Agent refused this 32K bundle because it requires at least 64K context. A captured Muse Code 1.4.8 file-task request rendered to 36,748 tokens, also exceeding the 32K bundle. A Muse Code task and long-context generation remain untested. |
| Instella-MoE Think BF16 | OpenHands is a proposed harness. | An identified 4K bundle completed short chat and streaming requests. A 128-token answer was incomplete; the same prompt completed at 256. This older generated library lacks ck_model_count_text_tokens, so an oversized prompt returned ambiguous HTTP 500 native_tokenization_failed; a later short request completed. Regenerate and retest that admission gate. Separate opt-in numerical work still reports full-logit divergence. |
Chat only, partial acceptance. This publisher variant declares no tool-output protocol; CKE returns HTTP 501 for tools. OpenHands coding tasks cannot pass this pairing yet. |
| Nemotron Nano 9B v2 Q4_K_M | LangChain Deep Agents | Three restricted LangGraph file-read attempts passed on one identified 2K bundle. A separate 20K-capacity P3 bundle completed a real Deep Agents read → edit → validator → exact-answer task and a failed-tool recovery task on 27 October 2026; see version/v8/reports/nemotron_deepagents_edit_test_20k_20261027.json. Its five turns used 3,242–3,519 input tokens and took 1,184.5 seconds. |
Two bounded Deep Agents tasks PASS. The shared server validates the client's nullable anyOf tool fields. A missing-file result was followed by a valid read and accurate answer on the same instance. Repeated reliability, 20K inputs, numerical parity, cancellation and subagents remain untested. |
| Cohere Command R7B Q4_K_M | LangChain ChatOpenAI for plain chat. | On an identified 2K bundle, plain chat and streaming completed; a counted 5,009-token prompt returned HTTP 400 before generation, and a later request completed on the same instance. Earlier 5,332 exact raw prompt IDs and first-logit differences remain separate numerical evidence. | Bounded chat PASS; tools unsupported. The selected imported template rejects tool calls with HTTP 501. Command A+ and Cohere North are different artifacts with separate gates. |
A client can connect to a CKE endpoint without the model being able to follow its tools. Do not copy a PASS between quantizations, context builds, publisher variants or model families.
1. Start an identified CKE endpoint
Use a complete resolved serving bundle: BUMP weights, compiled model/engine/tokenizer, manifest, circuit-linked profile and imported publisher assets. Build and resolve it with the v8 runbook; a GGUF filename or model name alone is not a serving contract. For the documented Qwen3.8 27B Q4_K_M artifact, this command downloads the checkpoint if needed, compiles a 16K runtime, and starts the server on the model host:
CK_NUM_THREADS=16 OMP_NUM_THREADS=1 \ version/v8/scripts/cks-v8-run serve \ hf://ggml-org/Qwen3.8-27B-GGUF/Qwen3.8-27B-Q4_K_M.gguf \ --run /path/to/qwen38-agent-runtime \ --context-len 16384 --force-compile \ --model-name qwen38-27b-q4km
Use the exact checkpoint, serving profile and generated bundle for another family; the coverage inventory identifies circuit linkage but does not itself certify an artifact. Once a bundle exists, this example starts it without rebuilding:
BUNDLE=/path/to/resolved-generated-bundle CKE_MODEL=the-model-id-you-chose CK_NUM_THREADS=16 OMP_NUM_THREADS=1 \ python3 version/v8/scripts/ck_serve_v8.py "$BUNDLE" \ --run "$BUNDLE" --no-build --model-name "$CKE_MODEL" \ --host 127.0.0.1 --port 8080
The server must finish startup verification. On the model host, inspect its advertised ID, compiled context and loaded hashes:
curl -fsS http://127.0.0.1:8080/v1/models curl -fsS http://127.0.0.1:8080/v1/cke/loaded-identity
If the client is on another machine such as P3 and the server is on Ryzen, keep the server bound to loopback and open an SSH tunnel from the client in a separate terminal:
ssh -N -L 8080:127.0.0.1:8080 <model-host> # On the client, use http://127.0.0.1:8080/v1 as the base URL.
For a containerized client, its 127.0.0.1 is the container, so give the container a
route to the forwarded host and set its base URL accordingly. Confirm /v1/models from
inside that client environment. Many clients require an API-key field even when this loopback
CKE endpoint does not authenticate; a placeholder such as cke-local satisfies that
client field. Keep the tunnel and server access restricted to trusted users.
2. Prove plain HTTP before opening a harness
curl -fsS http://127.0.0.1:8080/v1/responses \
-H 'Content-Type: application/json' \
-d '{"model":"REPLACE_WITH_ID_FROM_MODELS","input":"Say hello.","max_output_tokens":64}'
Check the terminal status and stop reason. An output-limit incomplete response is
not a completed answer. Record GET /v1/cke/loaded-identity before and after a task;
its serving identity, loaded session hash and server-instance ID bind the evidence to the process.
For cancellation and disconnect testing, use the black-box
server/http_lifecycle_acceptance.py runner with both expected hashes.
3. Qwen Code with the certified Qwen3.8 bundle
The CKE-tested Qwen Code profile uses a Responses connection and names the loaded model explicitly. Install the client using the official Qwen Code instructions; this example pins 0.24.6, a version exercised in CKE's bounded probes. On the client machine, from the CKE repository:
npm install -g @qwen-code/qwen-code@0.24.6 CKE_ROOT="$PWD" OPENAI_API_KEY=cke-local \ OPENAI_BASE_URL=http://127.0.0.1:8080/v1 \ OPENAI_MODEL=qwen38-27b-q4km \ QWEN_CODE_MAX_OUTPUT_TOKENS=2048 \ QWEN_CODE_SYSTEM_SETTINGS_PATH="$CKE_ROOT/server/qwen-code/interactive.settings.json" \ qwen --auth-type openai-responses --model qwen38-27b-q4km
Replace the model ID if /v1/models reports a different one. The interactive profile
advertises 16K context; its overnight counterpart requires a separately generated 256K runtime.
For the bounded disposable read/edit/test gate, run
server/qwen_harness_acceptance.py with the expected serving identity and session-library
hash as shown in the Qwen runbook.
Current Qwen Code releases also document modelProviders.openai with a per-model
wireApi: "responses"; use the
provider reference
when changing client versions.
4. Gemma4 with Antigravity SDK
Install the SDK in an isolated client environment. Use
LocalOpenAIAgentConfig to connect
to CKE; LiteRTAgentConfig runs a different model runtime and is not this path.
Select the bundle's explicit tools variant for tool tasks and its exact advertised model ID.
python3 -m venv /tmp/cke-antigravity-venv
/tmp/cke-antigravity-venv/bin/python -m pip install google-antigravity==0.1.20
# In a small client script:
import asyncio
from google.antigravity import Agent, LocalOpenAIAgentConfig
async def main():
config = LocalOpenAIAgentConfig(
model="REPLACE_WITH_GEMMA_TOOLS_MODEL_ID",
base_url="http://127.0.0.1:8080/v1",
).lightweight()
async with Agent(config) as agent:
reply = await agent.chat("Say hello in one sentence.")
async for chunk in reply:
print(chunk, end="", flush=True)
asyncio.run(main())
Start with a disposable workspace. For the retained read, two-file, edit/validate and
failed-tool recovery cases, use server/gemma_antigravity_acceptance.py with both
expected hashes and a per-case deadline. The published 2,048-token result does not establish
larger prompt capacity.
5. Experimental client connections
Instella and OpenHands
OpenHands documents local OpenAI-compatible servers through its
local-model setup.
In its Advanced LLM settings, use openai/REPLACE_WITH_ID_FROM_MODELS as Custom Model,
the CKE base URL visible to the OpenHands container, and cke-local as a placeholder
key. This is a connection diagnostic only: the pinned Instella Think profile rejects tool
requests with 501. Its current numerical and ordinary-chat limits are listed in the table;
do not start an unattended OpenHands coding task with this artifact.
Nemotron and LangChain Deep Agents
Install deepagents and langchain-openai using the
Deep Agents quickstart.
First verify a plain ChatOpenAI request against CKE, using the exact model ID:
from langchain_openai import ChatOpenAI
model = ChatOpenAI(
model="REPLACE_WITH_NEMOTRON_MODEL_ID",
base_url="http://127.0.0.1:8080/v1",
api_key="cke-local",
temperature=0,
)
print(model.invoke("Say hello.").content)
A restricted LangGraph probe has now completed three consecutive cat note.txt
tool calls, correlated results and correct final answers on one verified 2K bundle. Run
server/langgraph_read_acceptance.py with --base-url, --model,
--run-dir, --expected-serving-identity and
--expected-session-sha256; its optional client dependencies are
langgraph and langchain-openai. The retained result is
version/v8/reports/nemotron_langgraph_read_20261026.json. The initial
/think trial exhausted a 256-token output budget before a tool call; the passing
matrix used a frozen /no_think prompt. A later, separate 20K-capacity
P3 bundle passed one restricted LangChain Deep Agents read/edit/validator task using
deepagents 0.7.32 and langchain-openai 1.7.2. Its first attempt
failed before generation because CKE rejected nullable anyOf tool fields;
the patched server admitted the unchanged captured client schemas. The completed task
used 3,242–3,519 actual input tokens, took 1,184.5 seconds, and kept the same loaded
identity; see version/v8/reports/nemotron_deepagents_edit_test_20k_20261027.json.
A separate failed-tool case then returned a file-not-found error, read the valid file,
and reported both outcomes correctly in 702.4 seconds, with 3,044–3,167 input tokens.
These two bounded tasks do not establish longer-context or repeatability certification.
Test cancellation and subagents separately. NVIDIA Dynamo is serving infrastructure,
not the LangChain harness.
A later pinned 20K run on Ryzen completed two successive restricted Deep Agents
read → edit → validator tasks on one loaded instance, with a trusted passing
assertion each time. A separate injected transient read failure was returned to
the agent, retried against the same file, and followed by a successful edit and
validator. The optional scripts/local_nemotron_nightly.py lane runs
fresh → injected failure/recovery → fresh tasks against one isolated loopback
service. Its private JSON configuration pins the bundle libraries and weights,
service source commit, session library, model identity, client Python, context,
threads, output allowance, and a cgroup memory cap. Invoke it with
--config /path/to/pinned.json --state-dir /path/to/state; a
PAUSE file in the state directory prevents startup. The lane writes
the current attempt to latest.json before any work, retains every
child report, and stops only its own service. Missing assets or insufficient
memory are NOT_TESTED, while wrong identities and failed tasks are
FAIL. These checks certify a bounded workflow on the pinned artifact;
they do not establish native recurrent batching or general agent reliability.
Cohere text and LangChain
The same ChatOpenAI connection above can test plain chat with a deployed Cohere
Command R7B model ID. Keep tools disabled for its selected imported chat-only template. Cohere
North and Command A+ need their own exact artifact, template, numerical and task acceptance;
Cohere Transcribe uses a transcription interface instead of a coding-agent chat harness.
Muse Glimmer and Muse Code
Meta documents Muse Code as its terminal
agent for the hosted Muse service. The identified 32K CKE bundle completed a direct HTTP
muse.read_file call, correlated tool-result continuation and exact final answer,
as recorded in version/v8/reports/serving_family_http_sweep_20261026.json.
This was not a Muse Code CLI run. The pinned Hermes Agent client rejected the bundle before
inference because it requires a 64K context window. Do not misstate the compiled 32K window
to bypass that check; build and independently test a larger bundle if using that client.
The HTTP trial used at most 518 input tokens and needed 328 output tokens for continuation.
A separate Muse Code 1.4.8 file-task request rendered to 168,861 bytes and 36,748
tokenizer tokens with its bundled instructions and 32 namespace tools. The 32K runtime
cannot admit that request. A 64K-capacity bundle was built for an isolated client trial;
capacity alone does not certify its result. Keep a bounded output allowance and verify
loaded identity, actual tool execution, terminal status and follow-up recovery before
calling the CLI pairing ready. Neither trial certifies an edit-and-test workflow.
What counts as ready for work?
For each exact bundle, retain the loaded identity and test these gates independently: rendered bytes and token IDs; normal chat and streaming; declared tool call and correlated result; a real read → edit → test → summary task; cancellation/disconnect followed by another request on the same server; and numerical/state evidence under the provider contract. A missing asset-backed run is NOT_TESTED, not PASS. Queueing and two-slot batched decode are separate from whether a harness task succeeds. See the serving architecture page for the execution modes.
Read-only node discovery over ZeroMQ
The optional C node adapter is separate from inference. Any application can consume
cke.control.v1; CKE has no dependency on a studio, registry or gateway.
The consumer owns groups, pools, authorization and freshness display. This first version
supports node.describe, workers.list,
worker.capabilities and worker.status.
sudo apt-get install libzmq3-dev libjson-c-dev libcurl4-openssl-dev python3-zmq make -C server/control server/control/node-adapter p3 8120 whisper=8110 kokoro=8111 python3 server/control/client.py --node-id p3 --operation workers.list python3 server/control/client.py --node-id p3 --operation worker.status --worker-id whisper
Registrations are explicit local HTTP ports, not network discovery. The adapter binds
only to 127.0.0.1. It observes existing /health routes; it cannot load a model,
execute a shell command, submit training or cancel inference. Do not expose it remotely:
authenticated approved peers and resource admission must be implemented first. Loopback
access assumes trusted local users; this version is not a hostile multi-tenant service.
Each request is a single JSON frame, at most 8 KiB. There are at most 32 registered workers, per-peer ZeroMQ queue high-water marks of 16, a 500 ms worker probe deadline, and an 8 KiB HTTP response limit. Excess-size frames are disconnected; multipart requests reset the transport. Clients must recreate their REQ socket after timeout. These limits do not establish a global bound on arbitrarily many local peer connections.
Adapter health and worker health are separate. An offline worker is a successful
observation with observation_status=unavailable and unknown readiness.
Capabilities and worker boot identity remain null until the worker exports trustworthy
information. A local generation counter is not a boot identity. No raw paths, exception
text, tokens or request identifiers are forwarded. Adapter revision identifies the build,
not the worker's loaded artifacts. Registration and reachability do not certify numerical
correctness or model support.
The machine-readable contract, example fixtures, standalone client and action plan
are in server/control/. Run real-socket tests with
CKE_NODE_ADAPTER=$PWD/server/control/node-adapter python3 -m unittest discover -s server/control.
A dedicated PR workflow builds the native adapter and executes these tests. External
registry/UI integration and authenticated remote deployment remain separate acceptance gates.
Simulated-consumer acceptance
The independent probe runs node.describe → workers.list → worker.status +
worker.capabilities for each worker, then repeats node.describe as a final
identity check. Every response must match schema, request ID and node ID. The adapter boot
ID must remain constant across the entire pass. A restart, changed worker list count,
malformed response, wrong field type or duplicate worker rejects the pass.
python3 server/control/consumer_probe.py --node-id p3 --port 8120 \ --state /tmp/cke-p3-observations.json
The state file is single-writer and local to the consumer. Each run atomically publishes
INCOMPLETE before discovery and then PASS or FAIL. last_success retains the last
complete validated pass; last_attempt is always the current attempt, and
last_error retains the most recent failure even after recovery. An old success
is historical evidence, never a current health verdict. A process crash can leave INCOMPLETE;
consumers must apply their own freshness deadline. Filesystem write failure is a command
failure and cannot guarantee publication.
A PASS means the adapter passes the simulated-consumer contract. It does not mean every worker is ready: a correctly reported unavailable worker is a valid observation. Unknown capabilities and worker boot identities remain null. The generation counter must not be substituted for a worker boot ID. Known worker boot identity changing between status and capability queries rejects that pass; unknown IDs cannot prove worker continuity.
The probe uses fresh REQ sockets after timeouts, checks response size and frame count,
and stores only validated fields. It does not retain raw malformed responses or exception
text. Synthetic sanitized request/response templates and fault mutations with expected
outcomes live in server/control/fixtures/consumer-contract.json. External clients
can replay these without CKE model weights. Real-socket tests also stop/restart the actual
adapter and test malformed peer replies. The existing node-control PR workflow runs them.
This is a contract acceptance tool, not a central registry or production poller. Antsand integration, UI freshness, cross-group access control and remote authentication remain separate acceptance tests. No inference or workload commands are added.