Training Method

Methodical DSL Training Methods

A spec is not just a dataset or a model run. It is a contract between an asset library, a scene DSL, and a compiler. The model should learn structure against a stable contract, not absorb random changes in vocabulary, content, and rendering all at once.

Asset library Placeholder content Scene DSL Compiler gate Tokenizer gate Probe report

Why This Method Exists

This curriculum is not just local preference. It follows the same broad logic seen in scaling-law work, compute-optimal training research, and capability-predictability discussions: use smaller and cheaper experiments to validate the contract, the data, the tokenizer surface, and the evaluation loop before spending serious compute on larger runs.

Three Separate Contracts

Asset Memory layout map — what the compiler owns

Compiler IR lowering pipeline — deterministic stages

DSL Decision tree — gate failure diagnostics

These are real assets from the project library — the same asset families the training pipeline learns to plan and compose.

What We Are Actually Training

In this line, the model is not being trained to be a free-form SVG artist. It is being trained to choose structured visual intent against a stable compiler contract.

poster_stack layout family — the model chooses composition, theme, tone, and content slots. The compiler handles gradients, geometry, text wrapping, and final SVG rendering.

Worked Example

A concrete keyed-scene example should look like this:

Request prompt

[task:svg]
[layout:poster_stack]
[topic:memory_reality]
[theme:infra_dark]
[tone:green]
[density:compact]
[OUT]

Scene DSL emitted by the model

[scene]
[canvas:tall]
[layout:poster_stack]
[theme:infra_dark]
[tone:green]
[frame:card]
[density:compact]
[inset:md]
[gap:sm]
[hero:center]
[columns:1]
[emphasis:top]
[rail:accent]
[background:rings]
[connector:line]
[topic:memory_reality]
[header_band:@header_band.0.kicker|@header_band.0.headline|@header_band.0.subtitle]
[section_card:@section_card.0.title|@section_card.0.value|@section_card.0.caption|variant=hero|accent=amber]
[compare_bar:@compare_bar.0.label|@compare_bar.0.value|@compare_bar.0.caption|accent=red]
[table_row:@table_row.0.column_1|@table_row.0.column_2|@table_row.0.column_3|state=highlight|accent=amber]
[/scene]

content.json bound by the compiler

{
  "header_band": [
    {
      "kicker": "First Principle",
      "headline": "LLM Memory Reality",
      "subtitle": "The math marketing will not show you"
    }
  ],
  "section_card": [
    {
      "title": "Memory Capacity",
      "value": "25x more memory capacity",
      "caption": "Capacity sets the real context boundary"
    }
  ],
  "compare_bar": [
    {
      "label": "GPU VRAM",
      "value": "80 GB",
      "caption": "single device"
    }
  ],
  "table_row": [
    {
      "column_1": "128K",
      "column_2": "3x GPUs",
      "column_3": "Fits"
    }
  ]
}

The training target here is the scene decision, not the literal prose. The compiler can now render the same scene with different content payloads without retraining the model.

Topology Refs vs Content Refs

Some richer layout families need more than content slots. Trees, maps, and graph-style layouts also need stable structural handles so the compiler knows what connects to what.

[decision_node]
[node_id:start]
[title_ref:nodes.start.title]
[/decision_node]

[decision_edge]
[from_ref:start]
[to_ref:l2]
[branch_label_ref:edges.start_l2]
[/decision_edge]

This dual-reference pattern is not a bug. It is the right compiler boundary for tree, map, and topology layouts.

Why Explicit Tokens Instead Of Generic BPE Right Now?

Tokenizer architecture — the CK-Engine tokenizer uses explicit reserved tokens for the DSL surface. This makes every probe miss interpretable as a scene decision error, not subword spelling drift.

The short answer is: because this output language is small, formal, and brittle, and the current training line is optimizing for control and interpretability before token-efficiency.

A useful rule is:

Early line:
prefer explicit structural tokens so the contract is obvious.

Later line:
shrink or relax the reserved surface only after the model, compiler, and evals agree on the language.

One mistake to avoid is treating a whole component row as the permanent atomic token surface.

The first shape can be acceptable as a transition step. It should not be treated as the final production tokenizer boundary.

Bad tokenizer boundary

[compare_bar:@compare_bar.0.label|@compare_bar.0.value|@compare_bar.0.caption|accent=amber]

Better boundary

[compare_bar]
[field:label]
[@compare_bar.0.label]
[field:value]
[@compare_bar.0.value]
[field:caption]
[@compare_bar.0.caption]
[accent:amber]
[/compare_bar]

The rule is simple: reserve structure, not payload. If a token contains too much scene-specific data or too many keyed refs, it is probably the wrong token boundary.

Spec Lifecycle

Evolution timeline — each spec passes through the same lifecycle stages below. The milestones above show how the v6.6 line matured through this exact process.

Operational Training Process

The practical mistake is to let a rung answer too many questions at once. The better process is narrower: freeze measurement, name one failure class, teach family structure, and let validation close the last deterministic syntax gaps.

A useful default wording for the rung brief is: "Fix one named failure class while preserving the last good baseline on the frozen probe." That sentence is narrow enough to govern the materializer, the probe, and the post-run decision.

Context Budget Is A Gate

Do not guess the next context length. Measure the gold scenes with the current tokenizer family before training.

The important lesson from the keyed-scene line was that a layout can fail because of decode or context budget even when the model already learned the structure correctly. Budget diagnostics should therefore be part of the probe contract, not an afterthought.

How To Build Frontier-Style Intuition

No serious engineer would laugh at the core idea here. The separation of structure, content, compiler, and eval is exactly the kind of thinking that makes systems reliable. What strong teams would challenge is not the direction, but the discipline of the method.

One Primary Axis Per Run

The model can only teach intuition if each run is interpretable. Every run should declare one primary axis and list everything held constant.

A run is only methodical when it can be summarized as one sentence: “This run changed X, held Y constant, and answered Z.”

Restart-Safe Agent Handoff

Yes, this page is the right umbrella documentation to keep around for restarts. It now has a companion handoff file so a future agent does not need to reconstruct the current plan from scattered reports or stale run logs.

Suggested restart prompt

Read these first and treat them as the live source of truth:

1. docs/site/_pages/spec-training-method.html
2. docs/site/_pages/agent-handoff-template.md
3. version/v7/reports/SPEC[X]_EXECUTION_CONTRACT_YYYY-MM-DD.md
4. version/v7/reports/spec_family_autopilot_policy.json

Current intent:
- keep the last good spec[x] rung[y] as the training-method baseline, not the tokenizer ceiling
- build the successor DSL/compiler/tokenizer path explicitly
- keep payload/content external to the model DSL
- add one capability at a time
- make the next family[z] the active family-construction line
- do not launch training until the compiler, tokenizer corpus, launcher, and rung policy checklist are complete

Then continue from the current execution contract checklist and update the repo artifacts before starting any background training or autopilot.

Recommended Asset-to-DSL Workflow for SVG

Full pipeline flow — from input artifacts through IR stages to compiled output. The compiler validation phase in the workflow below ensures the DSL round-trips through this pipeline deterministically.

HTML Report Contract After Every Spec or Run

Every run should leave behind a readable HTML report that answers the same questions in the same order. The point is not decoration. The point is to make the next decision obvious.

Run summary template

Spec:
Run:
Primary axis changed:
Held constant:
Baseline:

Hypothesis:

Data delta:
DSL delta:
Compiler delta:
Tokenizer delta:
Budget delta:
Capacity delta:

Preflight result:
Compiler round-trip result:
Probe result:
Failure clusters:

Decision:
Next run:

What The Experiments Proved

Insight Start small — bridge the chasm with cheap specs first

Method Find the real bottleneck before scaling

Practice Fail fast, learn, freeze winners, redesign upfront

These conclusions did not come from theory alone. They came from repeated runs that failed, partially worked, regressed, recovered, and exposed the actual pressure points in the system. Across the full run history, the stable wins came from better contracts and better full curricula. Narrow raw repair churn became less reliable once a line was already mostly learned.

How We Reached The Current Design

table_matrix layout family — a structured byte-level reference. The experiments below showed that this level of visual detail requires a compiler, not a model that emits raw SVG.

How To Decide What Is Next

Diagnostic Gate failure decision tree — match symptom to action

Timeline How specs mature through repeated gate passes

Diagnostic Matrix

After a run finishes, use the probe report, loss curve, and per-layout breakdown to decide what to do next. Do not guess — match the symptom pattern to the action.

Critical — fix before retraining Warning — likely cause of weak results Info — plumbing or telemetry issue

Reading the Probe Report

The probe report is the real decision surface, not the loss curve. A healthy loss curve with bad probe results means the representation or curriculum is wrong. A noisy loss curve with good probe results means the model learned despite training-signal noise.

Reading the Loss Curve

Decision Actions

Promotion Rules

• Promote a run only if it improves the primary target metric without regressing the already-solved slices.
• Reject a run if it changes too many axes to interpret, even if the final number looks better.
• When a frozen raw winner exists, default to decode/repair and block more same-surface raw repair rungs until the next training idea is a true new branch.
• Branch a new spec only when the current spec ceiling is real and well measured.
• Make compiler fidelity a hard gate: use a gold pack, not just a couple of hand-picked examples.
• Scale model size only after the DSL and compiler contracts are stable.
• Keep run artifacts in ~/.cache/ck-engine-v7/models/train/; keep curated project docs in the repo.
• Measure output token budget against context window before training. If the longest gold mapping exceeds 80% of context length, compress the DSL or raise context first.

Spec Versus Run

The project uses spec[x] and r[y] for a reason. A spec is a new training contract. A run revision is a controlled iteration inside the same contract.

Start a new spec when the current ceiling is real and measured. Stay inside the same spec when the failure is narrow and the representation still matches the intended product boundary.

Working Definitions

Asset Scaling Strategy

Do not try to train on a large asset library immediately. Prove each capability level first, then widen. The pattern is: memorize → generalize → expand.

Each phase should be a separate spec or run. Do not skip phases — a failure at phase 2 means phase 3 data will be wasted compute. The gold asset count is a guide, not a rule. What matters is that each phase answers its specific question before the next one starts.

How To Build Training Intuition Without Frontier Compute

The useful lesson from frontier work is not that every internal circuit is understood. The useful lesson is that model behavior can still be shaped in predictable directions through disciplined control of data, interfaces, budgets, and evaluation.

In other words, practical intuition comes from asking: what changed, why did behavior change, and which layer actually caused it? This is why the spec/run method matters. It turns training into a ledger of answers instead of a collection of lucky outcomes.

What Frontier Labs Usually Know

The claim that "no one knows how LLMs work" is too broad to be useful. A better version is this: full mechanistic theory is still incomplete, but empirical control is strong, and product/system control is often stronger still.

In practice, serious teams may not be able to explain every internal circuit, but they can still learn, with real discipline, that a particular data mixture, architecture, scale, objective, post-training recipe, and evaluation set tends to produce particular kinds of behavior. That is not total understanding, but it is real engineering knowledge.

Why Data Curation Still Matters

Scaling does not erase the training distribution. The model still compresses what it sees. Bad data creates bad priors, noisy mixtures create unstable behavior, narrow data creates narrow generalization, and well-shaped data creates cleaner abstractions.

This is why data curation remains central even without a complete theory of generalization. The practical loop is still: define the contract, shape the data to teach that contract, measure the right behavior, and repair the actual failure layer.

The Failure-To-Repair Loop

In practice, much of the progress comes from a simple discipline: observe the weakness, classify the weakness, teach that weakness directly, and rerun from the best relevant checkpoint when the spec has not changed.

This is not the same as "keep adding more data." Before adding rows, decide whether the failure is mainly in the data, DSL, compiler, tokenizer, token budget, decode hygiene, or capacity. Then fix the right layer.

• If the model misses [/scene], add closure and continuation rows and tighten stop hygiene.
• If the model confuses two families, add contrast rows and family-local anchors.
• If the DSL is too verbose or ambiguous, start a new spec instead of piling on more data.
• If the compiler cannot express the target asset, fix the compiler before retraining.
• If a seeded checkpoint suddenly probes as all-zero before any new training, stop and verify seed/probe integrity before blaming the curriculum.
• If "negative" cleanup rows are added under ordinary cross-entropy supervision, remember they are still positive targets for the model; treat contamination rows as hazardous, not automatically corrective.

A practical rule is: same spec -> usually continue or rerun inside the same family; new spec -> usually start a new line. This is how failures become supervision instead of wasted compute.

Run Failure Matrix

The most useful next step is not "train more." It is to name the failure class correctly. The table below is the default matrix for interpreting a run before deciding whether to repair the data, change the DSL, fix the compiler, or scale capacity.

Minimum Run Scoreboard

Every run should publish the same small scoreboard. This keeps comparisons honest and makes failure classes visible without reading raw prompt dumps first.

A simple interpretation rule helps keep the read honest: low exact with high renderable usually means semantic or ordering drift; low renderable with low truncation usually means grammar corruption; high truncation is a budget problem; high exact with low materialized exact is a compiler or probe bug.

Repair Ladder Lessons

A recurring mistake in small-to-medium DSL runs is to treat every bad output as a direct template for the next repair row. That is too naive. The model does not understand that a row was meant as a warning; under ordinary supervised training it only sees another target distribution to imitate.

• Do not treat synthetic corruption rows as automatically "negative data." Under cross-entropy, they are still positive supervision unless a different objective is used.
• Protect full-scene replay anchors from the last good rung. Local continuation repairs should stay subordinate to full canonical scenes.
• Separate three gates before training: probe-path integrity, seed-copy integrity, and then curriculum quality. A broken probe can look exactly like a broken rung.
• If one small architecture already spans a wide quality range across runs, the next bottleneck is usually recipe shape, not model size.
• Use tiny seeded canaries after a failure. If a 0.1x or 0.2x corrective run still destabilizes the same contract, pause rung-chasing and rethink the materializer itself.

In other words: do not use bigger models to paper over a broken supervision surface. Scale capacity when the contract is stable and the same clean failure persists across multiple disciplined canaries. Until then, the higher-signal move is to improve the curriculum, replay anchors, and measurement path.

What To Copy From Frontier Practice

• Use proxy runs before expensive runs.
• Design evals aggressively instead of trusting loss alone.
• Run controlled ablations instead of relying on folklore.
• Keep explicit failure taxonomies.
• Control data mixtures carefully.
• Separate base learning, post-training, and system scaffolding.
• Do not expect one run to answer five different questions at once.

Ethical Scaling Path

Going from something simple to something bigger should not mean removing constraints faster than understanding improves. The safer path is to widen capability in stages while keeping the boundaries of responsibility clear.

Ethical scaling is not only about policy. It is also about research honesty: know what the model is really doing, know what the compiler is doing, know what the content system is doing, and report those boundaries clearly.

Pre-Training Checklist

Before launching any training run in a new spec, verify all of these gates. If any fails, fix it before spending compute.

What A Solved Structured-DSL Run Actually Proves

A solved spec[x] rung[y] does not mean the model learned open-ended infographic design. It means something narrower and more useful: the model can reliably map a bounded prompt contract to a compiler-facing scene DSL, and that DSL can then be combined with external content.json to produce exact SVG.

• The model is responsible for the scene program: layout family, scene components, refs, and top-level style fields.
• The compiler is responsible for deterministic layout and SVG assembly.
• The external content payload is responsible for visible copy, data values, and topic-specific facts.

Keep those responsibilities separate. When adding new DSL families, do not blend asset identity into the output grammar unless the structure truly requires it. Case ids, topic-specific facts, and payload-specific copy should stay in prompt text, routing metadata, and external content.json wherever possible. The output DSL should stay as generic as the renderer contract allows.

This boundary matters. A strong spec[x] rung[y] result proves the training method can stabilize structured program generation. It does not yet prove arbitrary topic generalization, arbitrary tree depth, free-written infographic copy, or unconstrained scene planning.

Prompt-Surface And Decode-Boundary Discipline

Hidden eval should cover at least two different generalization surfaces:

• Prompt-surface robustness: paraphrases, wording shifts, and bridge prompts that preserve the same underlying task.
• Semantic breadth: new topics, new assets, or new content payloads that test whether the contract scales beyond the first few gold mappings.

Solving prompt paraphrases is not the same as solving new semantics. Treat those as different milestones and report them separately.

Also keep decode-boundary hygiene separate from model quality. In raw CLI inference, a scene DSL model may generate a correct [scene] ... [/scene] block and then continue into the next training-style prompt if no structural stop marker is supplied. That is a decode configuration issue, not automatically a training failure. Measure the first valid scene block, then fix stop hygiene at the inference boundary.

Next Bridge After A Solved Explicit-Layout Line

The next step after a solved explicit-layout line is not open-ended prose. It is a bounded intent bridge where the prompt still names the topic and goal, but stops prescribing the scene directly.

The short rule is:

• keep content.json external
• keep the compiler deterministic
• let the model infer layout, theme, tone, density, and related scene-planning fields from topic + goal + audience
• measure planning quality separately from syntax and renderability

Example intent-bridge prompt:

[task:svg] [topic:topic_id] [goal:goal_id] [audience:audience_id] [OUT]

This keeps the task bounded. The model is not being asked to learn open-domain language or free-written infographic copy. It is being asked to choose a good scene plan under weaker prompt control.

Progress Goal After The Last Good Baseline

The immediate goal after the last good baseline is not “teach the model English.” It is narrower and more useful: preserve compiler-backed precision while adding more SVG families and better within-family generalization.

• Keep the renderer deterministic and visually strong.
• Keep semantic payloads external until the family contract is stable.
• Add new families one at a time so failure causes stay legible.
• Judge progress by exact/materialized SVG quality and held-out family transfer, not by conversational fluency.

Related Pages