Getting Started

Required

# Linux toolchain + isolated Python env
sudo apt install build-essential git make python3 python3-venv python3-pip

# Supported bootstrap path
make v7-init
make v7-doctor

Use the repo-local .venv for all v7 Python commands. If you do not activate it, invoke scripts explicitly with .venv/bin/python ....

Required Python packages (supported v7 workflows): numpy torch safetensors requests tqdm huggingface_hub gguf tokenizers transformers

This is the supported v7 bundle for run / train / parity workflows. Not every package is used on every execution path. If you prefer uv, conda, or manual pip, install that same package set into the interpreter you plan to use, then run make v7-doctor.

Manual Python Environment (pip example)

python3 -m venv .venv
. .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements-v7.txt
make v7-doctor

Optional (for profiling)

# Valgrind (memory profiling)
sudo apt install valgrind

# perf (CPU profiling)
sudo apt install linux-tools-common linux-tools-$(uname -r)

# Enable perf for non-root users
echo 0 | sudo tee /proc/sys/kernel/perf_event_paranoid

# FlameGraph (visualization) - clone to ~/Programs
git clone https://github.com/brendangregg/FlameGraph.git ~/Programs/FlameGraph

Optional (for documentation)

# Doxygen (API docs generation)
sudo apt install doxygen

Build Options

# Build without SIMD (scalar reference implementation)
make AVX_FLAGS=

# Force AVX2
make AVX_FLAGS="-mavx2"

# Debug build with symbols
make CFLAGS="-O0 -g -fPIC -fopenmp -Wall"

Inference Call Sequence

# 1) Init weights
ck_model_init("weights.bump");

# 2) Enable KV cache (inference-only)
ck_model_kv_cache_enable(context_window);

# 3) Prefill prompt (full forward once)
ck_model_embed_tokens(prompt_tokens, n);
ck_model_forward(NULL);

# 4) Decode tokens (one token per step, uses KV cache)
ck_model_decode(next_token, NULL);

Training/backprop uses the full forward+backward path and does not use KV-cache decode.

Step 2: Run Kernel Tests

make test

This runs Python unit tests comparing each kernel (GELU, RMSNorm, Attention, etc.) against PyTorch reference implementations. You should see output like:

Running unittest/test_gelu.py
Testing GELU forward...  OK
Testing GELU backward... OK
...
All Python kernel tests completed.

Step 3: Run End-to-End

make tiny-e2e

This:

1. Generates a config for a tiny model (64 hidden, 2 layers, 256 vocab)
2. Generates random weights and input tokens
3. Runs the codegen to produce C code
4. Compiles and runs forward pass
5. Outputs logits to build/tiny_logits.bin

Step 4: Run Codegen Demo

make ck

This shows the IR (Intermediate Representation) that the engine generates:

=== Forward IR ===
CKIRGraph: layers=2, hidden_size=64 ...
  L0 N0 RMSNORM       outputs=[L0:N0:0]   inputs=[IN]
  L0 N1 LINEAR_QKV    outputs=[L0:N1:0]   inputs=[L0:N0]
  L0 N2 ROPE          outputs=[L0:N2:0]   inputs=[L0:N1]
  L0 N3 ATTENTION     outputs=[L0:N3:0]   inputs=[L0:N2]
  ...

Expected Output

$ make tiny-parity
...
Step 0: C loss=10.2345, PyTorch loss=10.2345, diff=1.2e-06
Step 1: C loss=9.8765, PyTorch loss=9.8765, diff=1.1e-06
...
Max weight diff: 1.2e-05
PASS: Training parity verified!

Memory Check

make profile-memory

With our bump allocator, you should see:

LEAK SUMMARY:
   definitely lost: 0 bytes in 0 blocks  <-- Clean!

CPU Profile + Flamegraph

# Enable perf first (one-time)
echo 0 | sudo tee /proc/sys/kernel/perf_event_paranoid

# Run profiler
make profile-cpu

# Generate flamegraph
make flamegraph

# View in browser
firefox build/flamegraph.svg