kernels Directory Reference

Directories
	fused

Files
	add_kernels_bf16.c
	Element-wise addition kernels for BF16 tensors.
	attention_decode_fused.c
	Fused attention decode kernel (legacy v6/v6.5)
	attention_flash_true.c
	Flash-style attention (online softmax, causal, streaming)
	attention_kernels.c
	Attention score/softmax/output kernels with SIMD (SSE/AVX/AVX512)
	attention_kernels_sliding.c
	Sliding-window flash attention kernels split from attention_kernels.c.
	attention_oracle_ggml.c
	Strict ggml-backed full-attention oracles for parity debugging.
	attention_oracle_ggml.h
	audio_kernels.c
	Numerically explicit audio frontend reference kernels.
	axpy_kernels.c
	AXPY kernels for FP32: y = y + alpha * x.
	deepseek_kernels.c
	Scalar reference kernels for DeepSeek-style research ops.
	deltanet_kernels.c
	FP32 Gated DeltaNet kernels for Qwen3.5-style recurrent attention.
	dequant_kernels.c
	Dequantization kernels for GGML-compatible formats.
	embedding_kernels.c
	Token/position embedding lookup kernels.
	embedding_kernels_bf16.c
	Token/position embedding lookup kernels for BF16.
	fp16_convert.c
	FP32 <-> FP16 SIMD conversion utilities.
	geglu_kernels.c
	GeGLU kernels split from gelu_kernels.c.
	gelu_kernels.c
	GELU activation kernels with SIMD (SSE/AVX/AVX512)
	gelu_kernels_bf16.c
	GELU activation kernels for BF16 tensors.
	gemm_batch_int8.c
	Batch GEMM kernels for quantized weights with INT8 activations.
	gemm_fused_kernels.c
	Fused GEMM Kernels with activations.
	gemm_head_major_output.c
	Output projection from head-major attention (NO LAYOUT CONVERSION)
	gemm_kernels.c
	General matrix multiply (GEMM) kernels with SIMD (SSE/AVX/AVX512)
	gemm_kernels_amx.c
	AMX (Advanced Matrix Extensions) GEMM kernels.
	gemm_kernels_bf16.c
	Optimized BF16 GEMM Kernels for AVX-512.
	gemm_kernels_f16.c
	GEMM kernels with FP16 (half-precision) weights.
	gemm_kernels_nvfp4.c
	Packed NVFP4 weight kernels for CPU inference.
	gemm_kernels_q4_0.c
	GEMM/GEMV kernels with Q4_0 quantized weights.
	gemm_kernels_q4_1.c
	GEMM/GEMV kernels with Q4_1 quantized weights.
	gemm_kernels_q4k.c
	GEMM/GEMV kernels with Q4_K quantized weights.
	gemm_kernels_q4k_avx.c
	AVX Q4_K x Q8_K matvec kernel for Sandy/Ivy Bridge.
	gemm_kernels_q4k_q8k.c
	Q4_K (weights) x Q8_K (activations) kernels for inference.
	gemm_kernels_q4k_q8k_avx2.c
	AVX2 Q4_K x Q8_K matvec kernel (inference only)
	gemm_kernels_q4k_q8k_vnni.c
	VNNI Q4_K x Q8_K matvec kernel (inference only)
	gemm_kernels_q4k_sse.c
	SSE4.1 Q4_K x Q8_K dot product kernels.
	gemm_kernels_q5_0.c
	GEMM/GEMV kernels with Q5_0 quantized weights.
	gemm_kernels_q5_0_sse.c
	SSE4.1 GEMM for Q5_0 quantized weights.
	gemm_kernels_q5_0_sse_v2.c
	SSE-optimized GEMM kernels for Q5_0 x Q8_K quantization.
	gemm_kernels_q5_1.c
	GEMM/GEMV kernels with Q5_1 quantized weights.
	gemm_kernels_q5_1_q8_1.c
	Q5_1 x Q8_1 contract kernels used for ggml parity (Gemma-sensitive path)
	gemm_kernels_q5_k.c
	GEMM/GEMV kernels with Q5_K quantized weights.
	gemm_kernels_q6k.c
	GEMM/GEMV kernels with Q6_K quantized weights.
	gemm_kernels_q6k_q8k.c
	Q6_K (weights) x Q8_K (activations) kernels for inference.
	gemm_kernels_q6k_sse.c
	SSE-optimized GEMM kernels for Q6_K x Q8_K quantization.
	gemm_kernels_q8_0.c
	GEMM/GEMV kernels with Q8_0 quantized weights.
	gemm_kernels_q8_0_q8_0_contract.c
	FP32 API adapters that enforce Q8_0 x Q8_0 activation contract.
	gemm_microkernel.c
	GEMM Microkernel - High-Performance Register-Blocked Matrix Multiplication.
	gemma4_per_layer_embed.c
	gemv_omp.c
	gemv_omp.h
	ggml_runtime_compat.h
	hybrid_attention_kernels.c
	hyper_connection_kernels.c
	kv_cache_kernels.c
	KV-cache helper kernels (head-major layout)
	layernorm_kernels.c
	LayerNorm forward/backward kernels with SIMD (SSE/AVX/AVX512)
	layernorm_kernels_bf16.c
	LayerNorm kernels for BF16 tensors.
	layout_kernels.c
	logit_kernels.c
	loss_kernels.c
	Loss function kernels (cross-entropy, etc.)
	loss_kernels_bf16.c
	Loss function kernels for BF16 tensors.
	mamba2_kernels.c
	mlp_fused_decode.c
	Fully fused MLP decode kernel (T=1 token generation)
	mlp_kernels.c
	MLP (feed-forward) kernels with SIMD (SSE/AVX/AVX512)
	mlp_kernels_bf16.c
	Optimized BF16 MLP Kernels.
	optimizer_kernels.c
	Optimizer kernels for training (AdamW, SGD)
	optimizer_kernels_bf16.c
	BF16 optimizer kernels for training.
	qk_norm_kernels.c
	Per-head RMSNorm on Q and K (Qwen3-style QK norm)
	quantize_row_q8_k_avx.c
	AVX entrypoint for exact Q8_K row quantization.
	quantize_row_q8_k_avx2.c
	AVX2 entrypoint for exact Q8_K row quantization.
	quantize_row_q8_k_avx512.c
	AVX-512 entrypoint for exact Q8_K row quantization.
	quantize_row_q8_k_sse.c
	SSE-optimized Q8_K row quantization kernel.
	qwen4_exp_kernels.c
	recurrent_gate_kernels.c
	recurrent_norm_kernels.c
	recurrent_qk_norm_kernels.c
	recurrent_split_kernels.c
	recurrent_state_kernels.c
	relu_kernels.c
	ReLU activation kernels with SIMD (SSE/AVX/AVX512)
	relu_kernels_bf16.c
	ReLU activation kernels for BF16 tensors.
	rmsnorm_kernels.c
	RMSNorm forward/backward kernels with SIMD (SSE/AVX/AVX512)
	rmsnorm_kernels_bf16.c
	RMSNorm kernels for BF16 tensors.
	rmsnorm_kernels_int4.c
	RMSNorm kernels with INT4 output quantization.
	rmsnorm_kernels_int8.c
	RMSNorm kernels with INT8 output quantization.
	rope_kernels.c
	RoPE (Rotary Position Embedding) kernels with SIMD.
	rope_kernels_bf16.c
	RoPE (Rotary Position Embedding) kernels for BF16.
	sigmoid_kernels.c
	Sigmoid activation kernels with SIMD (AVX512)
	sigmoid_kernels_bf16.c
	Sigmoid activation kernels for BF16 tensors.
	softmax_kernels.c
	Softmax forward/backward kernels with SIMD (SSE/AVX/AVX512)
	softmax_kernels_bf16.c
	Softmax kernels for BF16 tensors.
	ssm_kernels.c
	FP32 SSM causal depthwise convolution kernels for qwen3next/Qwen3.5.
	swiglu_kernels.c
	SwiGLU activation kernels with SIMD (SSE/AVX/AVX512)
	swiglu_kernels_bf16.c
	SwiGLU activation kernels for BF16 tensors.
	topk_kernels.c
	Top-K selection kernels for MoE router dispatch.
	vision_kernels.c
	Vision kernels (im2patch, patch embedding, etc.)
	vision_kernels_bf16.c
	Vision kernels for BF16 tensors (im2patch, etc.)