kimi-k3-in-c
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
Automatically checked · GitHub checked 2026-11-17
Overview
Same short prompt at every size, and the output is byte-identical from the smallest machine to the largest; only the clock changes. One machine, 124 cores, fast NVMe drive: the first three rows still read the model from disk each step, so a slower drive is slower there, while the 128 GB+ row keeps everything in memory and no longer waits on the disk. On that same machine v1.0.0 made the math per token about 8× lighter, a follow-up question in a chat 3.9× faster, and long prompts about half as costly. (A token is roughly a short word-piece; the two runnable demos below are the original captures on a slower drive, so their clock reads a little higher.) Full data in docs/data/. The macOS, Windows and NEON ports, chat mode, the ultra preset, the faster MXFP4 kernels, cgroup-aware memory admission and a long list of fixes, most of 2.0.0 among them, came from the people below. Who did what. Slow, and answering correctly, in 8.24 GB, from a checkpoint of 1.56 TB. This particular batch command deliberately asks for a raw continuation. The official Kimi K3 checkpoint is also chat-capable; use the XTML REPL below when you want answers and multi-turn history. Give the same batch request more memory and the answer does not change, only the clock: Every figure in this document comes from the measurement output in docs/data/. The dense trunk stays in memory to whatever depth you choose and streams the rest; the 1.45 TB of routed experts are never resident, and are multiplied straight out of their packed 4-bit form. The consequence is that the same model runs in 8 GB and in 224 GB and produces byte-identical output at every budget between. Part II builds every box in both diagrams from scratch, one component at a time. - 1. Reading a 1.56 TB checkpoint from its headers The code in this part is condensed from the source for reading: names, comments and error handling are sometimes simplified, and paths a section does not discuss are left out. The files under src/ are the reference.