tokens/sec counts generated tokens over wall-clock inference time, including prompt prefill and both models’ work, excluding model loading. Draft acceptance counts accepted proposals divided by all proposals; it is N/A when speculation is off.

Target rounds counts logical verification blocks, not individual forward passes. Tokens/target-round is emitted tokens divided by rounds. Baseline target rounds is the analytical one-token-at-a-time count for this output length, including a stop-token decision if one occurs. It is not a separately timed run.

Choose 15M → 42M: “Once upon a time”, temperature 0, lookahead 4, 60 new tokens. The repository’s recorded run accepted 46 / 56 proposals (82.1%) in 14 target rounds, yielding 4.29 tokens/round against a baseline of 60.

Those are historical results, not prefilled live measurements. CPU throughput varies by host. Greedy output matches the target alone; sampled runs preserve its distribution but need not produce the same text, even with the same seed. Read the transcript ↗

lua-llama by jeorgexyz is the MIT-licensed inference runtime and experiment. The pretrained TinyStories 15M and 42M checkpoints come from Andrej Karpathy’s tinyllamas repository, whose model card declares MIT. They were not trained by jeorgexyz.

The tokenizer comes from llama2.c. Assets are downloaded from pinned upstream revisions and checked with SHA-256. View asset provenance ↗