Benchmarks

yzma is fast because it calls llama.cpp in the same process. There is no external server.

These are the results for yzma 1.30.0 with llama.cpp v0.7.0, which is build b11669. Keep three things in mind.

• Each number is the median of five runs.
• The text tables and the multimodal tables use different models and different prompts. Numbers in one table don’t compare with numbers in the other.
• The WebAssembly numbers come from the generation loop of the browser example, not from the Go benchmark. They do not compare with the native tables.

The numbers change with each llama.cpp release. The detail files in the repository are always the newest ones. The measurements here are from 2026-10-16 to 2026-10-19. The AMD EPYC rows are older, from build b11146 on 2026-09-24 and 2026-09-25.

Text generation

The model is SmolLM-135M.Q2_K. The benchmark uses 4 threads on each machine.

This model is small, so a CPU can be faster than a GPU. On the M4 Pro, for example, the CPU is 1.7 times faster than Metal. A larger model changes the order.

The GPUs are an RTX 4070 Laptop in the Intel Core i9-13900HX, a Radeon RX 7900 XTX in the AMD EPYC 7443P, and an RTX 3070 in the AMD Ryzen 9 7950X. The i9-13900HX and the Ryzen 9 7950X also have an integrated GPU, which is slower. The tables show the fastest device only.

The details are in linux.md, macos.md, and windows.md.

Multimodal

The model is SmolVLM-256M-Instruct-Q8_0 with its projector. The benchmark uses one thread for each performance core, so the CPU rows reflect processor size.

A projector does many calculations in parallel, which is what a GPU is good at. So a GPU helps a multimodal model more than a text model.

The details are in the same three files.

In a browser

The model is SmolLM-135M.Q2_K.

The build with more threads is 8.0 times faster than the build with one thread. A page gets more than one thread only with the COOP and COEP headers. The Build for a browser page has them.

On a model this small the CPU with more threads beats WebGPU. The RTX 4070 is 3.4 times faster than the Intel graphics. WebGPU in Chrome on Linux needs three switches.

The details are in webassembly.md.

Comparing engines

These benchmarks are to compare inference performance using 3 different engines that support GGUF models:

• yzma
• ollama
• Docker Model Runner

The engines are yzma 1.30.0, ollama 0.40.5, and Docker Model Runner v1.2.8, on the Intel Core i9-13900HX with the RTX 4070.

yzma calls llama.cpp in the same process. ollama and Docker Model Runner require an OpenAI compatible REST interface, so each request pays for a round trip.

The details are in comparison.md.

The detail files

Each file has the output of every run and the information for each device.

Run the benchmarks yourself

Get the library and the models, then run the script.

make download-llama.cpp
make download-benchmark-models
./benchmarks/run.sh

On Windows, use a PowerShell prompt at the repository root. A Command Prompt opens the file in an editor and does not run it.

powershell -ExecutionPolicy Bypass -File .\benchmarks\run.ps1

The script asks llama.cpp which devices the machine has. It runs the text suite and the multimodal suite for each one, and it writes each result to the platform’s file. It reads the llama.cpp tag from yzma-install.json in the library directory.

Without --threadpool, the system moves the threads between CPUs during the run. A short run then gives a low number, and the numbers vary from run to run. The flag works only on Linux. On macOS and Windows the benchmark stops with an error, because these systems don’t report which CPUs are performance cores.

The text suite uses 4 threads on each machine, because more threads make this small model slower. The threads wait for each other after each operation. The multimodal suite uses one thread for each performance core, as a yzma program does.

The PowerShell script takes the same flags with one dash and a capital letter, such as -Backend and -DryRun. The flags go after the file name.

The machine name is part of the section key. Give the same name each time, or the file gets two sections for one machine.

Compare yzma with other engines yourself

This suite needs the servers to run, and each one needs the model.

make download-compare-models
docker model pull hf.co/qwen/qwen3-vl-4b-instruct-gguf:q4_k_m
./benchmarks/compare.sh

Every engine must read the same GGUF file. So the commands use the Hugging Face file and not gemma4:e4b or ai/gemma3, which are vendor conversions. The script tells you which command gets a missing model.

See Environment variables for the variables that the benchmarks read.

Measure your own program

llama.PerfContextPrint(ctx)
llama.PerfSamplerPrint(sampler)

These print how much time each part used.