Reference on yzma

Command line

Mon, 01 Jan 0001 00:00:00 +0000

Install the command

go install github.com/hybridgroup/yzma@latest

The commands

NAME:
   yzma - YZMA command line tool

USAGE:
   yzma [global options] command [command options]

COMMANDS:
   install  Install llama.cpp libraries used by yzma
   verify   Check the installed llama.cpp libraries against their published digests
   system   Show llama.cpp system information
   llama    Show most recent llama.cpp version
   model    Manage models
   version  Show yzma version
   info     Show yzma version
   help, h  Shows a list of commands or help for one command

yzma install

Downloads the prebuilt llama.cpp libraries for this machine.

Environment variables

Mon, 01 Jan 0001 00:00:00 +0000

For your programs

YZMA_LIB

The directory that holds the llama.cpp shared library files. This is the only variable that a normal program needs.

export YZMA_LIB=/path/to/lib

set YZMA_LIB=C:\yzma\lib

The yzma install command and the yzma verify command also read it, so the --lib flag is not necessary when it is set.

Platforms

Mon, 01 Jan 0001 00:00:00 +0000

Operating systems and processors

Operating system	CPU	GPU
Linux	amd64, arm64	CUDA, Vulkan, HIP, ROCm, SYCL
macOS	arm64	Metal
Windows	amd64	CUDA, Vulkan, HIP, SYCL, OpenCL

A browser is also a target:

Target	CPU	GPU
Browser	wasm32 SIMD, one or more threads	WebGPU

In a browser the API is the smaller one of pkg/llamawasm. It has no audio, no video, no LoRA adapters, no saved state, and no quantization.

Boards

Board	Notes
Raspberry Pi 4 and 5	The 64 bit Raspberry Pi OS. Install with `--os trixie` or `--os bookworm`.
NVIDIA Jetson Orin	Install Jetpack first. CUDA or Vulkan.
Arduino UNO Q	Install with `--os trixie`.

Go version

yzma needs Go 1.26 or later.

Examples

Mon, 01 Jan 0001 00:00:00 +0000

The examples directory holds these programs. Each one has its own README.

Native

Example	What it shows
`hello`	The smallest program. Loads a model and prints an answer.
`chat`	An interactive chat that keeps the conversation.
`vlm`	Sends an image and a prompt to a Vision Language Model.
`describe`	Describes an image from a URL or a local file. Installs as a command.
`embeddings`	Turns text into a vector.
`tooluse`	One round of tool calling.
`multitool`	Tool calling in a loop, for a problem with steps.
`modelinfo`	Prints the description, the size, and the metadata of a model.
`systeminfo`	Lists the devices that `llama.cpp` found.
`installer`	Installs the `llama.cpp` libraries from Go code.
`resolver`	Installs from your own mirror with a custom resolver.

Browser

Example	What it shows
`wasm/chat`	A chat page.
`wasm/vlm`	A page that asks a question about an image.
`wasm/tools`	A page where the model calls tools.

Run one

Clone the repository, set YZMA_LIB, and download a model.

Packages

Mon, 01 Jan 0001 00:00:00 +0000

The complete API is on pkg.go.dev.

Package	Import path	What it does
llama	`pkg/llama`	The main bindings for `llama.cpp`.
mtmd	`pkg/mtmd`	Images, audio, and video.
download	`pkg/download`	Installs the libraries and the models.
message	`pkg/message`	Chat messages and tool calls.
template	`pkg/template`	Jinja chat templates.
llamawasm	`pkg/llamawasm`	The browser version. Build tag `js && wasm`.
vlm	`pkg/vlm`	An experimental high level type for Vision Language Models.
loader	`pkg/loader`	Opens a shared library and prepares a call.
utils	`pkg/utils`	Converts a Go string to a C string and back.

See Packages for what each one is for.

Roadmap

Mon, 01 Jan 0001 00:00:00 +0000

yzma covers more than 96 percent of the llama.cpp functionality.

The complete list changes with each release, so it stays in the repository:

ROADMAP.md

That page gives a table for each area. Each row names a llama.cpp function and gives two columns. One column is the state of the wrapper in yzma. The other column is the state of the same function in a browser.

The areas

Backend, Model, Vocab, Context, Backend Sampling, Memory, Batch, Sampling, Logging, Performance, Chat, State, LoRA, and mtmd.

Benchmarks

Mon, 01 Jan 0001 00:00:00 +0000

yzma is fast because it calls llama.cpp in the same process. There is no external server.

The complete measurements change with each release, so they stay in the repository:

BENCHMARKS.md

That page has results for Linux with a CPU on amd64 and arm64, Linux with CUDA, ROCm, and Vulkan, macOS with Metal, and Windows with a CPU, CUDA, and Vulkan. Each group has text model results and multimodal model results.

llama.cpp versions

Mon, 01 Jan 0001 00:00:00 +0000

Sometimes a change in llama.cpp breaks yzma. This page gives the known compatible versions.

For the current list, see the README.

Tagged releases

llama.cpp	yzma
v0.3.0	v1.25.0
v0.4.0	v1.26.0 to v1.26.1

A tagged release of yzma installs its own llama.cpp release by default. Thus yzma install with no --version flag gets the version in this table.

Use -version latest to get the most recent nightly build instead. A build from the main branch always uses the most recent nightly build.