Quick install guide on yzma

macOS

Mon, 01 Jan 0001 00:00:00 +0000

macOS on Apple silicon uses the Metal GPU. You do not need to install a driver.

Decide where you want to put the library files, then run this command:

yzma install --lib /path/to/lib

The yzma install command prints instructions for your system when it finishes. Follow them to complete the installation.

Then set the YZMA_LIB environment variable:

export YZMA_LIB=/path/to/lib

Next steps

Go to Download models.

Linux

Mon, 01 Jan 0001 00:00:00 +0000

Linux runs on amd64 and on arm64. Choose the tab for the processor that you want to use.

Decide where you want to put the library files, then run this command:

yzma install --lib /path/to/lib

To use an NVIDIA GPU, first install the CUDA drivers. See the CUDA installation guide.

Windows

Mon, 01 Jan 0001 00:00:00 +0000

Choose the tab for the processor that you want to use.

Decide where you want to put the library files, then run this command:

yzma install --lib C:\path\to\lib

To use an NVIDIA GPU, first install the CUDA drivers. See the CUDA installation guide.

Raspberry Pi

Mon, 01 Jan 0001 00:00:00 +0000

yzma runs on a Raspberry Pi 4 and on a Raspberry Pi 5. You need the 64 bit version of the Raspberry Pi OS.

The Raspberry Pi OS uses its own library versions, so you must name the operating system with the --os flag.

Use this command for the latest version of the Raspberry Pi OS:

NVIDIA Jetson Orin

Mon, 01 Jan 0001 00:00:00 +0000

yzma runs on the NVIDIA Jetson Orin. Install the latest version of the Jetpack software for your device before you start. Jetpack gives you the GPU drivers.

Decide where you want to put the library files, then run this command:

yzma install --lib /path/to/lib --processor cuda

Vulkan on the Jetson Orin needs a newer GLIBC. Upgrade the shared libraries first:

Arduino UNO Q

Mon, 01 Jan 0001 00:00:00 +0000

yzma runs on the Arduino UNO Q board. The board runs a Debian based Linux, so the installation uses the trixie operating system name.

yzma install --lib /path/to/lib --processor cpu --os trixie

Then set the YZMA_LIB environment variable:

export YZMA_LIB=/path/to/lib

The board has a small amount of memory. Use a small model. See Models.

Next steps

Go to Download models.

Browser

Mon, 01 Jan 0001 00:00:00 +0000

yzma runs in a browser with the WebAssembly build of llama.cpp.

yzma install --lib /path/to/web --os wasm

This command downloads all three builds. There is a build for WebGPU, a build with more than one thread, and a build with one thread. The JavaScript glue selects the best build that the page can run.

Build	What the browser must have
WebGPU	WebGPU with f16 shaders, and JSPI. Chrome or Edge 137 and later, or Firefox 153 and later with two settings in `about:config`.
More threads	`SharedArrayBuffer`, so the page must send the COOP header and the COEP header.
One thread	Nothing. It works in every browser.

A browser with no WebGPU still works. It runs on the CPU.

Manual installation

Mon, 01 Jan 0001 00:00:00 +0000

You do not have to use the yzma install command. You can download and extract the library files yourself.

Where to get the files

Most of the prebuilt llama.cpp binaries are here:

https://github.com/ggml-org/llama.cpp/releases

We build the Ubuntu arm64 CUDA and Vulkan binaries, and the WebAssembly builds. They are here:

https://github.com/hybridgroup/llama-cpp-builder/releases

Extract the files

Extract the library files into a directory on your machine. The file extension depends on the operating system.