WebAssembly

How yzma runs in a browser.

yzma runs in a browser. The model stays on the reader’s machine, and no server does the work.

Two modules

There are two WebAssembly modules in the page.

flowchart TD
    page["page (index.html)"]
    worker["Web Worker"]
    go["Go program (TinyGo)"]
    cpp["llama.cpp module (Emscripten)"]

    page -->|postMessage| worker
    worker --> go
    worker --> cpp
    go -->|through JavaScript| cpp

On a native platform yzma calls llama.cpp with libffi and opens the shared libraries at run time. A WebAssembly module has no dlopen and no libffi, and TinyGo cannot compile the C++ of llama.cpp.

So llama.cpp becomes a second WebAssembly module. Emscripten builds it with a small C shim, and the Go code calls that module through JavaScript. The shim is in the wasm directory of the llama-cpp-builder repository.

The generation loop stays in Go. It does one Decode and one SamplerSample for each token, the same as a native program.

Three builds

The JavaScript glue selects the best build that the browser can run.

Build	What the browser must have
WebGPU	WebGPU with f16 shaders, and JSPI. Chrome or Edge 137 and later, or Firefox 153 and later with two settings in `about:config`. Chrome on Linux needs three switches. In Firefox the loader uses the CPU, which is faster there.
More threads	`SharedArrayBuffer`, so the page must send the COOP header and the COEP header.
One thread	Nothing. It works in every browser.

A browser with no WebGPU still works. It runs on the CPU.

The page prints which build it got:

backend: webgpu (WebGPU)

The GPU helps most on a page that takes images. Running an image through a model’s projector takes a second or two on the GPU, compared to half a minute or more on the CPU.

Why a Web Worker

The generation loop takes a long time. It would stop the page if it ran on the main thread. The worker lets the page keep drawing and accepting clicks.

The worker sends each piece of text to the page with postMessage, so the page shows the answer while the model makes it.

The files

File	What it does
`yzma-loader.js`	Finds the best build that the browser can run and loads it.
`worker.js`	Runs `llama.cpp` and the Go program in a Web Worker.
`index.html`	A page that loads a model and generates text.
`vlm.html`	A page that asks a question about an image.
`tools.html`	A page where the model calls tools.
`serve/main.go`	A static server that sends the COOP header and the COEP header.

The API

A browser program uses pkg/llamawasm. It has the calls that text generation, embeddings, and images need. The call names and order are the same as in pkg/llama and pkg/mtmd, so you can move a program from one to the other by changing the import.

The package also has the extended batch, which runs the decision heads of exp/decide in a browser. It needs yzma 1.30.0 and llama.cpp v0.7.0 or later.

Limits

The browser package doesn’t support audio, video, LoRA adapters, saving state to a file, or quantization. It saves context state in memory.

The browser must download the model file before the first token. A 2 GB model is a 2 GB download.