Packages

What each Go package in yzma does.

yzma is a set of Go packages. Most programs use two or three of them.

Package What it does
pkg/llama The main bindings. Models, contexts, tokens, batches, samplers, and conversation memory.
pkg/mtmd Multimodal input. Images, audio, and video.
pkg/download Downloads the llama.cpp libraries and the models. Checks the digests.
pkg/message Chat messages and tool calls. Reads the tool call format of each model family.
pkg/template Jinja chat templates.
pkg/llamawasm The browser version of pkg/llama. Build tag js && wasm.
pkg/vlm An experimental high level type for Vision Language Models.
pkg/loader Opens a shared library and prepares a function call.
pkg/utils Converts a Go string to a C string and back.
exp/speculative Experimental. The NextN hidden state calls for MTP speculative decoding. They can change with a llama.cpp update.
exp/decide Experimental. Runs System One models that answer a typed question with a probability for each option. See Typed decisions.

pkg/llama

This is the package that most programs import. It has about 250 functions in these groups.

  • Load. Load, Init, Close, PrintSystemInfo.
  • Model. ModelLoadFromFile, ModelDefaultParams, ModelGetVocab, ModelChatTemplate, ModelMeta*.
  • Context. InitFromModel, ContextDefaultParams, NCtx, SetNThreads.
  • Tokens. Tokenize, Detokenize, TokenToPiece, VocabIsEOG.
  • Inference. Decode, Encode, BatchGetOne, GetLogits, GetEmbeddings.
  • Sampling. SamplerChainInit, SamplerChainAdd, SamplerSample, and a function for each sampler.
  • Memory. GetMemory, MemoryClear, MemorySeqRm, and the other sequence operations.
  • State. StateSaveFile, StateLoadFile.
  • LoRA. AdapterLoraInit, SetAdaptersLora.
  • Extended batches. BatchExtInit, BatchExtAddToken, BatchExtSetPos, BatchExtSetDecisionOrder, Process.
  • Backends. GGMLBackendLoadAll, GGMLBackendDeviceCount, GGMLBackendDeviceName, GGMLBackendLoadErrors.

pkg/mtmd

mtmd is the multimodal library of llama.cpp. Use it to send an image, a sound, or a video with the text.

  • InitFromFile loads the projector file.
  • BitmapInitFromFile reads an image.
  • BitmapInitFromAudio reads a sound.
  • VideoInit reads a video. This needs ffmpeg and ffprobe on the PATH.
  • SupportVision, SupportAudio, and SupportVideo report what the model accepts.

pkg/download

Use this package to install the libraries from inside your own program, and to download a model.

  • Install downloads and extracts the libraries for a Target.
  • Resolver names the files to install. Write one to use your own mirror.
  • VerifyInstall checks the files against the release digests.
  • GetModel downloads a model file.
  • DefaultModelsDir gives the default directory for models.

Which package for a browser

A browser program imports pkg/llamawasm instead of pkg/llama and pkg/mtmd. The call names and order are the same, so you can move a program from one to the other by changing the import.

The browser package is smaller. It has text generation, embeddings, images, every native sampler, logits, model metadata, and performance counters. It doesn’t support audio, video, LoRA adapters, saving state to a file, or quantization. It saves context state in memory.