Packages

The Go packages and where to read their API.

The complete API is on pkg.go.dev.

Package	Import path	What it does
llama	`pkg/llama`	The main bindings for `llama.cpp`.
mtmd	`pkg/mtmd`	Images, audio, and video.
download	`pkg/download`	Installs the libraries and the models.
message	`pkg/message`	Chat messages and tool calls.
template	`pkg/template`	Jinja chat templates.
llamawasm	`pkg/llamawasm`	The browser version. Build tag `js && wasm`.
vlm	`pkg/vlm`	An experimental high level type for Vision Language Models.
loader	`pkg/loader`	Opens a shared library and prepares a call.
utils	`pkg/utils`	Converts a Go string to a C string and back.
speculative	`exp/speculative`	Experimental. The NextN hidden state calls for MTP speculative decoding.
decide	`exp/decide`	Experimental. Runs System One models that answer a typed question with probabilities.

See Packages for what each one is for.

The most common calls

Call	What it does
`llama.Load(path)`	Opens the shared libraries. An empty path reads `YZMA_LIB`.
`llama.Init()`	Starts the backend and loads the backend libraries. Returns an error.
`llama.Version()`	The version of the loaded `llama.cpp`.
`llama.LogSet(llama.LogSilent())`	Silences the `llama.cpp` log messages.
`llama.Close()`	Closes the libraries.

Call	What it does
`llama.ModelDefaultParams()`	The default model parameters.
`llama.ModelLoadFromFile(path, params)`	Reads a GGUF file.
`llama.ModelGetVocab(model)`	Returns the vocabulary.
`llama.ContextDefaultParams()`	The default context parameters.
`llama.InitFromModel(model, params)`	Makes a context.
`llama.ModelFree(model)`	Frees the model.

Call	What it does
`llama.Tokenize(vocab, text, addBOS, parseSpecial)`	Text to tokens.
`llama.TokenToPiece(vocab, token, buf, lstrip, special)`	One token to text.
`llama.Detokenize(...)`	Tokens to text.
`llama.VocabIsEOG(vocab, token)`	Reports if a token is an end of generation token.

Call	What it does
`llama.BatchGetOne(tokens)`	Makes a batch from tokens.
`llama.Decode(ctx, batch)`	Runs the model.
`llama.SamplerSample(sampler, ctx, idx)`	Samples the next token.
`llama.GetEmbeddingsSeq(ctx, seq, n)`	Returns an embedding vector.

import "github.com/hybridgroup/yzma"

The root package holds the package documentation. The yzma version command prints the tool version.