Install from your program

Your application can install the llama.cpp libraries itself. Then your users don’t have to run the yzma command.

pkg/download handles this.

Install

import (
	"context"
	"runtime"

	"github.com/hybridgroup/yzma/pkg/download"
)

target := download.Target{
	Arch:      download.MustParseArch(runtime.GOARCH),
	OS:        download.MustParseOS(runtime.GOOS),
	Processor: download.CUDA,
	Version:   "latest",
}

err := download.Install(context.Background(), target, libPath, download.ProgressTracker, nil)

The last argument is the resolver. A nil value uses the built in table. See Custom resolvers.

See the example code.

The target

Find the processor

yzma can find CUDA and ROCm on the machine.

if ok, version := download.HasCUDA(); ok {
	target.Processor = download.CUDA
	target.CUDAVersion = version
} else if ok, _ := download.HasROCm(); ok {
	target.Processor = download.ROCm
} else {
	target.Processor = download.CPU
}

Set the CUDA version on the target, because on Linux it selects the CUDA 12 build or the CUDA 13 build. download.CUDA12 and download.CUDA13 name a build and ignore the version. The functions that take strings, such as download.Get, take the version with download.WithCUDAVersion.

The version

• An empty string uses download.DefaultVersion. That is the llama.cpp release that this yzma release was tested with. A development build of yzma leaves that empty, so an empty string then gets the most recent nightly build.
• "latest" always gets the most recent nightly build.
• A tag such as b10783 gets that release.

Install resolves the version to a release tag before it calls the resolver, so the resolver always sees a concrete version.

LlamaLatestVersion and LlamaNightlyTag read the current tags.

Do not install twice

if download.AlreadyInstalled(libPath, target) {
	return nil
}

Show the progress

download.ProgressTracker prints the progress to the terminal. DefaultProgressTracker makes a new one. Pass nil for no output. Write your own tracker to show the progress in your own user interface.

Download a model

err := download.GetModel(modelURL, download.DefaultModelsDir())

GetModelWithProgress and GetModelWithContext let you pass a tracker and a context.

Check the files

Install checks the SHA-256 of each asset before it writes anything. See Verify an installation.

Ship the libraries with your application

You don’t have to download anything. Put the library files beside your program and pass the path to llama.Load:

llama.Load(filepath.Join(exeDir, "lib"))

This gives you a program that works without a network. The cost is a larger download for the application, and you must build one package for each platform.