Tutorials on yzma

Hello world

Mon, 01 Jan 0001 00:00:00 +0000

This is the smallest yzma program.

Before you start

Install yzma and the llama.cpp libraries. See Quick install. Then set YZMA_LIB.

Download the model:

yzma model get -u https://huggingface.co/QuantFactory/SmolLM2-135M-GGUF/resolve/main/SmolLM2-135M.Q4_K_M.gguf

Run the example

$ go run ./examples/hello/


"Yes, I'm ready to go."

The complete program is on the Your first program page.

The steps

1. Open the libraries.

llama.Load(libPath)
llama.LogSet(llama.LogSilent())
llama.Init()

LogSilent stops the messages that llama.cpp prints. Remove that line to see what the library does.

Chat

Mon, 01 Jan 0001 00:00:00 +0000

The hello program sends one prompt. A chat program keeps the conversation, so the model remembers what came before.

Before you start

Download a model that follows an instruction:

yzma model get -u https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct-GGUF/resolve/main/qwen2.5-0.5b-instruct-fp16.gguf

Run the example

$ go run ./examples/chat/ -model ./models/qwen2.5-0.5b-instruct-fp16.gguf
Enter prompt: Are you ready to go?

Yes, I'm ready to go! What would you like to do?

Enter prompt: Let's go to the zoo


Great! Let's go to the zoo. What would you like to see?

Enter prompt: I want to feed the llama


Sure! Let's go to the zoo and feed the llama. What kind of llama are you interested in feeding?

See the code.

Vision

Mon, 01 Jan 0001 00:00:00 +0000

A Vision Language Model reads an image and text together. yzma sends both with the pkg/mtmd package.

Before you start

A VLM needs two files. Download both:

yzma model get -u https://huggingface.co/ggml-org/Qwen2.5-VL-3B-Instruct-GGUF/resolve/main/Qwen2.5-VL-3B-Instruct-Q8_0.gguf
yzma model get -u https://huggingface.co/ggml-org/Qwen2.5-VL-3B-Instruct-GGUF/resolve/main/mmproj-Qwen2.5-VL-3B-Instruct-Q8_0.gguf

The file with mmproj in the name is the projector. It turns the image into tokens that the model reads.

Run the example

$ go run ./examples/vlm/ -model ~/models/Qwen2.5-VL-3B-Instruct-Q8_0.gguf -mmproj ~/models/mmproj-Qwen2.5-VL-3B-Instruct-Q8_0.gguf -image ./images/domestic_llama.jpg -p "What is in this picture?"

The image features a white llama standing in a fenced-in area, possibly a zoo or a farm. The llama is positioned in the center of the image, with its body facing the right side. The fenced area is surrounded by trees, creating a natural environment for the llama.

See the code.

Embeddings

Mon, 01 Jan 0001 00:00:00 +0000

An embedding is a vector of numbers that stands for a piece of text. Two pieces of text with a similar meaning give two similar vectors. Use embeddings for search, for grouping, and for comparison.

Before you start

Download a model:

yzma model get -u https://huggingface.co/QuantFactory/SmolLM-135M-GGUF/resolve/main/SmolLM-135M.Q2_K.gguf

Run the example

$ go run ./examples/embeddings/ -model ~/models/SmolLM-135M.Q2_K.gguf -p "Hello World"
-0.009294 -0.003438 0.004629 -0.005551 0.003048 ...

See the code.

The steps

1. Turn on embeddings in the context parameters.

Tool calling

Mon, 01 Jan 0001 00:00:00 +0000

A model cannot do arithmetic well and it cannot read the clock. Tool calling solves this. You tell the model which functions it has, the model asks for one, and your Go code runs it.

Before you start

Use a model that supports tool calling:

yzma model get -u https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct-GGUF/resolve/main/qwen2.5-0.5b-instruct-fp16.gguf

Run the example

$ go run ./examples/tooluse/ -model ~/models/qwen2.5-0.5b-instruct-fp16.gguf
=== Tool Calling Example ===

User: What is 15 + 27?

Assistant: 15 + 27 = 42

The result of adding 15 and 27 is 42.

Ask your own question:

Browser

Mon, 01 Jan 0001 00:00:00 +0000

yzma runs in a browser. The model stays on the machine of the reader, and no server does the work.

Before you start

You need TinyGo 0.41.1 or later, and a clone of the yzma repository.

git clone https://github.com/hybridgroup/yzma.git
cd yzma

Get the WebAssembly build

make download-llama.cpp-wasm

This is the same as:

yzma install --lib ./build/wasm --os wasm

It downloads all three builds. There is a build for WebGPU, a build with more than one thread, and a build with one thread.