NVIDIA Jetson Orin

How to install the llama.cpp libraries on an NVIDIA Jetson Orin.

yzma runs on the NVIDIA Jetson Orin. Install the latest version of the Jetpack software for your device before you start. Jetpack gives you the GPU drivers.

Decide where you want to put the library files, then run this command:

yzma install --lib /path/to/lib --processor cuda

Vulkan on the Jetson Orin needs a newer GLIBC. Upgrade the shared libraries first:

sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt-get update
sudo apt-get install --only-upgrade libstdc++6

Then run this command:

yzma install --lib /path/to/lib --processor vulkan

The yzma install command prints instructions for your system when it finishes. Follow them to complete the installation.

Then set the YZMA_LIB environment variable:

export YZMA_LIB=/path/to/lib

Next steps

Go to Download models.