Linux

How to install the llama.cpp libraries on Linux.

Linux runs on amd64 and on arm64. Choose the tab for the processor that you want to use.

Decide where you want to put the library files, then run this command:

yzma install --lib /path/to/lib

To use an NVIDIA GPU, first install the CUDA drivers. See the CUDA installation guide.

Then run this command:

yzma install --lib /path/to/lib --processor cuda

To use an AMD GPU, first install the ROCm 7.2 drivers. See Install ROCm below.

Then run this command:

yzma install --lib /path/to/lib --processor rocm

yzma also finds ROCm without help. If ROCm is already installed, this command is sufficient:

yzma install --lib /path/to/lib

To use Vulkan, first install the Vulkan drivers. For example:

sudo apt install -y mesa-vulkan-drivers vulkan-tools

Then run this command:

yzma install --lib /path/to/lib --processor vulkan

The yzma install command prints instructions for your system when it finishes. Follow them to complete the installation.

Then set the YZMA_LIB environment variable:

export YZMA_LIB=/path/to/lib

Install ROCm

You need these things before you start.

  • An AMD GPU in the AMD supported GPU table, such as an AMD Instinct or a supported Radeon GPU.
  • A Linux distribution that ROCm 7.2 supports. Ubuntu 24.04 and Ubuntu 22.04 are the most common.
  • A compatible AMDGPU kernel driver. See the AMD driver installation instructions.
wget https://repo.radeon.com/amdgpu-install/7.2/ubuntu/noble/amdgpu-install_7.2.70200-1_all.deb
sudo apt install ./amdgpu-install_7.2.70200-1_all.deb
sudo apt update
sudo apt install python3-setuptools python3-wheel
sudo usermod -a -G render,video $LOGNAME
sudo apt install rocm
wget https://repo.radeon.com/amdgpu-install/7.2/ubuntu/jammy/amdgpu-install_7.2.70200-1_all.deb
sudo apt install ./amdgpu-install_7.2.70200-1_all.deb
sudo apt update
sudo apt install python3-setuptools python3-wheel
sudo usermod -a -G render,video $LOGNAME
sudo apt install rocm

Reboot the machine after you install ROCm. The render group and the video group need at least a new login.

Check the installation:

rocminfo

For other Linux distributions, see the ROCm installation guide.

Next steps

Go to Download models.