Linux
Linux runs on amd64 and on arm64. Choose the tab for the processor that you want to use.
Decide where you want to put the library files, then run this command:
yzma install --lib /path/to/lib
To use an NVIDIA GPU, first install the CUDA drivers. See the CUDA installation guide.
Then run this command:
yzma install --lib /path/to/lib --processor cuda
The command reads the machine’s CUDA version and picks the CUDA 12 or CUDA 13 build. To choose one, use --processor cuda-12 or --processor cuda-13.
To use an Intel CPU, GPU, or NPU with OpenVINO, run this command. OpenVINO is on amd64 only.
yzma install --lib /path/to/lib --processor openvino
The download has the OpenVINO runtime. A GPU or an NPU also needs the Intel drivers. See Configurations for Intel hardware.
OpenVINO uses the CPU unless you choose another device:
export GGML_OPENVINO_DEVICE=GPU
To use an AMD GPU, first install the ROCm 7.2 drivers. See Install ROCm below.
Then run this command:
yzma install --lib /path/to/lib --processor rocm
yzma can also find ROCm on its own. If ROCm is already installed, this command is enough:
yzma install --lib /path/to/lib
To use Vulkan, first install the Vulkan drivers. For example:
sudo apt install -y mesa-vulkan-drivers vulkan-tools
Then run this command:
yzma install --lib /path/to/lib --processor vulkan
The yzma install command prints instructions for your system when it finishes. Follow them to complete the installation.
Then set the YZMA_LIB environment variable:
export YZMA_LIB=/path/to/lib
Install ROCm
You need these things before you start.
- An AMD GPU in the AMD supported GPU table, such as an AMD Instinct or a supported Radeon GPU.
- A Linux distribution that ROCm 7.2 supports. Ubuntu 24.04 and Ubuntu 22.04 are the most common.
- A compatible AMDGPU kernel driver. See the AMD driver installation instructions.
wget https://repo.radeon.com/amdgpu-install/7.2/ubuntu/noble/amdgpu-install_7.2.70200-1_all.deb
sudo apt install ./amdgpu-install_7.2.70200-1_all.deb
sudo apt update
sudo apt install python3-setuptools python3-wheel
sudo usermod -a -G render,video $LOGNAME
sudo apt install rocm
wget https://repo.radeon.com/amdgpu-install/7.2/ubuntu/jammy/amdgpu-install_7.2.70200-1_all.deb
sudo apt install ./amdgpu-install_7.2.70200-1_all.deb
sudo apt update
sudo apt install python3-setuptools python3-wheel
sudo usermod -a -G render,video $LOGNAME
sudo apt install rocm
Reboot the machine after you install ROCm. At a minimum, log in again so the render and video group changes take effect.
Check the installation:
rocminfo
For other Linux distributions, see the ROCm installation guide.
Next steps
Go to Download models.