AI & Jellyfin

Introduction

The objective for Quince is to make it the power house Virtual Machine for the AI so it needs to have the Nvidia 5060TI GPU passthrough completed . At the same time it will also run a media server in the form of Jellyfin so that the GPU can do the transcoding. With the AI being serviced by this host we can use Blackberry for the Data Harvesting.

Docker Applications installed on Quince

We are going to need to install several applications that will share the GPU. The first will Dockge so that any new containers can be managed easily. We will also need to install Ollama so that we can run LLMs easily. Then to make use of the data archive we can use AnythingLLM.

Installation Strategy

Once the Blackwell GPU passthrough was verified on the Pear host, we transitioned to the Quince VM to set up the containerized environment. This allows us to run high-performance AI (Ollama) and media (Jellyfin) apps while keeping the base OS clean.

Docker Engine Installation

We use the official Docker repository to ensure access to v29+, which includes critical patches for Gen 5 PCIe and Blackwell architecture support.

sudo apt update
sudo apt install ca-certificates curl gnupg

Then setup the repository

sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Next Install Engine & Compose

sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

NVIDIA Container Toolkit (The "Magic Bridge")

This toolkit enables the libnvidia-container library, which maps the physical GPU device files (/dev/nvidia0, etc.) into the virtualized Docker namespace.

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt update
sudo apt install -y nvidia-container-toolkit

and last we configure the Nvidia Container tool Kit and restart Docker

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker