Linux Docker And GPU Passthrough - Revision history

Wikisailor: /* Introduction */

2026-02-09T10:52:00Z

Introduction

← Older revision		Revision as of 10:52, 9 February 2026
Line 1:		Line 1:
	==Introduction==		==Introduction==

	This page will be concerned with the actual hosts more general information about Docker can be found '''[[[[Docker Hosts \| here]]'''. We need to have some '''[[Virtual Machines]]''' to host the containers so that processing is reasonably separated and constrained with no one container gaining all of the CPU or GPU cycles of the entire Proxmox host. Another, possibly more important, consideration is that if the OS of Pear is upgraded we don't want to break a load of containers. The last point for VMs for containers is that Nvidia are fairly well known for breaking there drivers sometimes while they are being upgraded. So better to leave the GPU driver update until other people have tested it and do any Proxmox updates as a separate job.		This page will be concerned with the actual hosts more general information about Docker can be found '''[[Docker Hosts \| here]]'''. We need to have some '''[[Virtual Machines]]''' to host the containers so that processing is reasonably separated and constrained with no one container gaining all of the CPU or GPU cycles of the entire Proxmox host. Another, possibly more important, consideration is that if the OS of Pear is upgraded we don't want to break a load of containers. The last point for VMs for containers is that Nvidia are fairly well known for breaking there drivers sometimes while they are being upgraded. So better to leave the GPU driver update until other people have tested it and do any Proxmox updates as a separate job.

	==[[Data Archive\|Data Archive Host]]==		==[[Data Archive\|Data Archive Host]]==

Wikisailor: /* AI & Jellyfin */

2026-02-09T10:50:37Z

AI & Jellyfin

← Older revision		Revision as of 10:50, 9 February 2026
Line 9:		Line 9:
	==[[AI & Jellyfin]]==		==[[AI & Jellyfin]]==

	To partner with the data archive and collection services we also have some more AI and computationally intensive services that make use of the Nvidia 5060 Ti GPU		To partner with the data archive and collection services we also have some more AI and computationally intensive services that make use of the Nvidia 5060 Ti GPU such as [[AI & Jellyfin]]

	==GPU Host Quince==		==GPU Host Quince==

Wikisailor: /* Moving forward */

2026-02-09T10:49:10Z

Moving forward

← Older revision		Revision as of 10:49, 9 February 2026
Line 121:		Line 121:
	==Moving forward==		==Moving forward==

	Now that the GPU is fully functional on Quince we can install some docker applications containers to perform [[AI & Jellyfin \| AI & Jellyfin processing]]		Now that the GPU is fully functional on Quince we can install some docker applications containers to perform '''[[AI & Jellyfin \| AI & Jellyfin processing]]'''

Wikisailor: /* Proving the VM is using the GPU */

2026-02-09T10:48:49Z

Proving the VM is using the GPU

← Older revision		Revision as of 10:48, 9 February 2026
Line 118:		Line 118:
	LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+ EqualizationPhase1+		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+ EqualizationPhase1+
	This shows Speed 16GT/s which is what it should be.		This shows Speed 16GT/s which is what it should be.

			==Moving forward==

			Now that the GPU is fully functional on Quince we can install some docker applications containers to perform [[AI & Jellyfin \| AI & Jellyfin processing]]

Wikisailor: /* Docker Applications installed on Quince */

2026-02-09T10:44:44Z

Docker Applications installed on Quince

← Older revision		Revision as of 10:44, 9 February 2026
Line 118:		Line 118:
	LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+ EqualizationPhase1+		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+ EqualizationPhase1+
	This shows Speed 16GT/s which is what it should be.		This shows Speed 16GT/s which is what it should be.

	~~== Docker Applications installed on Quince ==~~

	~~===Installation Strategy===~~

	Once the Blackwell GPU passthrough was verified on the Pear host, we transitioned to the Quince VM to set up the containerized environment. This allows us to run high-performance AI (Ollama) and media (Jellyfin) apps while keeping the base OS clean.

	~~===Docker Engine Installation===~~

	~~We use the official Docker repository to ensure access to v29+, which includes critical patches for Gen 5 PCIe and Blackwell architecture support.~~
	~~sudo apt update~~
	~~sudo apt install ca-certificates curl gnupg~~
	~~Then setup the repository~~
	~~sudo install -m 0755 -d /etc/apt/keyrings~~
	~~curl -fsSL https://download.docker.com/linux/ubuntu/gpg \| sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg~~
	~~sudo chmod a+r /etc/apt/keyrings/docker.gpg~~
	echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" \| sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
	~~Next Install Engine & Compose~~
	~~sudo apt update~~
	~~sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin~~
	~~===NVIDIA Container Toolkit (The "Magic Bridge")===~~
	~~This toolkit enables the libnvidia-container library, which maps the physical GPU device files (/dev/nvidia0, etc.) into the virtualized Docker namespace.~~
	~~curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \| sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg~~
	curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \| sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' \| sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
	~~sudo apt update~~
	~~sudo apt install -y nvidia-container-toolkit~~
	~~and last we configure the Nvidia Container tool Kit and restart Docker~~
	~~sudo nvidia-ctk runtime configure --runtime=docker~~
	~~sudo systemctl restart docker~~
	~~===Verification & First App: Ollama ===~~
	~~We used the "Pull-on-Demand" feature to deploy Ollama. Docker automatically fetched the image from Docker Hub since it wasn't in the local "storage locker."~~
	~~docker run -d \~~
	~~--gpus all \~~
	~~-v ollama:/root/.ollama \~~
	~~-p 11434:11434 \~~
	~~--name ollama \~~
	~~--restart unless-stopped \~~
	~~ollama/ollama~~
	~~Note on Parameters~~
	* --gpus all: Crucial. Without this, the container sees a CPU only.
	* -v ollama:/root/.ollama: Preserves model weights (like Qwen or Llama) even if the container is deleted/upgraded.
	* Storage Logic: Docker Hub images are immutable; any changes you want to keep (like downloaded AI models) must be stored in a volume -v.
	* --restart unless-stopped: Ensures your AI is always online after a reboot.
	* Docker identifies the "app" by the Image Name at the very end of your command In this case, the name was ollama/ollama. Docker treats this like a URL. It looks at its local "storage locker" first. If it doesn't see ollama/ollama there, it automatically reaches out to Docker Hub (the global library of apps) to find it. This is the "Pull-on-Demand" feature.
	* Status Check: After running this, use docker exec -it ollama nvidia-smi to prove the container sees the Blackwell card.

	~~===Final Integration Step ===~~

	With Docker verified, we move from manual docker run commands to Docker Compose (.yaml). This allows for "Infrastructure as Code," where we can define our 16GB VRAM reservations and Pearpool log paths in a single, repeatable file.
	~~===The "Blackwell Stack" Compose File for Quince===~~

	~~The Compose file should be created in the home directory~~
	~~nano ~/compose.yaml~~
	~~The configuration for Quince to use the 16GB VRAM of the 5060 Ti efficiently is as follows.~~
	~~services:~~
	~~jellyfin:~~
	~~image: jellyfin/jellyfin:latest~~
	~~container_name: jellyfin~~
	~~network_mode: host # Best for DLNA/local discovery~~
	~~user: 1000:1000 # Assuming nigel is UID 1000~~
	~~volumes:~~
	~~- /mnt/jellyfin/docker/jellyfin/config:/config~~
	~~- /mnt/jellyfin/docker/jellyfin/cache:/cache~~
	~~- /mnt/jellyfin:/media~~
	~~deploy:~~
	~~resources:~~
	~~reservations:~~
	~~devices:~~
	~~- driver: nvidia~~
	~~count: 1~~
	~~capabilities: [gpu, video]~~
	~~restart: unless-stopped~~

	~~ollama:~~
	~~image: ollama/ollama:latest~~
	~~container_name: ollama~~
	~~volumes:~~
	~~- /mnt/jellyfin/docker/ollama:/root/.ollama~~
	~~ports:~~
	~~- "11434:11434"~~
	~~deploy:~~
	~~resources:~~
	~~reservations:~~
	~~devices:~~
	~~- driver: nvidia~~
	~~count: 1~~
	~~capabilities: [gpu]~~
	~~restart: unless-stopped~~

	~~open-webui:~~
	~~image: ghcr.io/open-webui/open-webui:main~~
	~~container_name: open-webui~~
	~~volumes:~~
	~~- /mnt/jellyfin/docker/open-webui:/app/data~~
	~~environment:~~
	~~- 'OLLAMA_BASE_URL=http://ollama:11434'~~
	~~- 'WEBUI_SECRET_KEY=change_me_to_a_long_random_string' # Crucial for security~~
	~~- 'ENABLE_SIGNUP=true' # Set to false after you create your account~~
	~~ports:~~
	~~- "3000:8080"~~
	~~extra_hosts:~~
	~~- "host.docker.internal:host-gateway"~~
	~~restart: unless-stopped~~

	~~If the test container is still running stop it with the command~~
	~~docker stop ollama && docker rm ollama~~
	~~Launch the stack with~~
	~~docker compose up -d~~
	~~Verify the three apps are running with~~
	~~docker ps~~
	~~change the secretkey to a random string for security~~
	~~'WEBUI_SECRET_KEY=change_me_to_a_long_random_string' # Crucial for security~~
	~~Once the OpenWebui has been logged in with a username and password change the signup to false~~
	~~- 'ENABLE_SIGNUP=true' # Set to false after you create your account~~

Wikisailor at 10:41, 9 February 2026

2026-02-09T10:41:48Z

← Older revision		Revision as of 10:41, 9 February 2026
Line 1:		Line 1:
	==Introduction==		==Introduction==

	This page will be concerned with the actual hosts more general information about Docker can be found '''[[~~Linux~~ Docker ~~And GPU Passthrough~~ \| here]]'''. We need to have some '''[[Virtual Machines]]''' to host the containers so that processing is reasonably separated and constrained with no one container gaining all of the CPU or GPU cycles of the entire Proxmox host. Another, possibly more important, consideration is that if the OS of Pear is upgraded we don't want to break a load of containers. The last point for VMs for containers is that Nvidia are fairly well known for breaking there drivers sometimes while they are being upgraded. So better to leave the GPU driver update until other people have tested it and do any Proxmox updates as a separate job.		This page will be concerned with the actual hosts more general information about Docker can be found '''[[[[Docker Hosts \| here]]'''. We need to have some '''[[Virtual Machines]]''' to host the containers so that processing is reasonably separated and constrained with no one container gaining all of the CPU or GPU cycles of the entire Proxmox host. Another, possibly more important, consideration is that if the OS of Pear is upgraded we don't want to break a load of containers. The last point for VMs for containers is that Nvidia are fairly well known for breaking there drivers sometimes while they are being upgraded. So better to leave the GPU driver update until other people have tested it and do any Proxmox updates as a separate job.

	==[[Data Archive\|Data Archive Host]]==		==[[Data Archive\|Data Archive Host]]==

Wikisailor at 10:30, 9 February 2026

2026-02-09T10:30:45Z

Show changes

Wikisailor: /* Docker Hosts */

2026-02-03T14:46:41Z

Docker Hosts

← Older revision		Revision as of 14:46, 3 February 2026
Line 10:		Line 10:
	==[[Docker Hosts]]==		==[[Docker Hosts]]==

	Quince will be the Host for GPU passthrough, Jellyfin and Ollama. ~~Nectarine~~ will be the host for the news archive. Further details on the docker hosts can be found '''[[Docker Hosts \| Here]].'''		Quince will be the Host for GPU passthrough, Jellyfin and Ollama. Blackberry will be the host for the news archive. Further details on the docker hosts can be found '''[[Docker Hosts \| Here]].'''

Wikisailor: /* Docker Hosts */

2026-02-01T14:31:29Z

Docker Hosts

← Older revision		Revision as of 14:31, 1 February 2026
Line 10:		Line 10:
	==[[Docker Hosts]]==		==[[Docker Hosts]]==

	Quince will be the Host for GPU passthrough, Jellyfin and Ollama. Nectarine will be the host for the news archive. Further details on the docker hosts can be found ''[[Docker Hosts \| Here]].'''		Quince will be the Host for GPU passthrough, Jellyfin and Ollama. Nectarine will be the host for the news archive. Further details on the docker hosts can be found '''[[Docker Hosts \| Here]].'''

Wikisailor: Created page with "==Introduction== It became apparent that We need to be able to run Docker containers as well as LXCs so some new '''Virtual Machines''' are being created with the sole purpose of running containers. With the new Linux drivers for Nvidia 5060ti GPUs it is now practical to use GPU passthrough to a Linux VM rather than a Windows 11 host. In turn with the GPU on Linux, it can and should be shared amongst any applications that use GPU to speed up their operation, obvious..."

2026-02-01T14:31:14Z

Created page with "==Introduction== It became apparent that We need to be able to run Docker containers as well as LXCs so some new '''Virtual Machines''' are being created with the sole purpose of running containers. With the new Linux drivers for Nvidia 5060ti GPUs it is now practical to use GPU passthrough to a Linux VM rather than a Windows 11 host. In turn with the GPU on Linux, it can and should be shared amongst any applications that use GPU to speed up their operation, obvious..."

New page

==Introduction==

It became apparent that We need to be able to run Docker containers as well as LXCs so some new '''[[Virtual Machines]]''' are being created with the sole purpose of running containers. With the new Linux drivers for Nvidia 5060ti GPUs it is now practical to use GPU passthrough to a Linux VM rather than a Windows 11 host. In turn with the GPU on Linux, it can and should be shared amongst any applications that use GPU to speed up their operation, obvious candidates are Jellfin an LLMs, To that end Linux has been installed on the host Quince and Docker containers can run on it.

==The Logical Choice==

Walnut had hosted Jellyfin for a while because Nvidia GPU drivers were not working well on Linux but did install easily on Windows 11 Pro and to have hardware transcoding Jellyfin could use the GPU so it made sense to have the GPU passthrough to Walnut but it was only ever intended as a temporary measure to make Jellyfin work until Nvidia got decent Linux drivers. Running Docker on a Window 11 pro VM was causing problems in that it is a '''[[Virtual Machines | Virtual Machine]]'''with another Virtualisation on top so it was only practical to run Jellyfin. However, now with working Linux drivers, the VM Quince is now using the GPU. Also Quince is running Docker. So with docker and GPU Jellyfin is setup as a docker container application and as the GPU is on the host it can share its processing for hardware transcoding. At the same time Quince also has Ollama running in a different container so it can also use the GPU. It seems reasonable that other docker images will be wanted at some point and some of them may benefit from the GPU's processing power while others will not really use the GPU at all, so to keep the load down and under control we will have at least one separate host. A possible second host will have a news archiving suite of containers that will not need the GPU at all but may well be moving large amounts of data around and we do not want it to interfere with transcoding or LLM processing on Quince.

==[[Docker Hosts]]==

Quince will be the Host for GPU passthrough, Jellyfin and Ollama. Nectarine will be the host for the news archive. Further details on the docker hosts can be found ''[[Docker Hosts | Here]].'''