Docker Hosts

Introduction

this page will be concerned with the actual hosts more general information about Docker can be found here. We need to have some Virtual Machines to host the containers so that processing is reasonably separated and constrained with no one container gaining all of the CPU or GPU cycles of the entire Proxmox host. Another, possibly more important, consideration is that if the OS of Pear is upgraded we don't want to break a load of containers. The last point for VMs for containers is that Nvidia are fairly well known for breaking there drivers sometimes while they are being upgraded. So better to leave the GPU driver update until other people have tested it and do any Proxmox updates as a separate job.

GPU Host Quince

We are using Quince 192.168.100.75/24 as the host for GPU Passthrough and as a consequence it will have Jellyfin and Ollama docker containers.

Quince Specification

To store the OS we have a 150gb drive allocated from the SSD Rpool and to keep all of the media files we have a 3TB hard drive allocated from Pearpool. As a temporary measure the media HD from Walnut has also been added to enable the media files to be copied to the new HD, it was impractical to keep the walnut HD on quince as it is NTFS and quince is Linux so while it would work it is not the preferred.

• Hostname is Quince
• IP Address is 192.168.100.75/24
• RAM is 32gb
• Processor is type Host and has 1 socket with 10 cores
• Bios is OVMF (UEFI)
• Machine type is q35
• OS Storage is 150gb allocated from Rpool
• Media storage is 3tb allocated from Pearpool
• NIC is on production VLAN
• Display is set to default
• PCI device is 0000:07:00 (the Nvidia 5060ti 16gb GPU)

GPU setup on Quince

We will include the full guide to GPU passthrough to a Linux host but it should be noted alot of the steps were already done while preparing to do the same on Walnut. Speaking of walnut we need to disable all of the GPU passthrough settings on walnut before we proceed so it should have the PCI device removed and a a display set to Virto-GPU. If walnut PCI is not reset the GPU passthrough will fail, obviously, but if the display is not changed it will have no screen to output to.

Host Preparation (The Proxmox Level)

First the host must be told to "ignore" the GPU so it can be handed over to the VM. Enable IOMMU in GRUB: on the Proxmox host

nano /etc/default/grub

The basic line to edit is

# For Intel CPUs:
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"
# For AMD CPUs:
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt"

However, as it didn't work with the first try the line was changed but it is not known if it was this change that made it work or some other trouble shooting step. So try the above line but if it does not work try

 GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt pcie_acs_override=downstream,multifunction pcie_aspm=off"

After save and close

update-grub 
reboot

The next step is Load VFIO Modules: Add these to /etc/modules to allow the "hand-off" to the VM

nano /etc/modules

and add the following settings. Please note Proxmox 8.x WebGUI will possibly work without these settings but it is better to add them to avoid race conditions between two GPUs and more importantly in case future versions of Proxmox GUI changes how it handles PCI devices.

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

Save and close. the last thing to do on the Proxmox host is to blacklist Drivers on Host to Prevent Proxmox from using the card (so that we can pass it through to a Guest) by creating /etc/modprobe.d/blacklist.conf

nano /etc/modprobe.d/blacklist.conf

and add the following lines

blacklist nouveau
blacklist nvidia
blacklist nvidiafb

Save and close.