Proxmox Server

Introduction

This is the server that will sit under my desk until I move house. It has the hostname Pear and will be the beginning of the Home Lab.

Create Virtual Machine from a template

it is possible to create a new VM based on a fixed template. The two possibilities are a full clone or a linked clone. the linked clone is the preferred as it should need less storage but the full clone would be completely independent of the template so it will use more storage and take longer to create.We can go down that rabbit hole when we look ant Ansible in a big way. For now just understand that we can call a new vm using the qm clone command. the will be more details to follow.

Pear

The first host has Proxmox installed with the following specificatio

Hardware Specification

The Main components were purchased in 2024 as follows

• The Cpu is AMD Ryzen 9 5950X Processor (16Cores/32Threads, 105W TDP, Socket AM4, 72MB Cache, Up to 4.9 GHz Max Boost, no cooler) £369.98
• The cooler is Noctua NH-D15, Premium CPU Cooler with 2x NF-A15 PWM 140mm Fans (Brown) £99.95
• Mainboard is Gigabyte B550M AORUS ELITE Motherboard - Supports AMD Ryzen 5000 Series AM4 CPUs, 5+3 Phases Pure Digital VRM, up to 4733MHz DDR4 (OC), 2xPCIe 3.0 M.2, GbE LAN, USB 3.2 Gen1 £103.48
• 128gb Ram is configured as two sets of CORSAIR VENGEANCE LPX DDR4 RAM 64GB (2x32GB) 3200MHz CL16-20-20-38 1.35V (CMK64GX4M2E3200C16) £112.20 for one set
• Power supply is Corsair RM750x 80 Plus Gold Fully Modular ATX 750 Watt Power Supply £109.00
• The case is Fractal Design Node 804 - Black - Compact Computer Case - mATX - High Airflow - Modular interior - 3x Fractal Design Silent R2 120mm Fans Included - Water-cooling ready - USB 3.0 -Window Side Panel £99.95
• SSD is Crucial P3 Plus SSD 4TB M.2 NVMe PCIe Gen4 Internal SSD, Up to 4800MB/s, Laptop & Desktop (PC) Compatible, Solid State Drive - CT4000P3PSSD801 £189.99 and £159.99
• Three 16 TB Seagate Ironwolf NAS drives make up the hard disk storage at a cost of £179.94 each total £ 539.82.

Configuration

Access

The IP Address and port of the host is 192.168.0.110:8006. I have my virgin media router forwarding all incoming traffic to 192.168.0.125 so it will go directly to the Pfsense firewall. All of the terminals can also send traffic to the WAN port of Pfsense. Edit with the change to Vodaphone the network address changed to 192.168.1.0/24 so Proxmox console is now 192.168.1.111:8006 and the WAN port of Pfsense is 192.168.1.125

Storage

The main storage is the three 16TB hard drives configure into ZFS z1 so that they have one redundant disk. As one disk is redundant, obviously, it has 32TB of storage available.

I couldn't get PCI passthrough to work reliably without significant effort and some expense so I set up ZFS on Proxmox itself or more precisely on the Debian Linux that Proxmox resides on. Unfortunately, that means that I have to do any config on the CLI. I may write some scripts to perform some of the admin tasks or more likely setup a Nginx container to automate as much of it as possible.

kiwi

kiwi is the second Proxmox host in the Homelab It is a much smaller chassis also a Fractal Design case but designed for an ITX mainboard. The main PC was bought on ebay as a cheap second hand PC with a old GPU, 32Gb ram an AMD AM4 Ryzen 5 CPU with a stock cooler and a 512gb crucial SSD. The intention was to upgrade the parts that were cheap and easy like SSD and RAM then add some big HDDs.

Hardware

Some of this will be filled in later when It is remembered so if this looks incomplete that is why.

SSD

Proxmox Installation

BIOS settings for Hypervisors (including Proxmox)

Generally most BIOS can be left at defaults and a PC as a PC will work well enough, maybe the boot priority may need to be changed from a HDD to a SSD but even that is unlikely. It is possible to optimise lots of things in BIOS and for some people that is an option but it is also easy to make mistakes that either stop the machine from working or making the machine unstable so for most people, most of the time, it is better to leave BIOS alone. A big exception is when a hypervisor is to be install because a few options that are off by default need to be set for Virtualisation to work properly. The main things to set are:-

• SVM Mode (AMD-V): Must be Enabled. This is what allows Proxmox to run VMs. Without it virtualisation will fail with most or possibly all hypervisors
• IOMMU: Set to Enabled. This is vital if you ever want to "pass through" a SATA controller or GPU directly to a VM. while not vital for the virtualisation itself most hypervisors have this enabled.
• Power Supply Idle Control: If you find the server randomly freezes when idle, set this to "Typical Current Idle". Ryzen chips sometimes drop voltage too low for some power supplies when idling, causing a crash. Again not vital but a "nice to have" if possible in the BIOS.

GPU Considerations

The Proxmox installation is a little bit more tricky than is standard for the majority of computers in that this has an old GPU that the installer does not recognise. A GPU is required so that the machine will pass POST but is not really need or used by the Proxmox guests so is generally of little benefit for the overall Proxmox day to day running. A lot of Hypervisor hosts have the cheapest and lowest power GPUs installed just enough to POST but no more than that. The reason that GPU is of low priorty for most hypervisors hosts is that they are generally run headless, that is without a monitor attached, all interations with the host is through a dedicated application like Citrix XEN, or more likely through a web browser like Proxmox. Web based GUIs are popular because it is a lot easier and lighter processor load to have a webserver attached than it is to design a whole application. Kiwi has an old Nvidia GPU that works in that it provides a display but that is about it. Unfortunately, it does give some problems for the installer as it is so old but therea re steps to overcome that as will be detailed below. We may add a bigger and better GPU at a later date but for the foreseeable future that is in the realms of an upgrade for later and dependant on inheriting a GPU from another box.

Proxmox Installer problems (The Kiwi exceptions)

As stated earlier the Proxmox installer does not work with the GPU. This is shown in that the machine boots and loads the first screen of the installer with welcome to.. but when the first option is tried it starts to load then freezes at the loading Nvidia drivers or some such thing(cant remember exactly what it says). so to fix that issue we need to restart the host and when we get to the welcome screen again we highlight "Install Proxmox VE (Graphical)" but do not press Enter instead we press "e" so a black screen with GNU GRUB appears. Edit the line

linux /boot/linux26 ro ramdisk_size=16777216 rw quiet splash=silent

so it reads

linux /boot/linux26 ro ramdisk_size=16777216 rw nomodeset

• By removing the quiet splash=silent we force the installer to show every single line of code as it loads. If it freezes again, we see the exact driver name where it stopped (e.g., sata_nv or i915).
• Adding nomodeset: This is the "magic bullet" for the nvidiafb hang. It tells the kernel not to touch the video drivers until the system is fully loaded. It is most likely where the installer hangs. As we should see every line of the execution of the installer if we have a different driver issue we should still see it.

To save the changes press ctrl+x or maybe F10. The GNU Grub screen will close and the graphical installer will continue until it gets to the "end user license agreement (EULA) screen. For anyone else reading this, If the installer freezes again before it gets to the EULA screen the error will most likely be in the last laine of text displayed. The most likely error will be a conflict between Debian kernel and Advanced Configuration and Power Interface (ACPI) in which case restart and press the "e" to get to the Grub config and edit the same line

linux /boot/linux26 ro ramdisk_size=16777216 rw nomodeset

so that it now has noapic added. note that this is just for the installer not the Proxmox application.

linux /boot/linux26 ro ramdisk_size=16777216 rw nomodeset noapic

After modifying that line press ctrl+x or F10 to save and proceed. if it stalls again you will need to look for the exact error in the last line or last few lines that the installer performed or any line that starts or ends with failed.

=Proxmox installer continued

After the earlier problems we should have got to the ELUA screen so press "I agree" button to continue. The next screen will be about choosing hard drives. Kiwi has a single SSD and three Ironwolf Pro 14 TB HDDs. Obviously, we will not install the OS onto a HDD or even all three HDDs as it would be too slow and as the SSD is fast we will choose that as the Target Harddisk