2026 Proxmox Servers
📋Introduction
After some time passed and we decided to pursue the AI technology trend we decided that the original server layout was not well suited to the new and changed role. Pear was a general purpose Proxmox host with a fast CPU and Maximum Ram and three 16 TB Ironwolf Pro HDDs making a great system for a Home Lab with only one host.
Changing roles
As time passed the quantity of virtual machines increased the host became bogged down with low demand always on services. Most of the services were interdependent and did not need alot of compute or ram but did need to be always on, for example the webservers needed to have the reverse proxy and also needed the mail server for alerting and the certbot to keep SSL certs up to date none of these things need to have alot of ram or compute but did need to be always on. As we moved research more towards AI and RAG DB creation these always on services were taking a bit of the available compute from AI compute intensive tasks.
Reassessing the Requirements
Eventually we decided to consolidate and redistribute the components to better able to perform the roles that were needed. Orchard(the new name for Pear) did not really need a 5060ti or a 5950x to run webservers and media server etc. However, all hosts will need to have some sort of GPU to POST so the choices were to get a really low power entry level GPU like a GT750 that would not draw alot of energy and would be really cheap to buy or buy something a bit faster. As it transpired a low end GT750 was not much cheaper than a old workstation GPU so we bought a circa 2017-18 P4000 GPU, which in turn gives some low end AI/GPU/transcoding capabilities.
We decided that the 16 cores of the 5950x in orchard were being under utilised and that a 8 core 5700x would be more than sufficient for the low power requirements of the webservers etc on Orchard. We decided that the power requirement for a Ryzen 9 CPU would be too high and th4e cooling would be too much, we did consider that we could get a Ryzen 9 and run it on eco mode so that we could then have a smaller cooler and reduce the power consumption, sadly the cheapest of the Ryzen 9s is still quite expensive. The relative costs meant that the easiest and cheapest method to obtain a 5700x and a Matx motherboard was to get it as a combo deal, so we bought a Asus prime B550 matx and a 5700x was a bundle deal.
RAM is still exceptionally expensive compared to what it was 18 months ago so the best solution was to redistribute the ram that we have. Fortunately, we had 96gb of RAM in Grape and 128GB in Orchard and another 64GB in Kiwi and finally 32GB unused that we were considering selling. RAM is the main resource that we run out of first so previously we bought alot and mainly as higher capacity sticks. In theory DDR4 will be unused in a few years and only available on Ebay but with the AI boom time that is happening in 2025- 2027? DDR4 is still in demand and still in production in a small scale. We did assume that as it went out of production and was only on Ebay there would be a massive over supply of 8 and 16 gb sticks so they would remain cheap, we assumed that the 32GB sticks would be a lot more scarce so would command a higher price. As things have happened the assumptions have been true with a few caveats, the first oddity was that DDR4 has come back into production and it can be bought new, the next thing is that DDR4 is still alot more expensive than it should be even 16gb sticks are expensive to the extent that 64gb kit that was bought in May 2025 for £85 is now over £200. DDR5 Ram has not come down in price either even though it should have been in increased production by middle of 2025 at the latest.
The odd RAM price spikes have meant that we must be a bit more careful on how we use the RAM we have so as to avoid buying any extra DDR4 and certainly avoid DDR5 altogether. As we had two 32gb kits and 4 64gb kits in total we have used 128 in Orchard and 128 in the new host Arbour we now have 32gb in Grape and 32 GB in Kiwi. This redistribution of ram should keep the various hosts in the best possible shape without having to buy any extra.
One point that is often missed in any home lab environment is the server sprawl. Often home labs continue to buy more and more equipment and then struggle to provide the power to them. If we do the same in a business environment there is less of a problem in that server rooms generally have over supply of electricity, having said that the AI revolution will have bee rewriting the rules on server room power requirements to a degree. In a domestic environment there is a hard wall of max power that the domestic supply can deliver. To this end we will limit the server count to two full sized server hosts and only one will have a large scale GPU.
The New Topology
Now that we have decided what we want to do with the Home Lab we can streamline the topology.
🍎Orchard Reformed as Webserver and NAS
We will have a lower powered Orchard and a higher powered compute oriented host, Arbour and maybe some small low power mini PCs that draw 20 or 30 watts total. We keep all of the low compute always on on Orchard and the Ironwolf Z1 array of hard drives but we change the motherboard and CPU to the Asus prime and 5700x bundle. to make this happen we can simply move the new Motherboard on to the 804 case and swap the Crucial P3 M.2 NVMEs to the Asus board. It may sound like the swap over the boot drives to the new hardware should be a guaranteed fail but in reality modern Linux Distros actually do a great job of abstracting the hardware so the swap worked without incident. We didn't want to use the 5060ti GPU in the lower powered Orchard so we used the P4000.
| component | Manufacturer | Model | Notes |
|---|---|---|---|
| Case | Fractal Design | 804 | |
| Motherboard | Asus | Prime B550 | MATX |
| CPU | AMD | Ryzen 7 5700x | |
| RAM | Corsair | Two 64GB kits | 128GB total |
| GPU | Nvidia | Quadro P4000 | |
| M.2 | Crucial | P3 4TB | Mirrored Pair |
| Storage | Lexar | SATA SSD 1TB | Cheap sacrificial scratch disk |
| Storage | Seagate | IronWolf Pro 16 TB | 3 drives as ZFS Z1 |
| NIC | Realtec | 2.5 GB p/s | cheap 2.5gb PCI NIC |
There are a few special considerations for Orchard to make the NAS work reliably.
- The Orchardpool is made available to the virtual machine through NFS shares. There are a few features to the NFS exports
- As soon as orchard exposes NFS to the VMs VLANs we also expose the management interface to the same VLANs and therefore VMs so to mitigate we use the host firewall to only allow the NFS traffic through the exposed IP address.
- To make sure we dont get locked out we make a rule in the Datacenter Firewall to allow 22 and 8006. Proxmox firewall has a hierarchy in the order that the rules are applied, Datacenter rules are read and applied first if no rule is found the host is cheZcked and if no rule is there the individual VMs are checked. So by adding the allow rule for 8006 and 22 at the data center level it should be applied first however if the host is ever split from the cluster the top level rules will be for the host so the same allow rules should also be added to the host.
- The VMs get to see the shares as shares but not the actual underlying directories that the shares are based on. This means that if we have have a snapshot created on orchard for a directory that we then share to Nextcloud, for example, Nextcloud can modify all of those files that NFS exposes but can never access the snapshot data and if the Nextcloud files were deleted they could easily be restored by the Orchard user via the ZFS snapshots.
- As soon as orchard exposes NFS to the VMs VLANs we also expose the management interface to the same VLANs and therefore VMs so to mitigate we use the host firewall to only allow the NFS traffic through the exposed IP address.
- We have installed a cheap SATA SSD with ext4 so that is can use is as a sacrificial fast disk. Mainly so as to reduce the wear of the primary M.2 NVME drives.
- One point is that it is purposely formatted as ext4 and not ZFS because we want no write amplification or RAM requirement for ARC. So as simple and fast as possible and as it is only a scratch disk there is no need to protect the data integrity.
- The first use of the sacrificial disk is as a swapfile for the host to make it as fast as possible there are two 64 GB swapfiles. We know that many forums and posts dislike the very idea of a swap file but mostly it is a misguided lack of understanding of zfs get -r secondarycache orchardpool/PFPhow different storage works.
- We also partition about half of the SSD to be used by Orchard as a temporary cache / fast scratch disk but most of that will probably be used as swap files for the VMs. Again swapfiles have a place in modern OS despite what current trends suggest, at the very least they make a great OOM killer protection.
- A more esoteric use of the SSD is to create a L2ARC. Ok so L2ARC is not so uncommon and most home labs avoid it because it uses too much RAM for it's index. However, we will make this L2ARC a bit special and only cache the metadata with the commands
zpool add orchardpool cache /dev/sdb2 zfs set secondarycache=metadata orchardpool zfs set secondarycache=metadata orchardpool/PFP
The first command adds the cache to orchardpool using sdb2 and the next two commands make it so that only metadata is cached. The reason for two commands it to make it propagate through the dataset. If we want to be sure that the zpool metadata is being cached we use the command
zfs get -r secondarycache orchardpool/PFP
For a home lab this is an ideal compromise to creating a full L2ARC or writing the metadata exclusively to SSD. A L2ARC that caches all files is discouraged on a home server especially if it is sharing RAM with other tasks, because as each block of data is cached ZFS must index it in ram and it will use a large amount of RAM to index a one TB L2ARC. The other idea of writing all metadata to a SSD is impractical because if the SSD is every corrupted or deleted the entire Zpool will be destroyed because the metadata is the index or map to the entire Zpool. Enterprise may well keep all metadata on fast NVME but they will mirror it and use high reliability NVME, sadly that is a bit too much to spend for the average home lab. By making it a L2ARC it is a read cache only so if the cache is lost or destroyed ZFS will just go back to reading the metadata from the data disks. To be sure an L2ARC will only help for read operations and will have no effect on write operations at all but it will also never risk losing data.
🪵 Arbour the new AI engine
With the 5950x and its Aorus Elite MB now spare we bought a second hand Fractal Design Meshify C case and a Be quiet PSU. The only spare M.2 was the 512gb one that we took out of Kiwi but as that was small and not a high end model we decided to check the availability on ebay and as we found a reasonably cheap enterprise one TB model we are using that along with a enterprise Samsung PM883 one TB SATA SSD and a another at two TB. We also added the Nvidia 5060ti 16gb GPU to make this the best candidate for the AI processing jobs.
| component | Manufacturer | Model | Notes |
|---|---|---|---|
| Case | Fractal Design | Meshify C | big enough for full ATX |
| Motherboard | Gigabyte | B550M AORUS ELITE | MATX |
| CPU | AMD | Ryzen 5950x | |
| Cooler | Noctua | NH-D15 | Premium CPU Cooler with 2x NF-A15 PWM 140mm Fans (Brown) |
| RAM | Corsair | Two 64GB kits | 128GB total |
| GPU | Gigabyte | Geforce RTX 5060 TI 16gb | Twin Fan |
| M.2 | Samsung | 1 tb | Enterprise model size is 110 mm |
| NIC | Intel | 2.5 GB p/s | to be delvered |
| Storage | Samsung | pm883 SATA SSD 1TB | sacrificial enterprise scratch disk |
| Storage | Samsung | pm883 SATA SSD 2TB | Enterprise disk |
Like Orchard we will also have a sacrificial scratch disk with ext4 format and like orchard we create an pair of swapfiles but this time set to 32GB each for a total swap file of 64GB. Both swapfiles are set to the priority of 100 so they should both be used equally (when an equal positive priority is set they are treated as a RAID 0) We have also set the swapiness to 80 so they should be used quite a lot. Many people advise against swap files or setting swapiness to 10 so they are hardly ever used but we feel this is old redundant advice.
As there is no Ironwolf ZFS array allocated on Arbour we do not use any part of the scratch disk for L2ARC. However, we are using Arbour for the AI related tasks so we will need fast SSD for temporary files to be held locally for processing, to that end we have created a NFS shares to both the SSDs.
Kiwi to be retired and sold
The small ITX host Kiwi with the Ryzen 5 CPU and the 3 x 14 TB Iron wolf can be decommissioned and sold. we have decided that it is not fast enough to be especially useful as a compute engine and does not have enough capacity to be a backup server. It was a good experiment but has proved to be not quite big and powerful enough bt at the same time too big to be host that can be used for odd jobs. So basically not big enough and not small enough. We will copy the OpenAlex data to the Orchardpool and remove the HDs we will also swap the 32GB ram back and swap the original 512 M.2 To add some extra value before it is sold we will add a spare 1 TB SATA SSD to make it into a much more useful Windows 11 Pro workstation.
🌿 Bower the third member of the Cluster
Proxmox clusters need to be able to reach a quorum before they can function properly so if there are only two members of a cluster if one goes offline the remaining host will not be able have a majority of the cluster because it will only be 50% of the membership and 50% is not a majority. To make the cluster work properly the usual membership is at least 3 members, in which case if any one host goes offline the remaining two will have two thirds of the votes and in so doing will still be able to reach a quorum. If we are to have a cluster of Orchard and Arbour we need a third member and Bower is a old mini PC that can fulfil that role. we upgraded the ram to the maximum of 8GB some time ago but then the HD stopped working so we added an old spinning 2.5 inch SATA HD to make it work. It is not very fast but it can run a standby Adguard LXC and shoukld be able to run a metrics monitoring solution like Victoria metrics and Grafana but that is about the limit and we will need to test how well Victoria Metrics runs in practice.
🌳 Copse the newer mini PC
We found a new mini PC on ebay that runs a mobile equivalent to a Ryzen 5. it has two slots for DDR5 Sodimm RAM and two slots for 80mm M.2 NVMEs. If we can get some DDR5 RAM we will be able to use the 2 TB Samsung 990 M.2 drive and have this run as a low power always on Proxmox host.
All of these mini PCs are essentially stripped down Laptops in a small form case. From an engineering viewpoint these make a lot of sense. The original design would be a laptop bt the keyboard, screen, battery and any other laptop specific components are removed and the new case is made. an additional advantage is that as they do not now need to fit in the ultra thin case they can change the cooler for a simpler design with a cheaper fan etc. The biggest drawback is likely to be that when they are under load the cheap cooler design could well be a bit louder than expected, if this turns out to be the case with this model we will look at replacing the internal fan and/or increasing the airflow with external fan.
| !component | Manufacturer | Model | Notes |
|---|---|---|---|
| Case | GMKtec | M6 | |
| CPU | AMD | Ryzen 5 6600H | |
| RAM | integral | 16gb | |
| GPU | AMD | CPU integrated | |
| M.2 | Samsung | 2 TB 990 | |
| M.2 | empty | empty |
Stitching the cluster together
The final component needed to bring the cluster together is a new 2.5 GB managed switch. We found a cheap Sodola managed switch SL902-SWTGW218AS on Amazon, with 8 x 2.5GB p/s ports and a 10gb SFP . It appears to work ok with some odd caveats.
- VLAN 1 cannot be removed
- The management interface appears to listen on all VLANs
- VLAN 1 appears to be a bit leaky so best not to use it all
- The terminology used on the switch is a bit odd so needs a bit of decoding
- The default IP address is 192.168.2.1
- default login is admin / admin
We have set the device name to Hyphae and the IP address to 192.168.1.167
| Port | Host | Port Type | VLAN 1 | VLAN 10 | VLAN 88 | VLAN 100 | VLAN 99 ,110, 111, 120, 130 |
|---|---|---|---|---|---|---|---|
| 1 | 🍎 Orchard vmbr0 | Trunk | N/A | Trunk | Trunk | N/A | N/A |
| 2 | 🪵 Arbour vmbr0 | Trunk | N/A | N/A | Trunk | Trunk | Trunk |
| 3 | 🌿Bower vmbr0 | Trunk | N/A | N/A | Trunk | Trunk | Trunk |
| 4 | 🌳 Copse vmbr0 | Trunk | N/A | N/A | Trunk | Trunk | Trunk |
| 5 | 🍎 Orchard vmbr1 | Trunk | N/A | N/A | N/A | Trunk | Trunk |
| 6 | Grove vmbr0 | Trunk | N/A | N/A | Trunk | Trunk | Trunk |
| 7 | 🍇Grape | Access | N/A | N/A | N/A | Access VLAN | N/A |
| 8 | 🐌dumb switch | Trunk | native | trunk | trunk | N/A | N/A |
More details can be found on the Network Configuration page in the switch config section Here
Expected Virtual Machine Allocation
While any allocation of VMs to specific hosts is going to be subject to some variation and in a enterprise cluster may not be set so that the HA will work properly, in our case the Silvan cluster is not going to be able to do the full HA. We should be able to move some vms from one host to another but some will not make much sense for example Quince will have the Nvidia 5060 ti GPU assigned as a PCI passthrough. In general most of the VMs will likely stay on the same hosts for long periods so tret the following table as a guide to what VM goes where.
| 🍎 Orchard | 🪵 Arbour | |||||||
| ID | Name | Purpose | Comment | ID | Name | Purpose | Comment | |
| 109 | Outns1 | Adguard | External DHCP/DNS for 192.168.1.0/24 | 308 | Quince | Docker AI Apps | Ollama to VM, Docker AI APPS and 5060 GPU | |
| 110 | ctns1 | Internal DNS and DNS cache | Resolves local IP addresses to Silvan VMs | 312 | Peach | General Purpose Server | Various Light weight Scripts | |
| 105 | Pfsence | Firewall | Firewall for the Silvan Cluster | 340 | Elderberry | Linux desktop | Desktop on arbour | |
| 115 | MySQLServer | MySQL DB server | Hostname Manderin | 405 | Debian13 | Server Template | Comment | |
| 120 | ReverseProxy | Nginx Reverse Proxy | Hostname Raisin | 406 | cinnamon22.3 | Desktop Template | Comment | |
| 128 | mango | Metrics monitoring | Comment | |||||
| 130 | mgtConsole | Old management console | mgt console on 99 now retired | |||||
| 131 | Cinnamon | New MGT console | The new console for silvan management on 99 | |||||
| 220 | L3-Plum | Webeserver Email relay | Apache and Postfix | 🌿Bower | ||||
| 224 | FigCloud | Webserver | Nextcloud | ID | Name | Purpose | Comment | |
| 230 | Juneberry | Linux desktop Jellyfin | nomachine, P4000 PCI Passthrough | 108 | bowoutns1 | Adguard | duplicate DHCP/DNS for 192.168.1.0/24 | |
| 240 | Cherry | Minecraft | Vanilla Minecraft | |||||
| 241 | Apple | Minecraft | FTB University | 🌳 Copse | ||||
| 303 | Voavanga | OpenVPN server | Comment | ID | Name | Purpose | Comment | |
| 304 | Vanilla | Wireguard Server | Comment | |||||
| 306 | Wahoo | Win11 host | RDP host through WG only | |||||
| 310 | Blackberry | Docker Apps | Mostly archive Applications | |||||
| 311 | Strawberry | Samba server | Provides SMB shares to NFS shares | |||||
| 400 | DebianBase13 | Server Template | ||||||
| 403 | Mint-22.3-Cinnamon | Desktop Template | ||||||
These are only probable virtual machine and should not be relied on. The allocations do give an indication of the sort of loading to expect.
