I Made Something I Built Better On Accident

I Made Something I Built Better On Accident

Introduction

When I first stumbled into the world of homelabbing, my goal was simple: experiment with new hardware, run a few containers, and maybe host a personal blog. What I didn’t anticipate was how a modest hardware upgrade would transform my entire infrastructure stack, turning a collection of isolated services into a cohesive, self‑hosted ecosystem. The turning point came when I acquired a used NUC with 16 GB of RAM at a price that matched a brand‑new memory stick. That accidental purchase forced me to rethink my Proxmox VE (PVE) deployment, leading to a more resilient, performant, and maintainable setup.

For sysadmins and DevOps engineers who manage their own infrastructure, the lesson is clear: sometimes the most valuable improvements emerge from unplanned detours. In this guide we will dissect the exact scenario that triggered the upgrade, explore the underlying technologies, and walk through a complete workflow for scaling a homelab from a single node to a multi‑node environment. Readers will learn how to evaluate hardware choices, plan node additions, configure storage and networking, and harden the system for production‑grade workloads.

Key takeaways include:

• Understanding the role of Proxmox VE as a hypervisor‑based management platform for self‑hosted services.
• Recognizing the impact of RAM constraints on virtual machine (VM) and container density.
• Learning a step‑by‑step process for integrating a second node without disrupting existing workloads.
• Applying best‑practice configurations for storage, networking, and security.
• Leveraging monitoring and backup strategies to maintain reliability.

By the end of this article you will have a reproducible blueprint for turning an accidental hardware find into a deliberate, scalable infrastructure improvement.

Understanding the Topic

What is Proxmox VE?

Proxmox VE is an open‑source virtualization platform that combines KVM hypervisors, LXC containers, and software‑defined storage behind a web‑based management interface. It is purpose‑built for homelab and small‑to‑medium enterprise environments where administrators need granular control over VMs, containers, and storage without the overhead of a full‑blown hyper‑converged solution.

Key capabilities include:

• KVM virtualization for full‑featured VMs with support for virtio drivers, nested virtualization, and GPU passthrough.
• LXC containers for lightweight, OS‑level isolation ideal for microservices and CI/CD pipelines.
• Software‑defined storage via Ceph, ZFS, or traditional LVM, enabling thin provisioning, snapshots, and replication.
• Integrated networking with software bridges, VLAN tagging, and bonding for complex network topologies.
• Web UI and API for automated provisioning, making it a natural fit for DevOps pipelines.

Historical Context

Proxmox emerged in 2008 as a community‑driven fork of the older PVE platform, aiming to provide a free, open‑source alternative to commercial hypervisors. Over the years, it has matured from a simple VM manager to a full‑featured orchestration layer, incorporating features such as HA (High Availability), clustering, and live migration. The release of version 7 introduced a redesigned web UI and improved clustering capabilities, making it increasingly attractive for self‑hosted environments.

Core Features and Capabilities

• Clustering: Nodes can be grouped into a single logical cluster, allowing VMs and containers to be moved across hosts without downtime.
• HA Resources: Automated failover for critical VMs using fencing and resource groups.
• Backup Engine: Built‑in snapshot‑based backup for both VMs and containers, with remote storage support.
• Role‑Based Access Control (RBAC): Fine‑grained permissions for multi‑user environments.
• Extensive API: RESTful API enables integration with external orchestration tools (e.g., Ansible, Terraform).

Pros and Cons

Pros	Cons
Open‑source with active community support	GUI can feel cluttered for newcomers
Combines KVM and LXC in a single platform	Requires a dedicated management node for large clusters
Rich set of storage backend options	Some advanced features (e.g., Ceph) need separate licensing for enterprise use
Robust HA and backup mechanisms	Performance of certain storage backends may lag behind dedicated solutions

Use Cases and Scenarios

• Development sandboxes: Spin up disposable VMs for testing new OS versions or language runtimes.
• Home‑automation hubs: Host Home Assistant, Zigbee2MQTT, or Node‑RED in isolated containers.
• CI/CD pipelines: Run GitLab Runner or Jenkins within VMs, leveraging snapshots for clean environments.
• Media servers: Deploy Plex or Jellyfin on dedicated VMs with GPU passthrough for hardware‑accelerated transcoding.

Current State and Future Trends

The Proxmox community continues to push forward with improvements to the web UI, better integration with OpenStack, and enhanced support for ARM‑based hardware. The rise of edge computing is also driving interest in lightweight, highly available clusters that can be deployed in remote locations. As hardware becomes more affordable, the barrier to entry for multi‑node homelabs is lower than ever, encouraging more engineers to experiment with self‑hosted infrastructure.

Comparison to Alternatives

• VMware ESXi: Enterprise‑grade, but proprietary and cost‑prohibitive for most hobbyists.
• Hyper‑Kit (macOS): Limited to Apple hardware, not applicable to generic x86 servers.
• Kubernetes on bare metal: Offers container orchestration at scale, but introduces complexity that may be unnecessary for small homelabs.
• XenServer: Mature open source, but lacks the integrated LXC container support that Proxmox provides.

Overall, Proxmox VE strikes a balance between flexibility and simplicity, making it a strong candidate for engineers who want to experiment with both VMs and containers under a single management umbrella.

Prerequisites

Before embarking on a node expansion, verify that your environment meets the following requirements.

Hardware Requirements

Component	Minimum Spec	Recommended Spec
CPU	2 cores (Intel Xeon E3 or AMD EPYC 730)	4 cores or more, VT‑x/AMD‑V support
RAM	8 GB	16 GB or more (to accommodate additional VMs)
Storage	1 × 120 GB SSD (OS) + 1 × 1 TB HDD (data)	2 × 500 GB SSD (RAID‑1 for OS) + multiple HDDs for VM images
Network	1 GbE NIC	Dual‑port NIC for redundancy or bonding
Motherboard	BIOS/UEFI with virtualization extensions enabled	Latest firmware for stability

The used NUC referenced in the original Reddit post provides a compact form factor, low power consumption, and sufficient CPU cores for light VM workloads. However, the RAM limitation (initially 8 GB) prompted the upgrade to a 16 GB configuration, which directly enabled the addition of a second node without sacrificing performance.

Software Requirements

• Operating System: Debian‑based Proxmox VE 7.x (or newer).
• Packages: pve-manager, pve-kernel, pve-storage, pve-backup.
• Dependencies: lvm2, zfsutils-linux (if using ZFS), ceph (optional for distributed storage).
• Network Configuration: Static IP address for each node, with proper DNS resolution.

Network and Security Considerations

• Static IP Allocation: Use DHCP reservations or manually assign IPs within a private subnet (e.g., 192.168.10.0/24).
• Firewall Rules: Allow only necessary ports (8006 for the web UI, 22 for SSH, 3000‑3100 for VM traffic).
• SSH Hardening: Disable root login, enforce key‑based authentication, and consider fail2ban for intrusion prevention.
• TLS Certificates: Generate self‑signed or use Let’s Encrypt certificates for the web UI to prevent plaintext transmission.

User Permissions

• Root Access: Required for installing Proxmox and managing cluster resources.
• Administrative Users: Create non‑root users with sudo privileges for day‑to‑day operations.
• RBAC Roles: Define custom roles if you need to delegate specific tasks (e.g., backup operator).

Pre‑Installation Checklist

1. Verify BIOS virtualization settings are enabled.
2. Confirm that the system boots in UEFI mode (recommended for secure boot).
3. Back up existing data and create a recovery image of the current PVE installation.
4. Document current network configuration, including MAC addresses and VLAN tags.
5. Ensure you have an external backup destination (e.g., NFS share, external HDD).

Installation & Setup

With the prerequisites satisfied, the next phase involves adding the second node to the existing Proxmox cluster. The process can be broken down into three major stages: initial node configuration, cluster formation, and verification.

1. Prepare the New NUC

1. Install Proxmox VE on the new hardware using the official ISO image.
2. During installation, select the “All‑in‑One” partitioning scheme for simplicity, then create a dedicated ZFS pool for VM storage (e.g., rpool/data).
3. Assign a static IP address, for example 192.168.10.2, and configure the hostname as pve-node2.

# Example: Set static IP via /etc/network/interfaces
auto eno1
iface eno1 inet static
    address 192.168.10.2/24
    gateway 192.168.10.1
    dns-nameservers 8.8.8.8 8.8.4.4

1. Update the system to the latest packages:

apt update && apt upgrade -y

1. Install the Proxmox VE repository packages if they were not included during the base installation:

pveupgrade me

2. Join the New Node to the Existing Cluster

From the original node (e.g., pve-node1), execute the following command to generate a cluster token:

pvecm add pve-node2

The command will output a token and a join command. Copy the join command and run it on the new node:

pvecm add <token>

After successful execution, both nodes will appear in the Cluster view of the web UI.

3. Configure Shared Storage (Optional but Recommended)

If you plan to store VM images on a common pool, consider using ZFS replication or NFS. Below is an example of creating an NFS export on a dedicated storage server and mounting it on both nodes.

Storage Server (storage01)

# Create export directory
mkdir -p /export/pve-nfs
# Export for both nodes
echo "/export/pve-nfs *(rw,sync,no_subtree_check,no_root_squash)" >> /etc/exports
exportfs -ra

Both Nodes

# Install NFS client
apt install -y nfs-common

# Mount the export
mkdir -p /mnt/pve-nfs
mount -t nfs 192.168.10.10:/export/pve-nfs /mnt/pve-nfs
# Add to /etc/fstab for persistence
echo "192.168.10.10:/export/pve-nfs /mnt/pve-nfs nfs defaults 0 0" >> /etc/fstab

After mounting, create a Proxmox storage entry:

pvesh create -type storage -storage nfs01 -path /mnt/pve-nfs -storage-type nfs -content images

4. Verify Cluster Health

Navigate to the Cluster tab in the web UI. You should see both nodes listed with a green “Online” status. Perform a quick health check:

• Ping each node from the other to confirm network connectivity.
• Check that the pve-cluster service is running on both hosts:

systemctl status pve-cluster

• Validate that the pve-no-quorum fencing agent is active (important for HA setups).

5. Common Installation Pitfalls and Solutions

Issue	Symptom	Solution
Node fails to join cluster	“Authentication failed” error	Ensure both nodes share the same pve-ssl-key.pem and that the firewall allows port 8006.
VMs cannot start	“No storage available”