Every Self-Hosting Setup Ever

Every Self-Hosting Setup Ever

Introduction

If you’ve spent any time in the homelab or DevOps community, you’ve likely encountered the infamous “It’s not DNS” meme. This phrase has become the battle cry of sysadmins worldwide, representing the universal truth that DNS issues are often the culprit behind mysterious network problems. In this comprehensive guide, we’ll explore every self-hosting setup ever—from the humble Raspberry Pi homelab to enterprise-grade infrastructure—while addressing the DNS-related challenges that plague even the most experienced DevOps engineers.

Self-hosting has exploded in popularity over the past decade, driven by the desire for data privacy, cost savings, and the satisfaction of building your own infrastructure. Whether you’re running a personal media server, a development environment, or a full-scale production system, understanding the fundamentals of self-hosting is crucial. This guide will walk you through the entire spectrum of self-hosting setups, providing you with the knowledge and tools to tackle any infrastructure challenge—including those pesky DNS issues that seem to crop up at the most inconvenient times.

Understanding Self-Hosting Setups

Self-hosting refers to the practice of running and maintaining your own servers, applications, and services rather than relying on third-party providers. This approach gives you complete control over your data, infrastructure, and workflows. Self-hosting setups can range from simple single-machine configurations to complex distributed systems spanning multiple data centers.

The Evolution of Self-Hosting

The concept of self-hosting has evolved significantly since the early days of computing. What started with physical servers in data centers has transformed into a diverse ecosystem of virtualization, containerization, and cloud-native technologies. Today’s self-hosting landscape includes:

• Traditional bare-metal servers
• Virtual machines (VMs) using hypervisors like Proxmox, VMware, or Hyper-V
• Container-based deployments using Docker and Kubernetes
• Edge computing setups for IoT and distributed applications
• Hybrid cloud solutions combining on-premises and cloud resources

Key Components of Every Self-Hosting Setup

Regardless of the specific implementation, every self-hosting setup shares several fundamental components:

1. Hardware Infrastructure: The physical or virtual machines that run your services
2. Operating System: The foundation that manages hardware resources and provides a platform for applications
3. Networking: The connectivity layer that enables communication between services and with the outside world
4. Storage: The persistent data layer for applications and user data
5. Security: The mechanisms that protect your infrastructure from unauthorized access
6. Monitoring and Logging: The observability tools that help you understand system behavior

Common Self-Hosting Scenarios

Let’s explore the most common self-hosting setups you’ll encounter:

The Beginner Homelab

This setup typically involves a single machine running multiple services. Popular choices include:

• Raspberry Pi clusters for lightweight services
• Old desktop computers repurposed as servers
• Mini-PCs like Intel NUC or ASRock Beebox
• Single-board computers with various capabilities

The Intermediate Setup

As your needs grow, you might upgrade to more robust configurations:

• Dedicated server hardware with multiple cores and RAM
• Virtualization platforms like Proxmox or VMware ESXi
• Network-attached storage (NAS) for centralized file storage
• Managed networking equipment with VLAN support

The Advanced Infrastructure

For those requiring enterprise-grade capabilities:

• Multiple physical servers in a cluster
• Container orchestration platforms like Kubernetes
• Software-defined networking and storage
• High-availability configurations with failover capabilities
• Comprehensive monitoring and alerting systems

Prerequisites for Self-Hosting

Before diving into self-hosting, you need to understand the prerequisites for different setups. The requirements vary significantly based on your chosen architecture and intended use cases.

Hardware Requirements

Entry-Level Setup

• CPU: Dual-core processor (1.5+ GHz)
• RAM: 2-4 GB DDR3/DDR4
• Storage: 32-64 GB SSD or 120-250 GB HDD
• Network: Gigabit Ethernet
• Power: Standard 12V/5V power supply

Mid-Range Setup

• CPU: Quad-core processor (2.5+ GHz)
• RAM: 8-16 GB DDR4
• Storage: 256 GB SSD + 1-2 TB HDD
• Network: Gigabit Ethernet with VLAN support
• Power: 80+ efficiency power supply

Enterprise Setup

• CPU: Multi-core Xeon or equivalent (8+ cores)
• RAM: 32-128 GB ECC DDR4
• Storage: Multiple SSDs in RAID with large-capacity HDDs
• Network: 10 Gigabit Ethernet with redundant connections
• Power: Redundant power supplies with UPS

Software Requirements

Operating Systems

• Linux Distributions: Ubuntu Server, Debian, CentOS, Fedora
• BSD Variants: FreeBSD, OpenBSD for specific use cases
• Windows Server: For .NET applications or specific enterprise needs

Virtualization Platforms

• Proxmox VE: Open-source virtualization with web interface
• VMware ESXi: Enterprise virtualization platform
• KVM/QEMU: Linux-based virtualization
• Hyper-V: Microsoft’s virtualization solution

Container Platforms

• Docker: Container runtime and orchestration
• Kubernetes: Container orchestration for complex deployments
• Podman: Docker alternative with different architecture

Network Requirements

Every self-hosting setup requires careful network planning:

• Static IP Addresses: For services that need consistent addressing
• Port Forwarding: For external access to services
• DNS Configuration: For domain name resolution
• Firewall Rules: For security and access control
• VPN Access: For secure remote management

Installation and Setup

The installation process varies significantly based on your chosen setup. Let’s walk through the most common scenarios.

Setting Up a Basic Homelab with Proxmox

Proxmox is an excellent choice for beginners and intermediate users due to its user-friendly web interface and robust feature set.

# Download Proxmox ISO from official website
# Create bootable USB drive
# Boot from USB and install Proxmox

# After installation, access web interface at https://your-proxmox-ip:8006
# Default credentials: root / your-installation-password

# First steps after login:
# 1. Update system packages
apt update && apt upgrade -y

# 2. Configure network settings
# 3. Set up storage (local-lvm, NFS, etc.)
# 4. Create your first VM or LXC container

Installing Docker for Container-Based Services

Docker has become the de facto standard for container deployments.

# Install Docker on Ubuntu/Debian
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

# Add user to docker group for non-root access
sudo usermod -aG docker $USER

# Install Docker Compose for multi-container applications
sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

# Verify installation
docker --version
docker-compose --version

Kubernetes Setup for Advanced Users

Kubernetes provides powerful orchestration capabilities for complex deployments.

# Install Kubernetes on Ubuntu using kubeadm
# Install container runtime (Docker or containerd)
sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl

# Add Kubernetes signing key
sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg

# Add Kubernetes repository
echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list

# Install kubelet, kubeadm, and kubectl
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

# Initialize Kubernetes cluster
sudo kubeadm init --pod-network-cidr=10.244.0.0/16

# Configure kubectl for your user
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

# Install CNI plugin (Flannel)
kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml

Configuration and Optimization

Proper configuration is essential for reliable, performant self-hosting setups.

Network Configuration Best Practices

# Example network configuration for Proxmox
network:
  version: 2
  renderer: networkd
  ethernets:
    eth0:
      dhcp4: false
      addresses:
        - 192.168.1.10/24
      gateway4: 192.168.1.1
      nameservers:
        addresses: [8.8.8.8, 1.1.1.1]
  vlans:
    vlan100:
      id: 100
      link: eth0
      addresses:
        - 10.0.0.10/24

Docker Compose Configuration

version: '3.8'

services:
  nginx:
    image: nginx:latest
    container_name: nginx_reverse_proxy
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
      - ./ssl:/etc/nginx/ssl
    restart: unless-stopped
    networks:
      - proxy

  portainer:
    image: portainer/portainer-ce:latest
    container_name: portainer
    ports:
      - "9000:9000"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - portainer_data:/data
    restart: unless-stopped
    networks:
      - management

networks:
  proxy:
    driver: bridge
  management:
    driver: bridge

volumes:
  portainer_data:

Kubernetes Configuration

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.21
        ports:
        - containerPort: 80
        resources:
          requests:
            memory: "64Mi"
            cpu: "50m"
          limits:
            memory: "128Mi"
            cpu: "100m"

Usage and Operations

Once your self-hosting setup is configured, you need to understand how to operate and maintain it effectively.

Daily Operations Commands

# Proxmox management
# List all VMs and containers
pct list
qm list

# Start/stop operations
pct start 100
pct stop 100
qm start 101
qm shutdown 101

# Docker management
# List running containers
docker ps

# View container logs
docker logs $CONTAINER_ID

# Execute commands in containers
docker exec -it $CONTAINER_NAME bash

# Kubernetes operations
# Check cluster status
kubectl get nodes
kubectl get pods --all-namespaces

# Deploy applications
kubectl apply -f deployment.yaml

# Scale applications
kubectl scale deployment nginx-deployment --replicas=5

Monitoring and Logging

Effective monitoring is crucial for maintaining healthy self-hosted services.

# Prometheus configuration for monitoring
global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'node-exporter'
    static_configs:
      - targets: ['node-exporter:9100']

  - job_name: 'cadvisor'
    static_configs:
      - targets: ['cadvisor:8080']

  - job_name: 'proxmox'
    static_configs:
      - targets: ['proxmox-exporter:9221']

Backup Strategies

Regular backups are essential for data protection.

# Proxmox VM backup script
#!/bin/bash

BACKUP_DIR="/mnt/backups"
DATE=$(date +%Y%m%d_%H%M%S)
VM_ID=$1

if [ -z "$VM_ID" ]; then
    echo "Usage: $0 <VM_ID>"
    exit 1
fi

echo "Starting backup for VM $VM_ID at $DATE"
pct backup $VM_ID $BACKUP_DIR/vm-$VM_ID-$DATE.tar.zst --remove=1
echo "Backup completed successfully"

Troubleshooting Common Issues

Even with proper setup, issues will arise. Here are solutions to common problems.

DNS Issues

DNS problems are the most common self-hosting headaches.

# DNS troubleshooting commands
# Check DNS resolution
nslookup google.com
dig google.com

# Test specific DNS server
nslookup google.com 8.8.8.8

# Check local DNS configuration
cat /etc/resolv.conf

# Restart DNS resolver
sudo systemctl restart systemd-resolved

# Common DNS fixes
# Flush DNS cache
sudo systemd-resolve --flush-caches

# Check for DNS leaks
nslookup whoami.akamai.net

Network Connectivity Issues

Network problems can bring down your entire setup.

# Network troubleshooting
# Test basic connectivity
ping -c 4 google.com

# Trace route to identify bottlenecks
traceroute google.com

# Check port connectivity
nc -zv google.com 443

# Restart network services
sudo systemctl restart networking
sudo systemctl restart NetworkManager

Container and Service Issues

When services fail, you need quick diagnostic tools.

# Docker troubleshooting
# Check container status and logs
docker ps -a
docker logs $CONTAINER_ID --tail=100

# Inspect container configuration
docker inspect $CONTAINER_NAME

# Check resource usage
docker stats

# Kubernetes troubleshooting
# Check pod status
kubectl get pods -o wide

# Describe pod for detailed information
kubectl describe pod $POD_NAME

# Check pod logs
kubectl logs $POD_NAME

Advanced Topics

For those ready to take their self-hosting to the next level, consider these advanced topics.

High Availability Configuration

# Keepalived configuration for HA
global_defs {
   notification_email {
     admin@example.com
   }
   notification_email_from noreply@example.com
   smtp_server 192.168.1.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass password
    }
    virtual_ipaddress {
        192.168.1.100
    }
}

Security Hardening

# Firewall configuration with UFW
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow ssh
sudo ufw allow http
sudo ufw allow https
sudo ufw enable

# Fail2ban configuration
sudo apt install fail2ban
sudo systemctl enable fail2ban
sudo systemctl start fail2ban

# SSH hardening
sudo nano /etc/ssh/sshd_config
# Change: Port 22 -> Port 2222
# Change: PermitRootLogin yes -> PermitRootLogin no
# Add: AllowUsers your-username
sudo systemctl restart sshd

Automation with Ansible

---
- name: Configure self-hosting infrastructure
  hosts: all
  become: yes
  tasks:
    - name: Update all packages
      apt:
        update_cache: yes
        upgrade: yes

    - name: Install essential packages
      apt:
        name:
          - docker.io
          - docker-compose
          - nginx
          - certbot
        state: present

    - name: Start and enable Docker
      service:
        name: docker
        state: started
        enabled: yes

    - name: Deploy application
      docker_compose:
        project_src: /opt/my-app
        build: yes

Conclusion

Self-hosting represents a journey of continuous learning and improvement. From the simple Raspberry Pi setup to complex multi-server infrastructures, every self-hosting setup faces similar challenges—particularly around networking and DNS resolution. The key to success lies in understanding the fundamentals, implementing proper monitoring, and developing a systematic approach to troubleshooting.

Remember that every experienced sysadmin has faced the “It’s not DNS” moment, and those experiences build the expertise needed to maintain reliable self-hosted services. Start small, learn continuously, and don’t be afraid to experiment. The self-hosting community is vast and supportive, with countless resources available for those willing to dive in.

As you progress in your self-hosting journey, you’ll develop an intuition for identifying and resolving issues quickly. You’ll learn to anticipate problems before they occur and build more resilient systems. Most importantly, you’ll gain the satisfaction of controlling your own infrastructure and the skills that are increasingly valuable in today’s technology landscape.

The world of self-hosting is constantly evolving, with new tools and technologies emerging regularly. Stay curious, keep learning, and don’t hesitate to contribute back to the community that has helped you along the way. Your self-hosting adventure is just beginning, and the possibilities are endless.