So Apparently Google Will Spam Your Self-Hosted Email Just Cuz You Thats Why

So Apparently Google Will Spam Your Self-Hosted Email Just Cuz You Thats Why

INTRODUCTION

If you’ve ever spent hours configuring DKIM, DMARC, SPF, rDNS, and a dozen other DNS records only to watch your carefully crafted homelab mail land in the Gmail spam folder, you’re not alone. The frustration is real: you’ve double‑checked every DNS entry, verified that the EHLO hostname resolves correctly, and even ran third‑party deliverability tests that report “all checks passed.” Yet Gmail still treats your messages as junk, and the only thing you can hear is the echo of your own sigh.

This guide is built for experienced sysadmins and DevOps engineers who run self‑hosted mail servers in a homelab or small‑scale production environment. It will walk you through the exact reasons why a perfectly configured mail stack can still be flagged by Gmail, how to diagnose the hidden factors that Google’s spam filters consider, and what concrete steps you can take to reclaim inbox placement.

By the end of this article you will:

• Understand the full landscape of email authentication – SPF, DKIM, DMARC, rDNS, and beyond.
• Learn how Google evaluates sending reputation, including factors that are often overlooked.
• Follow a step‑by‑step troubleshooting workflow with real‑world command examples.
• Implement best‑practice hardening and monitoring to keep your homelab mail delivery reliable.

Keywords such as self‑hosted, homelab, DevOps, infrastructure, automation, and open‑source are woven throughout to help search engines surface this guide when you search for “why my self‑hosted email goes to spam” or “Gmail spam filtering self‑hosted mail”.

---

UNDERSTANDING THE TOPIC

The Core Concept

Self‑hosted email is a cornerstone of many DevOps homelabs. It provides full control over messaging, data privacy, and integration with internal services. However, the moment you try to send mail to external domains – especially to Gmail – you step into a complex ecosystem of reputation, policy, and algorithmic filtering.

Google’s spam detection pipeline is not a simple rule engine; it blends traditional checks (SPF, DKIM, DMARC) with proprietary signals such as:

• Sending reputation – historical volume, bounce rates, and complaint rates.
• Content analysis – presence of certain keywords, HTML structure, and attachment types.
• IP and domain warm‑up patterns – how quickly you scale up volume.
• Reverse DNS (rDNS) consistency – whether the IP’s PTR record matches the hostname you present in the SMTP EHLO command.

If any of these signals conflict, Gmail may place the message in the spam folder even when all published authentication records resolve correctly.

Historical Context

Early mail infrastructure relied on simple relaying and trust. The introduction of RFC 7208 (DKIM), RFC 7208 (DMARC), and RFC 1912 (SPF) gave administrators standardized ways to prove authenticity. Yet spam‑filters quickly evolved to incorporate behavioural data, forcing mail administrators to think beyond static DNS records.

In the DevOps world, tools like Postfix, Haraka, Mailcow, and Modoboa have made it easier to spin up a complete mail stack inside a Docker container. However, the convenience of containerization can mask misconfigurations that only surface when external mail providers evaluate the message.

Key Features

Feature	Purpose	Typical Implementation
SPF	Verify that the sending IP is authorized to send for the domain	TXT record: v=spf1 ip4:192.0.2.10 -all
DKIM	Sign messages with a private key; verify with a public key in DNS	OpenDKIM signing table, opendmarc signing
DMARC	Provide policy for handling failures of SPF/DKIM	TXT record: v=DMARC1; p=reject; rua=mailto:postmaster@example.com
rDNS / PTR	Ensure the IP’s reverse DNS matches the HELO/EHLO hostname	PTR record in the IP block pointing to mail.example.com
TLS	Encrypt the SMTP session to prevent downgrade attacks	smtpd_tls_security_level = may in Postfix
Rate limiting	Prevent bursts that look like spam	postfix anvil rate maps or Docker --restart unless-stopped policies

Pros and Cons

Pros

• Full control over data and privacy.
• Ability to integrate with internal services (e.g., CI pipelines, monitoring).
• Cost‑effective for low‑volume homelab use.

Cons

• Reputation must be built from scratch – no shared IP pool.
• Requires continuous monitoring and periodic warm‑up.
• Misconfigurations can cause silent delivery failures that are hard to debug.

Real‑World Scenarios

• Scenario 1 – A homelab runs a Postfix container with correct SPF/DKIM/DMARC, but the IP’s PTR record points to mail.host.com while the container’s myhostname is set to mail.example.com. Gmail rejects the mismatch.
• Scenario 2 – A new IP address is used to send 500 messages per minute. Gmail’s volume‑based filter flags it as potential bulk spam, moving all messages to the junk folder.
• Scenario 3 – The email body contains a URL shortener that is flagged by Google’s Safe Browsing API, causing the entire message to be classified as spam regardless of authentication.

Understanding these nuances is essential before diving into configuration steps.

---

PREREQUISITES

System Requirements

• A dedicated virtual or physical host with at least 2 CPU cores, 4 GB RAM, and 20 GB of storage.
• A static IPv4 address (or a floating IP) that can be assigned a reverse DNS (PTR) record.
• Access to the DNS provider’s control panel for adding TXT, MX, and PTR records.

Software Stack

Component	Minimum Version	Reason
Docker Engine	24.0+	Provides isolation for mail services.
Docker Compose	2.20+	Orchestrates multi‑container mail stack.
Postfix	3.9+	Reliable MTA with extensive configuration options.
OpenDKIM	2.13+	Handles DKIM signing and verification.
OpenDMARC	1.3+	Enforces DMARC policy and reporting.
Certbot (optional)	2.9+	Issues TLS certificates for SMTP over TLS.
Linux Distribution	Ubuntu 22.04 LTS or Debian 12	Well‑supported package versions.

Network and Security

• Open inbound ports 25 (SMTP), 587 (Submission), and 465 (SMTPS) on the host firewall.
• Restrict outbound traffic to only necessary services (e.g., DNS, NTP).
• Use a non‑root user for container processes; avoid running containers as root.

Pre‑Installation Checklist

1. Reserve a static IP and request a PTR record from your provider (e.g., mail.example.com → 203.0.113.45).
2. Create DNS records:
   • MX: example.com. 3600 IN MX 10 mail.example.com.
   • SPF: example.com. 3600 IN TXT "v=spf1 ip4:203.0.113.45 -all"
   • DKIM: default._domainkey.example.com. 3600 IN TXT "v=DKIM1; k=rsa; p=..."
   • DMARC: example.com. 3600 IN TXT "v=DMARC1; p=reject; rua=mailto:postmaster@example.com"
3. Verify that dig +short MX example.com returns the correct mail server.
4. Ensure the host’s hostname resolves to the static IP (both forward and reverse).

---

INSTALLATION & SETUP

Below is a complete, reproducible Docker‑Compose based installation for a self‑hosted mail stack that includes Postfix, OpenDKIM, OpenDMARC, and a lightweight web UI for monitoring.

1. Directory Layout

mkdir -p ~/homelab/mail/{postfix,opendkim,opendmarc,certs}
cd ~/homelab/mail

2. Docker‑Compose File

version: "3.8"

services:
  postfix:
    image: catatnight/postfix:latest
    container_name: $CONTAINER_NAMES_POSTFIX
    restart: unless-stopped
    environment:
      - myhostname=mail.example.com
      - mydomain=example.com
      - mynetworks=127.0.0.0/8 [YOUR_IP/32]
      - smtpd_tls_security_level=may
      - smtpd_tls_cert_file=/certs/tls.crt
      - smtpd_tls_key_file=/certs/tls.key
    ports:
      - "25:25"
      - "587:587"
      - "465:465"
    volumes:
      - ./postfix:/var/spool/postfix
      - ./opendkim:/var/run/opendkim
      - ./certs:/certs:ro

  opendkim:
    image: comanigy/opendkim:latest
    container_name: $CONTAINER_NAMES_OPENDKIM
    restart: unless-stopped
    environment:
      - DKIM_SELECTOR=default
      - DKIM_KEY_LENGTH=2048
      - DKIM_DOMAIN=example.com
      - DKIM_KEY_PERMISSIONS=0640
    volumes:
      - ./opendkim:/etc/opendkim

  opendmarc:
    image: dmarc/dmarc:latest
    container_name: $CONTAINER_NAMES_OPENDMARC
    restart: unless-stopped
    volumes:
      - ./opendmarc:/etc/dmarc

  certbot:
    image: certbot/certbot:latest
    container_name: $CONTAINER_NAMES_CERTBOT
    command: certonly --standalone -d mail.example.com --non-interactive --agree-tos --email admin@example.com
    volumes:
      - ./certs:/etc/letsencrypt

Explanation of Key Elements

• $CONTAINER_NAMES_POSTFIX, $CONTAINER_NAMES_OPENDKIM, etc., are placeholders for the actual container names you will assign. Replace them with meaningful identifiers (e.g., mail_postfix_1).
• The myhostname and mydomain variables configure Postfix’s internal domain handling.
• mynetworks includes only the loopback and your static IP to prevent open relay.
• TLS certificates are mounted read‑only from ./certs.

3. Generate DKIM Keys

mkdir -p ./opendkim/keys
opendkim-genkey -s default -d example.com -b 2048 -D ./keys
chown -R opendkim:opendkim ./keys
chmod 640 ./keys/default.private

Copy the public key into the OpenDKIM configuration:

cat ./keys/default.txt > ./opendkim/keys/default.txt

4. Populate Configuration Files

Postfix main.cf (minimal example)

cat > ./postfix/main.cf <<'EOF'
myhostname = $mydomain
mydomain = $mydomain
myorigin = $mydomain
mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain
relayhost =
mynetworks = 127.0.0.0/8 [YOUR_IP/32]
mailbox_command =
mailbox_transport = lmtp:unix:/var/run/cyrus/socket/lmtp
inet_interfaces = all
inet_protocols = all
smtpd_tls_cert_file = /certs/tls.crt
smtpd_tls_key_file = /certs/tls.key
smtpd_use_tls = yes
smtpd_tls_security_level = may
smtpd_auth_enable = yes
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
smtpd_recipient_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_unauth_destination
EOF

OpenDKIM Key Table

mkdir -p ./opendkim/tables
cat > ./opendkim/tables/KeyTable <<'EOF'
default._domainkey.example.com example.com:/keys/default.private
EOF

OpenDKIM Signing Table