Troubleshooting & Hardening

Chapter 8 — Troubleshooting and Security Hardening

A VPN that appears to be working may still be leaking. This final chapter covers the four most common failure modes — DNS leaks, IPv6 leaks, kill switch gaps, and MTU fragmentation — along with logging, server hardening, and a final checklist that applies to both self-hosted and commercial VPN setups.

DNS Leaks — Finding and Fixing Them

Chapter 7 introduced DNS leaks conceptually. Here we go deeper into the specific causes and their fixes on a self-hosted setup.

Cause 1 — systemd-resolved ignoring VPN DNS

diagnosing systemd-resolved DNS routing
# see which DNS is being used per interface philip@laptop:~$ resolvectl status Global DNS Servers: 82.132.240.1 ← ISP DNS still global! Link 4 (tun0) DNS Servers: 10.8.0.1 ← VPN DNS only on tun0 # fix — set VPN interface DNS as the global default philip@laptop:~$ resolvectl dns tun0 10.8.0.1 philip@laptop:~$ resolvectl domain tun0 "~." # "~." means "use this interface for ALL domains" — makes tun0 the default resolver # permanent fix for OpenVPN — add to server.conf # push "dhcp-option DNS 10.8.0.1" (already in Chapter 5 config) # and add to client.ovpn: # script-security 2 # up /etc/openvpn/update-resolv-conf # down /etc/openvpn/update-resolv-conf

Cause 2 — browser DNS-over-HTTPS bypassing VPN

# Chrome — disable DoH (or point it at VPN provider's DoH endpoint) # chrome://settings/security → Use secure DNS → turn off OR set custom # Mullvad's DoH: https://dns.mullvad.net/dns-query # ProtonVPN's DoH: https://dns.protonvpn.com/dns-query # Firefox — about:preferences#privacy → DNS over HTTPS # Set to "Off" or enter provider's DoH URL in the custom field

Cause 3 — IPv6 leaking outside the tunnel

disable IPv6 to prevent leaks (WireGuard / OpenVPN)
# check if IPv6 is active philip@laptop:~$ ip -6 addr show | grep -v "::1" inet6 2a00:23c7:a48:8400::1/128 scope global ← IPv6 active, may leak # disable IPv6 for the session philip@laptop:~$ sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1 philip@laptop:~$ sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1 # make permanent philip@laptop:~$ echo "net.ipv6.conf.all.disable_ipv6 = 1" | sudo tee -a /etc/sysctl.conf philip@laptop:~$ echo "net.ipv6.conf.default.disable_ipv6 = 1" | sudo tee -a /etc/sysctl.conf # WireGuard alternative — tunnel IPv6 too (add to client config) # AllowedIPs = 0.0.0.0/0, ::/0 ← routes both IPv4 and IPv6 through VPN

Kill Switch — Implementing It Yourself

Commercial clients have kill switches built in. For a self-hosted WireGuard or OpenVPN setup, you implement the kill switch yourself with iptables or nftables — blocking all non-VPN traffic at the firewall level.

WireGuard kill switch via iptables (add to wg0.conf)

# /etc/wireguard/wg0.conf — PostUp/PreDown kill switch # Replace eth0 with your actual external interface PostUp = iptables -I OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) \ -m addrtype ! --dst-type LOCAL -j REJECT; \ ip6tables -I OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) \ -m addrtype ! --dst-type LOCAL -j REJECT PreDown = iptables -D OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) \ -m addrtype ! --dst-type LOCAL -j REJECT; \ ip6tables -D OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) \ -m addrtype ! --dst-type LOCAL -j REJECT
How this works: WireGuard marks its own packets with a firewall mark. The iptables rule rejects any OUTPUT packet that is not on the WireGuard interface and does not carry WireGuard's mark — meaning all non-VPN traffic is dropped. The rule is added when the tunnel comes up (PostUp) and removed when it goes down (PreDown), so you regain internet access when you manually disconnect.

OpenVPN kill switch via UFW

openvpn kill switch with ufw
# allow traffic only on the VPN interface and to/from the VPN server itself philip@laptop:~$ sudo ufw default deny outgoing philip@laptop:~$ sudo ufw default deny incoming # allow the VPN server endpoint (replace with your server IP and port) philip@laptop:~$ sudo ufw allow out to 203.0.113.42 port 1194 proto udp # allow all traffic on the VPN tunnel interface philip@laptop:~$ sudo ufw allow out on tun0 philip@laptop:~$ sudo ufw allow in on tun0 # allow DNS on the VPN interface only philip@laptop:~$ sudo ufw allow out on tun0 to any port 53 philip@laptop:~$ sudo ufw enable # result: no internet without VPN — only the VPN server and tun0 traffic allowed

MTU Issues — The Silent Performance Killer

MTU (Maximum Transmission Unit) fragmentation is one of the most common and least obvious VPN problems. When VPN encapsulation adds overhead to each packet, packets that were exactly at the network's MTU limit become too large and must be fragmented — or dropped entirely if the Don't Fragment bit is set.

Standard Ethernet frame: 1500 bytes MTU
WireGuard adds overhead: 60 bytes (headers + encryption)
Effective WireGuard payload: 1420 bytes (wg-quick sets this automatically)
OpenVPN UDP overhead: ~50–80 bytes depending on cipher
OpenVPN TCP overhead: even higher — TCP headers on top
Symptom of wrong MTU: large sites work, large downloads stall, SSH hangs on banner
finding and fixing the right MTU
# test the largest packet size that doesn't fragment (do this with VPN connected) # Linux — ping with Don't Fragment bit, decrease size until it works philip@laptop:~$ ping -M do -s 1400 8.8.8.8 PING 8.8.8.8: 1400 data bytes ← if this works, try larger philip@laptop:~$ ping -M do -s 1450 8.8.8.8 ping: local error: message too long, mtu=1420 ← 1420 is the limit # WireGuard — set MTU explicitly in wg0.conf [Interface] if auto-detection fails # MTU = 1420 # OpenVPN — add to server.conf and client.ovpn # tun-mtu 1400 # mssfix 1360 # quick fix on a running interface philip@laptop:~$ sudo ip link set dev tun0 mtu 1400 # verify current MTU philip@laptop:~$ ip link show tun0 | grep mtu 4: tun0: <...> mtu 1400 ...

Reading VPN Logs

key log lines to recognise
── WireGuard ───────────────────────────────────────────────── root@server:~# wg show wg0 peer: X7nK2vP9... latest handshake: 14 seconds ago ← connected latest handshake: (none) ← never connected / key mismatch transfer: 1.44 MiB received, 248 KiB sent ← traffic flowing ── OpenVPN server log (/var/log/openvpn/openvpn.log) ────────── Initialization Sequence Completed ← server started OK client1/.../54123 MULTI: Learn: 10.8.0.6 ← client connected, got IP TLS Error: TLS key negotiation failed ← cert/key mismatch or wrong direction AUTH_FAILED ← revoked cert or wrong credentials read UDPv4: Connection refused ← nothing listening on that port ── OpenVPN client log (journalctl or --log file) ────────────── Initialization Sequence Completed ← tunnel up SIGTERM received, sending exit notification ← clean disconnect Connection reset, restarting ← tunnel dropped, reconnecting NOTE: --mute triggered ← too many repeated messages suppressed # increase OpenVPN verbosity for TLS debugging philip@laptop:~$ sudo openvpn --config client1.ovpn --verb 6 2>&1 | grep -i tls

Common Problems — Quick Reference

Large file downloads stall or hang
MTU fragmentation — large packets are dropped because the VPN overhead pushes them over the physical MTU limit.
Test with ping -M do -s 1400 8.8.8.8 and decrease until it works. Set MTU = 1420 in WireGuard or tun-mtu 1400 + mssfix 1360 in OpenVPN.
VPN connected but DNS still leaks
systemd-resolved is using the system-wide DNS rather than the VPN interface's DNS, or the browser has DoH enabled.
Run resolvectl domain tun0 "~." to make the VPN interface the default resolver. Disable browser DoH or point it at the VPN provider's DoH endpoint.
Tunnel drops every ~2 minutes on mobile
NAT table on the router or mobile carrier times out idle UDP sessions after 120 seconds.
Set PersistentKeepalive = 25 in the WireGuard client config. For OpenVPN add keepalive 10 60 to the server config.
IPv6 sites work but show real location
VPN is only tunnelling IPv4. IPv6 traffic bypasses the tunnel entirely.
Add ::/0 to WireGuard AllowedIPs, or disable IPv6 on the client: sysctl -w net.ipv6.conf.all.disable_ipv6=1.
WireGuard handshake times out from one network
UDP port 51820 is blocked by a corporate firewall or ISP.
Change ListenPort to 53 (DNS) or 443 on the server, update UFW and the client's Endpoint port. Or switch to OpenVPN on TCP 443.
OpenVPN TLS handshake failed
key-direction mismatch (client must be 1, server must be 0), or remote-cert-tls server failing because the server cert was signed as a client cert.
Verify key-direction 1 in the client .ovpn. Confirm the server cert was signed with sign-req server, not sign-req client. Use verb 6 for detailed TLS output.

Server Security Hardening Checklist

  • Change the default port. WireGuard default 51820 and OpenVPN default 1194 are well-known. Using a non-standard port eliminates automated scan noise. Common alternatives: 443, 53, 4433.
  • Use key-based SSH only on the VPN server. Disable password authentication: PasswordAuthentication no in /etc/ssh/sshd_config. The VPN server is a high-value target — it must not be brute-forceable.
  • Run fail2ban on SSH. Even with key-only auth, fail2ban stops log noise and blocks systematic scanning.
  • Keep WireGuard and OpenVPN updated. Both have had security patches. Run apt upgrade wireguard openvpn regularly and set up unattended-upgrades for security packages.
  • Restrict firewall to VPN port only. The server should have SSH (on a non-standard port if possible) and the VPN port open — nothing else. ufw default deny incoming with only those two ports allowed.
  • Rotate client credentials periodically. WireGuard: regenerate the client key pair and update the server's [Peer] block. OpenVPN: issue a new certificate and revoke the old one via Easy-RSA CRL.
  • Don't log more than necessary. If your goal is privacy, minimise what the server records. For WireGuard there is no built-in logging. For OpenVPN, set verb 0 or verb 1 in production and rotate logs with logrotate.
  • Use a preshared key in WireGuard. The PresharedKey directive adds a layer of symmetric encryption on top of Curve25519 — useful post-quantum insurance. Generate one per peer with wg genpsk.
  • Verify your public IP after every connection. Make it a habit: connect, run curl ifconfig.me, confirm it shows the VPN server's IP. Takes five seconds and catches misconfiguration immediately.
  • Run the full DNS + WebRTC + IPv6 leak test suite periodically — especially after OS updates or VPN client upgrades. A system update can reset DNS settings or re-enable IPv6 without warning.
Course Complete — VPN
From understanding what a VPN actually does and doesn't protect, through building WireGuard and OpenVPN servers from scratch, connecting every type of client, using commercial providers intelligently, and verifying that nothing is leaking — you now have a complete, practical understanding of VPN technology from both sides of the tunnel.
8
Chapters
3
Protocols covered
4
Platforms (Win/Mac/iOS/Android)
10
Hardening checklist items