Barring the doors
Hardening isn't a phase you execute once, tick a box, and forget about. It’s the whole job, applied everywhere, consistently, and boring on purpose. The "Hack the Planet" crowd only needs to find one open back door to trash your entire file system.
Part One was about getting the engine running. Part two is about making sure nobody else can steer the ship.
Paranoid defaults
First up were the kernel knobs, which are the unglamorous, blue-collar work of sysctl hardening. Reverse-path filtering turned on, ICMP redirects and source routing disabled, SYN cookies enabled, kernel pointers hidden from unprivileged processes, ptrace scoped down, and the usual hardlink and symlink protections. None of it is exciting. All of it just closes doors that were left wide open by default for reasons nobody in tech remembers anymore.
Then came the firewall. I staged the rules offline and brought firewalld up with them already in place, marking Tailscale as trusted from the very first rule. The tunnel was never filtered for even a heartbeat. The one interface I could not afford to lock myself out of was the escape pod I’d just built. Public ports came down to exactly three: SSH, HTTP, and HTTPS. Everything else, including the box's own management, lives strictly on the mesh.
The watchful blue box
Next came fail2ban. People love to argue that brute-force protection is pointless once you've disabled password authentication. They’re wrong in a quiet, evidentiary way. Within seconds of the service coming up, it had already banned four real IPs mid-knock.
The internet scans you constantly. A public box on the open internet is a police box sitting under a low, steady flow of Daleks. An endless queue of them glides up to the door, plunger raised, rasping for entry. Even with keys-only auth, fail2ban is the Last Centurion at the door. It never sleeps. It never stands down. It clocks every knock, bans the IP mid-EXTERMINATE, and turns away the things that go bump in the night before they ever cost you a single log line of worry.
I also left SELinux in enforcing mode. In my early years, just about every senior admin told me to disable it out of the gate and move on. I chose to actually sit down and learn how it ticks instead, and that investment has always paid off. Leaving it on blocks container escapes and cripples the blast radius of random ransomware scripts. If a compromised web container tries to rewrite host configurations or encrypt your storage directories, SELinux steps in at the kernel level and slams the door. In some tech circles leaving it on is considered heresy, but in my opinion it is just a sensible upgrade. Docker behaves itself beautifully once you properly label your bind mounts, so the juice is worth the squeeze.
One landmine worth watching out for is that the sshd config is first-match-wins, and the cloud image fights you. The base image drops its own custom snippet that quietly re-enables password authentication. Because the first value for a setting wins, my hardening file, which sorted later alphabetically, was politely ignored. Name your config files so they sort first. And always check the effective configuration with sshd -T, not the file you fondly believe you wrote.
A key for the front gate
Then came the fun part: hardware MFA. And here's the thing nobody tells you up front. A YubiKey isn't just one tool. It’s got more personalities than a Time Lord mid-regeneration.
Touch it one way and it acts like a USB keyboard, typing out a one-time code. Touch it another way and it executes a cryptographic handshake over a completely different protocol that needs to talk directly to whatever is performing the check. That difference is the whole ballgame. The second kind cannot reach a key plugged into your laptop when the verifier is a server on the other side of the planet. People wire up the wrong mode, get baffled when remote logins hang, and blame the hardware.
Radar ended up with two mechanisms wearing the same little metal hat. Getting in uses hardware-backed SSH keys. The secret never leaves the physical silicon, the touch happens on my side, and the server just verifies the signature. Escalating to root uses the typed one-time code. Login and privilege escalation are both gated by a physical object I hold in my hand.
The door I hid in the walls
Because I'd already managed to lock myself out of two separate boxes simultaneously earlier in this project (hi, part one), I built a break-glass account. I'm not going to tell you what I named it. Let's just say it's the one that can't die, named after a character who's famously hard to kill, so that I can easily remember it.
It’s key-only, with the key sealed deep inside my password manager. Crucially, it leans on neither the YubiKey nor the Tailscale mesh. If the key dies, if Tailscale is having an off day, or if I fat-finger the authentication stack into absolute oblivion, that account still lets me in.
Every fortress needs an escape route that only you know about. Unlike Moria, the password isn't just "friend." And unlike Moria, I'm the only thing waiting on the other side of it.
The afternoon I lowered my own shields
Hardening also means taking things off the box. Every package you don't absolutely need is an attack surface you're carrying around for free. So I went through Radar's installed packages and lined up the ones a headless Docker host has no business running: an old mail server with a long rap sheet of remote-code-execution bugs, a web admin console I'd never open, a directory client I no longer needed, a Bluetooth stack on a server sitting in a datacenter, and a few others.
Then I ran the removal command with the yes to everything flag and didn't read the transaction screen. That's the whole mistake, right there, in one sentence.
The mail server I was yanking turned out to be required by a small add-on of fail2ban, which was the brute-force protection I'd spent the entire morning praising. Dependency resolution, being a literal genie, granted my wish to the letter. It pulled the mail server, and on the way out, it took fail2ban with it.
I had disabled my own shields mid-orbit and hadn't even noticed.
I caught it on the verify step, which is the step I will defend to my grave. I reinstalled fail2ban without the mail-server hook, and it came straight back up. Protection was down for a couple of minutes. My ego was down somewhat longer.
I'm telling you this because the flawless write-up is a lie of omission, and because twenty years in this industry does not make you immune to stupidity. It just changes which mistakes you make. The thing that saved me wasn't being too senior to screw up. It was the verify step and the break-glass account I'd built for exactly this kind of self-inflicted wound.
Read the transaction before you say yes. Using -y under flow is the dark side. It is quicker, easier, and more seductive, and it will absolutely take your hand off.
Where we are
Doors barred, a hardware key on the front gate, a hidden door in the walls, and a fresh, humbling reminder to read before I sign. The last piece of the puzzle is getting code onto this box from CI without leaving a spare key under the doormat for anyone who finds the welcome sign.
Part three is where I wire up the pipeline. And try not to build a backdoor for the rest of the internet.
Full Series
Three posts, one migration:
- vps01 is dead. Long live Radar. The dead two-years-EOL CentOS 7 box, and why you regenerate instead of resurrect.
- Barring the Doors: Hardware Keys and Self-Inflicted Wounds. Hardening the new host with sysctl, fail2ban, SELinux and a YubiKey, plus the firewall I removed by accident. (You're here.)
- Pull, Don't Push: No Spare Key Under the Doormat. Keyless deploys that never reach into the box. (You're here.)