Hardening Linux Entry Points: Advanced SSH and Access Control

The night I almost lost a production server to a brute force attack

I still remember the panic I felt about five years ago. I was managing a decently trafficked web server for a client, and I had been lazy. I left the SSH port on 22, used a decent password (but not an SSH key), and figured, "Who's going to target this random IP?" I woke up at 3 AM to monitoring alerts screaming that the disk was full. It wasn't the database. It wasn't user uploads. It was the /var/log/auth.log file. It had ballooned to gigabytes in size because a botnet was hammering the server with thousands of login attempts per second. While they didn't get in, they successfully Denial-of-Service'd the machine just by knocking on the door too hard.

That was my wake-up call. Securing the "introduction"—that initial handshake where a user (or bot) asks the kernel for entry—is the single most significant security task you have as a sysadmin. It doesn't matter how secure your internal permissions are if the front door is made of paper. Over the years, I've moved far beyond just changing ports and disabling root login. If you're running Linux in production today, you need to be thinking about Single Packet Authorization, hardware-backed keys, and kernel-level restrictions.

1. Moving Beyond Basic SSH Keys: Ed25519 and FIDO2

Most tutorials tell you to generate an RSA key and call it a day. Look, RSA 2048 is arguably dead, and even RSA 4096 is getting heavy and slow for handshake overhead on smaller embedded devices. For the last few years, I've switched exclusively to Ed25519 keys. They are smaller, faster, and offer better security margins. But if you want to get properly "advanced," you should be using hardware-backed SSH keys.

Since OpenSSH 8.2 (released back in 2020), native FIDO/U2F support has been built in. This means you can generate an SSH key that requires your YubiKey or verified hardware token to be physically touched before the authentication goes through. If a hacker steals your private key file from your laptop, it's useless to them without the physical USB stick.

Here is how I generate my keys these days:

ssh-keygen -t ed25519-sk -C "laptop-2024-yubi"

The -sk stands for security key. Once you set this up, you need to enforce it in your /etc/ssh/sshd_config. Don't just allow keys; specify the algorithms. I strictly limit the Key Exchange Algorithms to drop older, weaker SHA1 dependencies:

KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com

Lesson Learned: I once updated a legacy server and pasted my modern config file into it without checking the OpenSSH version. The server was running an ancient CentOS 6 version that didn't support the Elliptic Curve algorithms. I locked myself out completely and had to use the hosting provider's VNC web console to fix it. Always check ssh -V before applying aggressive crypto policies.

2. The Invisible Door: Single Packet Authorization (SPA)

You've probably heard of Port Knocking. That's where you hit a sequence of closed ports (like 7000, 8000, 9000) to open the SSH port. Honestly? Port Knocking is kind of a gimmick. It suffers from replay attacks—if someone sniffs your traffic, they can just replay the knock sequence.

The grown-up version of this is Single Packet Authorization (SPA). I use a tool called fwknop (FireWall KNock OPerator). Here is the concept: your firewall defaults to DROP for everything, including SSH. The port shows as closed/filtered to Nmap scans. To get in, your client sends a single, encrypted UDP packet to the server. This packet contains a one-time use payload that tells the server, "Hey, open port 22 specifically for my IP address, and close it again in 30 seconds."

The beauty of SPA is that it is passive. The server doesn't reply to the packet. If the decryption fails, it just drops it. There is no handshake to exploit. I run this on my jump boxes.

To set it up, you install fwknop-server on the Linux box and fwknop-client on your laptop. The configuration file typically lives at /etc/fwknop/access.conf:

SOURCE: ANY
OPEN_PORTS: tcp/22
KEY: [Your-Base64-Key]
FW_ACCESS_TIMEOUT: 30

This setup makes your server virtually invisible on the internet. Shodan can't index what it can't see.

3. Automated Active Defense: CrowdSec over Fail2Ban

For a decade, Fail2Ban was the gold standard. It parses logs, finds bad IPs, and updates the firewall. It works, but it's reactive and isolated. If an IP attacks me, Fail2Ban bans it, but my neighbor's server has to wait until it gets attacked by the same IP to do the same.

I've recently migrated most of my infrastructure to CrowdSec. It’s open-source and behaves like a collaborative WAF (Web Application Firewall). When my server detects a brute force attack, it bans the IP locally, but it also sends that signal to the central CrowdSec database. If enough verified users report the IP, it gets added to a global blocklist distributed to everyone else.

The installation is surprisingly smooth compared to the regex hell that is Fail2Ban configuration:

sudo apt install crowdsec
sudo apt install crowdsec-firewall-bouncer-iptables

That's practically it. I ran a test last month comparing the two. On a fresh DigitalOcean droplet, Fail2Ban caught about 400 bad interactions in 24 hours. CrowdSec, leveraging the community blocklist, preemptively blocked over 4,000 IPs that hadn't even touched my server yet. That is a massive reduction in noise and risk.

4. Kernel Hardening and the "Introduction" Process

Securing the introduction isn't just about the network; it's about what happens immediately after a user logs in. If an attacker manages to get a shell, you want to limit the damage they can do immediately. This brings us to sysctl tuning.

I see so many people ignoring the kernel pointer restrictions. If an attacker gets in, their first move is usually privilege escalation. Hiding kernel symbols makes that significantly harder. Edit your /etc/sysctl.conf and add:

kernel.kptr_restrict = 2
kernel.dmesg_restrict = 1
fs.suid_dumpable = 0

dmesg_restrict = 1 prevents a standard user from viewing the kernel message buffer. This log often contains memory addresses and hardware info that exploits need to function. It drives me crazy when I audit servers and see users able to read the entire boot history.

Another Mistake I Made: I once set kernel.modules_disabled = 1 without thinking. This prevents new kernel modules from loading. It's great for security, but I forgot that I needed to mount a USB drive later that day which required a filesystem module that wasn't loaded yet. I couldn't mount the drive until I rebooted the machine. Use that setting with caution.

5. The Human Element: MFA at the PAM Level

Even with keys, sometimes you need a fallback, or corporate policy mandates Multi-Factor Authentication (MFA). You can hook Google Authenticator directly into the Linux Pluggable Authentication Modules (PAM) layer. This implies that even if someone steals your SSH key, they still need the code from your phone.

The package is usually libpam-google-authenticator. After running the initialization tool, you have to edit /etc/pam.d/sshd. But here is the trick that trips people up: the order matters. If you put the auth required line at the top, it might prompt for the MFA code before the SSH key check, which breaks some automation tools like Ansible.

I prefer to set up "AuthenticationMethods" in SSH to require both a public key AND a password (or token). In sshd_config:

AuthenticationMethods publickey,keyboard-interactive

This forces the "AND" logic. Just having the key isn't enough. Just having the password isn't enough. You need the physical file and the ephemeral code.

FAQ: Questions I Get Asked Constantly

Is changing the SSH port actually useful?

Yes and no. Security purists will yell "Security by Obscurity is not Security!" and they are technically right. A targeted attacker will scan all 65,535 ports and find your SSH on port 2222 in minutes. However, changing the port drastically reduces log noise. It filters out the 99% of dumb scripts that just hit port 22 and move on. I change it just to keep my logs clean, not because I think it makes me unhackable.

What happens if I lose my FIDO2 hardware key?

You are locked out, simple as that. This is why you must have a backup strategy. I usually keep a standard Ed25519 key (without FIDO) encrypted with a very long, complex passphrase, and I store that key on an encrypted USB drive in a physical safe. It's the "break glass in case of emergency" key. Never rely on a single hardware token for root access.

Why not just use a VPN to access the server?

Using a VPN (like WireGuard) to access a private management network is actually the best practice. If you can do this, do it. SSH shouldn't be exposed to the public internet at all if you can help it. However, for many standalone VPS setups or public-facing web servers where setting up a mesh VPN is overkill or impossible, the hardening steps above are the standard.

Can I use these settings on older Linux distros?

Be very careful. As I mentioned in my story above, settings like Ed25519-sk or specific KexAlgorithms are dependent on the OpenSSH version. CentOS 7, for example, ships with an older OpenSSH that doesn't support FIDO keys natively without compiling from source (which I don't recommend for production stability). Always check your version numbers.

My Take: Paranoia vs. Usability

There is a fine line between a secure server and a server that is so annoying to use that you stop maintaining it. I've been there—I've locked down a box so hard with port knocking, MFA, and IP allow-lists that I dreaded logging in to run updates. When you dread logging in, you delay updates, and that makes you insecure.

Start with the basics: disable root login, switch to Ed25519 keys, and install CrowdSec. If you are handling sensitive data, implement the FIDO2 hardware keys. But don't implement Single Packet Authorization just to feel like a spy if it disrupts your Ansible workflows. Security is a process, not a destination, and it has to work for you, not against you.