Linux Server Hardening Essentials for Business Servers
The essential steps to harden a Linux server: SSH keys, automatic security updates, a default-deny firewall, fail2ban, minimal services, and logging.
Hardening a Linux server comes down to reducing what an attacker can reach and making every remaining entry point harder to abuse. The essentials are: use SSH keys and disable password and root logins, apply security updates automatically, run a default-deny firewall that only opens required ports, protect exposed services with fail2ban or similar, remove software you do not need, and keep logs you actually review. These steps apply to Debian, Ubuntu, AlmaLinux, Rocky Linux, and most other distributions with minor differences.
Before changing anything on a production server, take a snapshot or backup and keep an existing SSH session open while you test, so a mistake does not lock you out.
1. Start with a clean, minimal install
Every installed package is something to patch. Install only what the server's role requires, and remove leftovers:
- List listening services with
ss -tulpnand question anything you do not recognize. - Disable services you do not need with
systemctl disable --now <service>. - Avoid installing desktop environments or development tools on production servers.
2. Create named admin accounts
Do not work as root. Create a named account for each administrator, grant elevated rights through sudo, and remove accounts that are no longer used. Named accounts make logs meaningful: you can see who did what.
3. Harden SSH
SSH is usually the most exposed management service. Use key-based authentication, ideally Ed25519 keys protected with a passphrase, then tighten the daemon configuration. On recent distributions you can place overrides in a file under /etc/ssh/sshd_config.d/. OpenSSH uses the first value it reads for each setting, so give the file a name that sorts first, such as 00-hardening.conf, and check for conflicting files left by cloud images:
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
MaxAuthTries 3
AllowUsers alice bob
X11Forwarding no
Validate the configuration before reloading, and test a new login from a second terminal before closing your current session:
sudo sshd -t
sudo sshd -T | grep -Ei 'passwordauth|permitroot'
sudo systemctl reload ssh # Debian/Ubuntu
sudo systemctl reload sshd # RHEL, AlmaLinux, Rocky
Where possible, restrict SSH to a VPN or to known office IP addresses at the firewall. Changing the SSH port reduces log noise but is not a security control on its own.
4. Turn on a default-deny firewall
Allow only what the server needs and deny everything else inbound.
On Ubuntu or Debian with UFW:
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow from 203.0.113.0/24 to any port 22 proto tcp
sudo ufw allow 443/tcp
sudo ufw enable
On AlmaLinux or Rocky Linux, firewalld provides the same result using zones and firewall-cmd. Replace the example network with your own trusted range. If the server sits behind a cloud provider, use security groups or the provider firewall as an additional layer.
5. Automate security updates
Unpatched software is one of the most common ways servers are compromised.
| Distribution | Tool for automatic security updates |
|---|---|
| Debian, Ubuntu | unattended-upgrades |
| AlmaLinux, Rocky, RHEL | dnf-automatic |
Automatic security updates suit most small business servers. For critical applications, pair them with monitoring and a schedule for kernel updates and reboots, or use live patching where your distribution supports it.
6. Block brute-force attempts
Tools like fail2ban watch log files and temporarily ban IP addresses that repeatedly fail authentication. Enable a jail for SSH and for any exposed web login, mail service, or control panel. On cPanel and similar hosting servers, tools such as Imunify360 and CSF provide comparable protection integrated with the panel.
7. Protect data and permissions
- Keep file permissions tight, especially for configuration files containing credentials.
- Use separate system users for each application so a compromise in one does not expose the others.
- Encrypt backups and store at least one copy off the server.
- Keep SELinux (RHEL family) or AppArmor (Ubuntu, Debian) enabled rather than disabling it to make something work.
8. Log, monitor, and review
- Make sure
journaldand syslog retain enough history, and forward logs to a central system if you manage several servers. - Enable
auditdon servers that handle sensitive data. - Monitor disk space, failed logins, and unexpected new listening ports.
- Review logs on a schedule, not only after an incident.
Our server setup and repair service applies these baselines to new and existing servers, and our cybersecurity services add ongoing monitoring and periodic reviews.
FAQ
Is changing the SSH port enough to secure a server?
No. It reduces automated noise in your logs, but determined attackers scan all ports. Key-only authentication, firewall restrictions, and updates are what provide real protection.
Will automatic updates break my applications?
Security-only updates rarely cause problems, but it can happen. Test major upgrades in staging, monitor after updates, and keep current backups or snapshots so you can roll back.
Do these steps apply to cloud servers too?
Yes. Whether the server runs on AWS, Azure, Google Cloud, OVHcloud, or a Proxmox host in your office, the operating system still needs hardening. Cloud firewalls add a layer but do not replace the server configuration.
Next step
Max, our founder, has administered Linux and hosting servers for more than 20 years and is a certified cPanel & WHM System Administrator and Imunify360 professional. Request a free IT assessment and we will review your servers against these essentials.
