How to Secure a Public Linux Server Without Locking Yourself Out

A public Linux server has a difficult job. It must accept the connections you actually need while rejecting access that shouldn’t be allowed.

For a website, that might mean permitting public HTTPS traffic while restricting administrative SSH access. For a private application, the intended exposure might be much narrower.

The problem is that security controls can affect legitimate users, too.

A firewall doesn’t automatically know which connection belongs to the owner. SSH doesn’t know whether a disabled authentication method was the only one its administrator could use.

That’s why server hardening is not simply about making settings more restrictive. It’s about making them more restrictive without removing the access you still require.

1. Why Can a Security Change Lock You Out?

Imagine that your VPS accepts SSH connections over TCP port 22. You currently connect using a username and password.

You read that SSH keys are preferable to password authentication, so you immediately disable SSH passwords.

But your public key hasn’t been installed correctly. Your next login fails.

Now imagine a different scenario. Your SSH keys work, but you enable a firewall rule that blocks inbound connections to the SSH port.

SSH is configured correctly, yet the network connection cannot reach it.

Both situations have the same visible result: you can’t log in. But their causes are different.

This distinction matters because fixing one problem doesn’t necessarily fix the others.

If a provider firewall blocks the connection, changing an SSH password won’t help. If your key isn’t authorized, opening more firewall ports won’t solve authentication.

For a systematic connection diagnosis, see Can’t SSH Into Your Linux Server? 12 Checks Before You Reboot.

2. The Safe Order for Hardening a Public Linux Server

The most important security improvement in this article isn’t a single command.

It’s the order in which you perform the work.

Do not reverse this sequence.

For example, disabling password access should be the result of successfully testing SSH keys — not the first step in setting them up.

Likewise, restrictive firewall policies should follow a verified inventory of required services and management connections.

3. Before Changing Anything: Your Safety Checklist

Hosting providers may offer browser consoles, serial consoles, rescue environments or snapshots. These are not interchangeable, and their availability varies.

A snapshot may help with restoration, but it is not the same as a tested emergency management channel. A browser console may also depend on credentials or provider access you haven’t tested yet.

If SSH is your only access method and you have no reliable recovery option, postpone high-risk changes until you have a workable recovery plan.

For broader change planning, read Before You Change a Linux Server: 10 Checks to Avoid Downtime.

4. Check Your Existing SSH Configuration First

Before making restrictions, identify how SSH currently works.

On many Ubuntu and Debian installations, the relevant service is ssh.service. On many RHEL-family systems, it is sshd.service.

Start with read-only checks:

systemctl status ssh.service --no-pager

systemctl status sshd.service --no-pager

sudo ss -lntp

You may find that only one of the service names exists. Some systems also use systemd socket activation, so inspect the applicable SSH service and socket arrangement before changing ports or service behavior.

Next, examine the effective SSH settings:

sudo sshd -T | grep -E \
'^(passwordauthentication|kbdinteractiveauthentication|pubkeyauthentication|permitrootlogin) '

The sshd -T command displays effective server configuration values after parsing the main configuration and included files.

Pay attention to:

If the server uses conditional Match blocks, the effective settings can differ by user, source address or other connection properties. Review those rules and use the appropriate sshd -T -C connection parameters when needed.

For a deeper explanation of SSH keys and authentication, see SSH Keys Explained: How Secure Server Access Works.

5. Set Up SSH Keys Before Disabling Passwords

SSH public-key authentication lets your client prove possession of a private key without transmitting that private key to the server.

The public key is installed for the account you want to access. The matching private key remains on your computer.

This section assumes you have an existing account authorized for SSH access and a working way to install its public key.

If your only login is root, establish and test a non-root administrative account before disabling root access. User creation and sudo configuration depend on the distribution and your organization’s access policy.

ssh-keygen -t ed25519 \
  -f ~/.ssh/id_ed25519_sa_vps

ssh-copy-id \
  -i ~/.ssh/id_ed25519_sa_vps.pub \
  YOUR_ADMIN_USER@YOUR_SERVER_IP

ssh \
  -o IdentitiesOnly=yes \
  -o PreferredAuthentications=publickey \
  -o PasswordAuthentication=no \
  -o KbdInteractiveAuthentication=no \
  -o ControlMaster=no \
  -o ControlPath=none \
  -i ~/.ssh/id_ed25519_sa_vps \
  YOUR_ADMIN_USER@YOUR_SERVER_IP

whoami

sudo -v

6. Harden SSH Authentication Carefully

Once key-based administration is working, you can consider restricting other login methods.

A common policy for a simple server administered through SSH keys is:

PubkeyAuthentication yes
PasswordAuthentication no
KbdInteractiveAuthentication no
PermitRootLogin no

These are SSH server configuration directives, not shell commands.

They are not appropriate for every environment. For example, disabling keyboard-interactive authentication may break an existing MFA system or other authentication workflow. Do not apply this policy without first verifying all legitimate access requirements.

Where should these settings go?

On Ubuntu, OpenSSH commonly reads /etc/ssh/sshd_config and snippets under /etc/ssh/sshd_config.d/.

However, OpenSSH generally uses the first obtained value for many configuration directives. A setting in an earlier included file may take precedence over one you add later. Conditional Match blocks introduce additional considerations.

Therefore, do not blindly append these settings to the end of a file and assume they will take effect.

Inspect your existing files, make a backup, and edit the appropriate global configuration location so that it produces the intended effective settings.

Validate the configuration before applying it

sudo sshd -t

If the command reports a configuration error, do not reload or restart SSH.

When validation succeeds, inspect the effective configuration again:

sudo sshd -T | grep -E \
'^(passwordauthentication|kbdinteractiveauthentication|pubkeyauthentication|permitrootlogin) '

For environments using conditional Match rules, validate the relevant user and connection context rather than relying on the global output alone.

A successful syntax test establishes that SSH can parse the configuration. It does not guarantee that you will still be able to connect over the network.

Apply the change only when ready

On an Ubuntu installation with an active ssh.service that supports reload, you can typically use:

sudo systemctl reload ssh.service

Other distributions and SSH service arrangements may require a different procedure. Check the active service and socket configuration, particularly on systems using SSH socket activation.

Keep your original terminal open and immediately test a new SSH connection from your computer.

If the new connection fails, use the still-working session or your verified recovery console to investigate and restore the previous configuration.

7. Firewall Rules: The Most Common Remote Lockout Trap

Strong SSH authentication is important, but it doesn’t decide which remote clients can reach the service.

A firewall can enforce network restrictions independently of SSH authentication.

That is valuable, but it creates another way to accidentally block yourself.

Suppose your server currently accepts SSH traffic from any internet address. You decide to restrict it to your own public IP.

If you accidentally enter the wrong address, your new connections may be rejected.

If your internet connection uses a changing public IP, a rule that works today may prevent access after your address changes.

Inspect the current firewall first

On Ubuntu servers using UFW:

sudo ufw status verbose

sudo ufw status numbered

If UFW is not installed or isn’t the active firewall manager, these commands may not describe your actual filtering policy.

You may also have firewall rules configured in your cloud provider’s control panel.

Changing one layer does not automatically update the others.

Review source restrictions

UFW supports rules allowing connections from specific source addresses.

For example, the following command previews a possible rule for TCP port 22:

sudo ufw --dry-run allow proto tcp \
  from YOUR_TRUSTED_PUBLIC_IP \
  to any port 22

Replace YOUR_TRUSTED_PUBLIC_IP with the correct external source address before attempting to apply such a rule.

The address must represent the source seen by the server. Depending on your setup, that could be your home connection, office gateway or VPN exit point.

If the preview is appropriate and you have verified a recovery method, the corresponding rule can be added with:

sudo ufw allow proto tcp \
  from YOUR_TRUSTED_PUBLIC_IP \
  to any port 22

This example assumes SSH really uses TCP port 22. If it uses another port, adapt the rule accordingly.

Adding an allow rule does not necessarily make the firewall active, remove existing broader allow rules, or guarantee that the intended policy takes effect.

Check the resulting configuration and account for rule ordering, existing policies and provider-level filtering.

What about enabling UFW?

For a server where UFW is currently disabled, sudo ufw enable is a consequential operation. It may affect established SSH sessions and other services.

Do not run it as an isolated first step.

Before activation, you must know which inbound connections the server requires, configure the appropriate rules, confirm how IPv4 and IPv6 are handled, and have an independently tested recovery option.

For example, a public website may need HTTP and HTTPS access in addition to your restricted SSH management path.

Only enable the firewall when that full policy has been reviewed, and verify the outcome immediately from a new external connection.

See the official Ubuntu firewall documentation and Linux Firewall Explained for the underlying concepts.

8. Don’t Forget IPv6 and Provider Firewalls

Many VPS environments support IPv6 alongside IPv4.

Restricting one address family does not automatically mean the other is equally protected.

Similarly, a cloud firewall may be configured independently of UFW or another host firewall.

The goal is not to install every possible firewall tool. It’s to understand the controls that are actually active and avoid conflicting or poorly understood policies.

9. A Safe Change Timeline: What Good Hardening Looks Like

Let’s follow a hypothetical administrator securing a newly deployed VPS.

These times are illustrative, not results from a live experiment.

The lesson is not that server hardening must take exactly 35 minutes.

It’s that each step has a clear purpose, an expected result and a point where you stop if something doesn’t work.

10. What Should You Do If a New SSH Connection Fails?

The first question is whether your original session still works.

If it does, keep it open. Do not make several additional changes at once.

Start by identifying where the new connection fails.

If you retain a working administrative session, inspect the last change before doing anything else.

For example, after changing SSH settings, you can run:

sudo sshd -t

sudo sshd -T | grep -E \
'^(passwordauthentication|kbdinteractiveauthentication|pubkeyauthentication|permitrootlogin) '

For firewall changes, inspect the relevant host and provider policies.

If no SSH session remains available, use the verified recovery method rather than repeatedly guessing passwords or changing unrelated settings.

11. Does Changing the SSH Port Improve Security?

You may have heard that changing SSH from port 22 to another port makes a server safer.

Moving SSH can reduce some indiscriminate scans targeting the default port. But it does not fix weak authentication, outdated software, compromised credentials or unnecessary privileges.

It also adds another possible lockout condition.

You must coordinate the listening port, host firewall, provider firewall and client configuration.

On some Ubuntu installations using systemd socket activation, SSH listening-port changes involve additional socket configuration behavior rather than simply editing one directive and restarting the service.

For beginners, changing the SSH port should not be the first hardening priority.

Verified key-based access and correctly scoped network exposure provide a stronger starting point.

12. What About Fail2ban and Automated Blocking?

Tools such as Fail2ban can react to repeated authentication failures and temporarily restrict selected sources.

They can be useful, but they add another policy layer that requires configuration, maintenance and testing.

They may also block legitimate users if thresholds are poorly chosen or if repeated errors occur behind a shared address.

Don’t use automated blocking as a replacement for strong authentication.

First establish reliable SSH access, understand your logging sources, and confirm the firewall behavior.

Then evaluate automated defenses based on the needs of your server and the operational risks they introduce.

13. A Practical Public Server Hardening Checklist

After completing your changes, review the following conditions.

These checks establish a safer administrative baseline. They are not a guarantee that the server has no vulnerabilities or that every security risk has been eliminated.

Continue reviewing updates, exposed applications, user permissions and backups as part of routine maintenance.

For a broader assessment, see Linux Server Security Audit Checklist: 15 Checks to Find Hidden VPS Risks.

14. Frequently Asked Questions

15. You Have Completed the Public Linux Server Course

Over three lessons, we’ve followed a Linux server through three connected questions.

First, we learned why receiving a public IP address can make network services discoverable even without a published domain.

Second, we investigated unfamiliar SSH authentication attempts and learned why a rejected login is not proof of compromise.

Finally, we’ve developed a safer way to protect remote access: establish recovery options, verify authentication, change access rules deliberately, and test the result.

The core principle is simple:

Make your server harder for unauthorized clients to access — without making it impossible for authorized administrators to manage.

Primary Technical Sources

• Ubuntu Server — OpenSSH Server — SSH key configuration, testing, and server administration.
• OpenSSH — sshd_config(5) — Authentication directives and configuration precedence.
• OpenSSH — sshd(8) — Server validation and effective configuration testing.
• Ubuntu Server — Firewall Documentation — UFW rules and source restrictions.
• Ubuntu Manual — ufw(8) — Firewall commands and remote management considerations.