Secure SSH access with certificate based authentication and firewall rules

When I set up a new Linux server, I usually take couple of steps to secure the way I connect to the box using SSH. In this post I will walk you through the process to secure SSH access using firewall rules, changing the SSH port and enabling certificate based authentication. In this example I used a CentOS 7 box.

In this example, I used a CentOS 7 based Linux box using the minimal x64 iso image with the latest updates applied.

Tasks

The suggested way to secure ssh access using certificate-based authentication consists of the following steps:

• Install OpenSSH server
• Harden Linux Box
• Change ssh port
• Create a new user
• Generate public and private keys on client machine
• Disable port 22
• Deactivate password based login

Some of the steps described below are either executed on the server or on a client computer. To avoid confusion, I have added the following markers to indicate where a command has to be executed: Server Client

Install OpenSSH server

Server

The first step is to install OpenSSH server on the Linux server. In my case, openssh server was already pre-installed, but in case other Red Hat based Linux distributions require OpenSSH server to be installed, the following command can be used:

[user@server]$ sudo yum install openssh-server

Harden Linux server

Server

In this step we will improve the general security of the Linux server, by installing and configuring the firewall, closing all not required ports and changing the default SSH port.

The last step might be questionable, as it is quite easy to identify the correct ssh port using port scanning techniques. I think it is still a recommended step to enhance security at least a little bit. Additionally, it could be the first line of defense for simple attacks.

Install firewalld

Server

Let's first install the CentOS 7 default firewall (firewalld).

[user@server]$ sudo yum install firewalld

Create a service specification

Firewalld organizes the rules to be applied using service specification files. These are human-readable XML files. The example below shows the SSH service specification (available at this location: /usr/lib/firewalld/services/ssh.xml).

<?xml version="1.0" encoding="utf-8"?>
<service>
  <short>SSH</short>
  <description>Secure Shell (SSH) is a protocol for logging into and executing commands on remote machines. It provides secure encrypted communications. If you plan on accessing your machine remotely via SSH over a firewalled interface, enable this option. You need the openssh-server package installed for this option to be useful.</description>
  <port protocol="tcp" port="22"/>
</service>

To use a different port for ssh we have to create a custom ssh rule. This can be done by copying the existing ssh.xml file to the /etc/firewalld/services/ directory, as shown below

[user@server]$ sudo cp /usr/lib/firewalld/services/ssh.xml /etc/firewalld/services/ssh_custom.xml

Let's edit this file with

[user@server]$ sudo vi /etc/firewalld/services/ssh_custom.xml

Here I changed the port to 2222 and changed the short name to SSH-Custom. Feel free to choose whatever port you want, but I would recommend not to use port numbers smaller than 1024. In case you change the port to a different number please change it in all other steps accordingly.

<?xml version="1.0" encoding="utf-8"?>
<service>
  <short>SSH-Custom</short>
  <description>Secure Shell (SSH) is a protocol for logging into and executing commands on remote machines. It provides secure encrypted communications. If you plan on accessing your machine remotely via SSH over a firewalled interface, enable this option. You need the openssh-server package installed for this option to be useful.</description>
  <port protocol="tcp" port="2222"/>
</service>

Reload service specifications

Server

We have to start or restart firewalld to reload the service specifications.

[user@server]$ sudo service firewalld restart

Then we have to check if our new ssh_custom service specification is available.

[user@server]$ sudo firewall-cmd --get-services

In my case, the command returns the following list and we can see that the new service specification ssh_custom is available.

RH-Satellite-6 amanda-client amanda-k5-client bacula bacula-client bitcoin bitcoin-rpc 
bitcoin-testnet bitcoin-testnet-rpc ceph ceph-mon cfengine condor-collector ctdb 
dhcp dhcpv6 dhcpv6-client dns docker-registry dropbox-lansync elasticsearch 
freeipa-ldap freeipa-ldaps freeipa-replication freeipa-trust ftp ganglia-client 
ganglia-master high-availability http https imap imaps ipp ipp-client ipsec 
iscsi-target kadmin kerberos kibana klogin kpasswd kshell ldap ldaps libvirt 
libvirt-tls managesieve mdns mosh mountd ms-wbt mssql mysql nfs nrpe ntp openvpn 
ovirt-imageio ovirt-storageconsole ovirt-vmconsole pmcd pmproxy pmwebapi pmwebapis 
pop3 pop3s postgresql privoxy proxy-dhcp ptp pulseaudio puppetmaster quassel 
radius rpc-bind rsh rsyncd samba samba-client sane sip sips smtp smtp-submission 
smtps snmp snmptrap spideroak-lansync squid ssh ssh_custom synergy syslog 
syslog-tls telnet tftp tftp-client tinc tor-socks transmission-client vdsm 
vnc-server wbem-https xmpp-bosh xmpp-client xmpp-local xmpp-server

Now we can add the service specification - that means the actual firewall rule - permanently.

[user@server]$ sudo firewall-cmd --permanent --add-service=ssh_custom

The command should print out 'success'. To apply the new rule we have to restart firewalld.

[user@server]$ sudo service firewalld restart

Let's check if the new rule has been applied using this command:

[user@server]$ sudo firewall-cmd --list-services

In my case, the output is shown below. At this point in time both firewall rules are applied, the original ssh and the new ssh_custom rule. It is important that the original ssh rule stays active until OpenSSH server has been reconfigured to use the new port. Otherwise, the firewall would block all traffic on port 22 which doesn't allow us to connect through ssh anymore.

ssh dhcpv6-client ssh_custom

Change the SSH port

Server

After changing the firewall rules we can finally reconfigure OpenSSH to use the new port 2222. But before we can change the port, another package has to be installed. This package is required to notify the SELinux system that the port has been changed.

[user@server]$ sudo yum install policycoreutils-python

Create backup of sshd_config file

Server

Additionally, before we change the configuration, a backup of the original configuration file should be created.

[user@server]$ sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.backup

Change the ssh port

Server

Edit the OpenSSH configuration file.

[user@server]$ sudo vi /etc/ssh/sshd_config

Then uncomment the Port parameter and change the value to 2222.

# If you want to change the port on a SELinux system, you have to tell
# SELinux about this change.
# semanage port -a -t ssh_port_t -p tcp #PORTNUMBER
#
Port 2222
#AddressFamily any

Notify SELinux system about the change

Server

Then we have to notify the SELinux system that we changed the ssh port.

[user@server]$ sudo semanage port -a -t ssh_port_t -p tcp 2222

Restart OpenSSH server

Server

And finally, we have to restart OpenSSH server to apply the change.

[user@server]$ sudo service sshd restart

Check if the new port is being used

Client

Let's quickly check if the new port is actually being used. In a client computer terminal try the following command - of course using the IP address of your Linux server - here I will use 192.168.1.73.

[user@client]$ ssh -p 2222 user@192.168.1.73

You should be able to connect through port 2222, while port 22 cannot be used anymore. The firewall doesn't block traffic through port 22, but there is simply no service listening to port 22.

Configure the port on your client

Client

In case you don't like to provide the port using parameter "-p" every time to connect through ssh, you can create a configuration file on your client computer.

[user@client]$ vi ~/.ssh/config

For each host you can specify the port you want to use separately, as shown below

Host 192.168.1.73
    Port 2222

You can check if the configuration works as expected by trying to connect to the server using without providing the "-p" parameter.

[user@client]$ ssh user@192.168.1.73

Create a new user

Server

To further improve the security of our Linux server, I would recommend creating another user. This user will be used to connect to the Linux server through ssh, but doesn't belong to the sudoers group. For this example, I will use a user named git.

[user@server]$ sudo useradd git
[user@server]$ sudo passwd git

Now let's switch to the new user and create the ~/.ssh directory, a file named ~/.ssh/authorized_keys and change the permissions as required

[user@server]$ su git
Password:
[git@server]$ mkdir ~/.ssh
[git@server]$ touch ~/.ssh/authorized_keys
[git@server]$ chmod 700 ~/.ssh
[git@server]$ chmod 600 ~/.ssh/authorized_keys

The ~/.ssh/authorized_keys file will later be used to store the public certificates that this service will trust.

Generate public and private keys on the client machine

Client

We are getting closer to the final solution. To allow a client to communicate with a server, based on certificates, without the need of entering a password, we have to create the corresponding private and public key on the client computer.

[user@client]$ ssh-keygen -t rsa

The output of the command is shown below. Please provide a file name to store the private key. I will use git-test_rsa as an example. You can enter a passphrase, that is required every time the private key is being accessed - if you want. But it depends on your specific use case - basically if you need this additional layer of security or not. I will not use a passphrase to avoid entering it every time I want to access the key.

[user@client]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/user/.ssh/id_rsa): .ssh/git-test_rsa
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in git-test_rsa.
Your public key has been saved in git-test_rsa.pub.
The key fingerprint is:
SHA256:DvvaoUfjvq1pYItMqCM6caKwaANWlHohOK2nwk0fxAc user@host
The key's randomart image is:
+---[RSA 2048]----+
|..  oE.          |
|o..+ o .         |
| o+ o .          |
|...+ .           |
|.o=.. o S        |
|Boo...o+o        |
|B* o o.=o.       |
|Oo  o .+++       |
|=o.   o=Bo.      |
+----[SHA256]-----+

Executing the command results in two files:

• git-test_rsa.pub: This is the public key - sometimes also called public certificate. This file will later be copied to the Linux server.
• git-test_rsa: This is the private key. Access to this file has to be restricted to the user that owns the key. This file is the secret and should never be copied to a different location or server!

Copy the public key to the Linux server

Client

In this step, we are going to copy the public key to the Linux server.

[user@client]$ scp ~/.ssh/git-test_rsa.pub git@192.168.1.73:/home/git

Add the public key to the authorized_keys file

Server

Next, we log in to the Linux server - using the new user git - and add the public key to our authorized_keys file.

[git@server]$ cat ~/git-test_rsa.pub >> ~/.ssh/authorized_keys 

Check if certificate-based authentification works

Client

The final step is to check if the certificate-based authentication actually works. Simply try to login from your client computer to the Linux server using the command we already know.

[user@client]$ ssh git@192.168.1.73

If everything has been set up correctly, you will not be required to enter the password of the git user. The certificate is sufficient to authenticate you.

Disable port 22

Server

After you ensure that you are able to connect through port 2222 we can reconfigure the Linux firewall to also block traffic for port 22. Log in to the Linux server and enter the following commands.

[user@server]$ sudo firewall-cmd --permanent --remove-service=ssh
[user@server]$ sudo service firewalld restart

Disable password based login

Server

Despite the fact that certificate-based authentication works as expected, it is still possible to login to the Linux server using a password. The final step to secure your Linux server is, to disable password based ssh logins. This is done by editing the sshd_config file.

[user@server]$ sudo vi /etc/ssh/sshd_config

Simply set the PasswordAuthentication configuration parameter to no.

PasswordAuthentication no

To apply the new configuration the sshd service has to be restarted.

[user@server]$ sudo service sshd restart

That's it! After applying all the steps you already have a Linux box with a pretty high level of security.

Here are the key points:

• We only allow users, that have the private key, that matches the public key stored in the authorized_keys file.
• In our setup, the user, that is allowed to connect, doesn't belong to the group of privileged users (sudoers).
• We changed the ssh port to a non-standard port.
• Only ssh traffic through the new port is being allowed, all other traffic is being blocked.

Of course, there are other ways to improve the security, e.g. installing a simple intrusion detection system - like fail2ban. Maybe, I will cover additional topics in future posts.