While we can now log in with our SSH key, we are still allowing logins via password. A chain is only as strong as its weakest link, and a system is only as secure as its least secure component. Therefore, now that we can log in using SSH, it's best to disable login via password.
Double-check that you are able to log in to your server using your SSH key before disabling password-based authentication; otherwise, you'll be locked out of the server.
On the remote virtual server, open up the configuration file for the SSH daemon at /etc/ssh/sshd_config (note that this is not the same as /etc/ssh/ssh_config, which is the configuration file for the SSH client). Search for an entry called PasswordAuthentication and set it to no:
PasswordAuthentication no
Again, reload the SSH daemon to ensure that it is updated with our changes:
root@hobnob:# systemctl reload ssh.service