Do you really know ssh?

Ask any system administrator if he knows SSH and you will hear definitive answer “of course”. But is that really true? I know it is not.

Many system administrators do not have extensive knowledge of ssh client and server applications. They do not know how to tunnel connections, use socks proxy, use ssh jump servers (in config), properly use SCP or even run control commands for ssh client (did you know that you can close malfunction ssh client with control command “~.” ?).

Debugging ssh

Actually, I know that there is a huge number of problems why someone cannot log in to your server. Some of the usuals are locked account, incorrect rights of keys or problems with SELinux.

Junior system administrator will try to look into logs but will usually find nothing. Definitely, not everyone knows that you can run SSHD daemon in debug mode.

Pitfalls? There are no settings to this mode in the config file, you have to run it manually and watch the output on the console. But what are your options when you are connected to a remote server in a datacenter and wants to debug why some user cannot connect to it?

There is a small hack. You can run manually SSHD daemon on a different port and still be connected to a server, it will look like this. Good luck with this tip!

~$ sudo /usr/sbin/sshd -p 2000 -D -d
debug1: sshd version OpenSSH_7.2, OpenSSL 1.0.2g 1 Mar 2016
debug1: private host key #0: ssh-rsa SHA256:oxy9lIjUjqFEAZFeNvevBdza4WkN4lBlUO6ooWgNsAg
debug1: private host key #1: ssh-dss SHA256:91xThrZINLqnbj5voj4LA9NOqEoIRRHNemyeGaq78bU
debug1: private host key #2: ecdsa-sha2-nistp256 SHA256:zl+p1pJVJICs7MrM1EXDbCvGeF7HK4DclLgkHNmQlJo
debug1: private host key #3: ssh-ed25519 SHA256:DBM9227SKZ0S0mGOdcSEfcvBhXtw44IR2GdrM/PfEhw
debug1: rexec_argv[0]='/usr/sbin/sshd'
debug1: rexec_argv[1]='-p'
debug1: rexec_argv[2]='2000'
debug1: rexec_argv[3]='-D'
debug1: rexec_argv[4]='-d'
debug1: Set /proc/self/oom_score_adj from 0 to -1000
debug1: Bind to port 2000 on 0.0.0.0.
Server listening on 0.0.0.0 port 2000.
debug1: Bind to port 2000 on ::.
Server listening on :: port 2000.