![[Steve Friedl Logo]](/images/unixwiz-logo-140x80.gif){kind=logo}
This site uses advanced css techniques
![[OpenSSH logo]](../images/openssh.gif)
With the Core-SDI Advisory on the
SSH1 CRC-32 compensation attack detector vulnerability and the associated
exploit running around in Fall 2001, many people are rushing to upgrade their
vulnerable secure shell servers. The exploit appears to be easy to use and
quite effective, rooting boxes right and left.
We were among those rushing around upgrading systems, and it was quite a chore. So we've written down some rough instructions that report how we did it on our own systems. These are not intended to be tutorial, but they might give somebody a start. These instructions only apply to UNIX systems, and in particular we have installed on Red Hat Linux 6.2 and Solaris 2.7.
And we wish to note that this tip was created years ago, and much has changed since then even though the fundamentals should be familiar. Always check your local documentation to see what's different in the version you're on now.
There are three separate packages that have to be installed, and each is found in a different place throughout the World Wide Web.
We typically do our work in a /home/source area, so the unpacking will look like this:
# cd /home/source # gtar -xzvf zlib-1.1.4.tar.gz # gtar -xzvf openssl-0.9.7a.tar.gz # gtar -xzvf openssh-3.5p1.tar.gz
The three packages have to be built in order, and we do them each in turn:
# cd zlib-1.1.4 # ./configure # make # make install # cd ../openssl-XXXXX # ./config Note: not './Configure' # make # make test # make install # cd ../openssh-XXXXX # ./configure --with-ipv4-default --with-md5-passwords # make # no "make install" yet!
We chose the arguments to ./configure by feeling around, and we think this is the right set for our environment. These instructions have been working for us on a number of systems.
--with-ipv4-default appears to tell SSH to use only IPv4 -- "regular" internet addresses -- instead of the next-generation IPv6 addresses. We're not sure if this is required or not.
--with-md5-passwords enables the better password hashing as found in Linux, and it probably isn't required on other systems or if you only use RSA authentication.
We suspect that --with-pam might also be useful, but so far the system is working fine for us without it.
NOTE - for recent releases of OpenSSH, this is where the sshd user and group should be added to support privilege separation. See the end of this document for the details.
If this is a first-time installation of the Secure Shell on this system, it's safe to just do a make install in the OpsnSSH area and it will drop its files in the proper places.
But if there is a working SSH installation then it's wise to go find the key setup files and save them off to the side first. This installation also parks files in different target directories than the older one we used, so it's wise to poke around and save or delete the old stuff. We typically look in these directories:
/etc /usr/local/bin /usr/local/sbin
Pay particular note to the ssh_config and sshd_config files, which define important server and client options. Save these files to the side.
It's also wise to keep an alternate method of accessing the system, such as the console or telnet, until you're really sure that you have gotten SSH installed properly. It's a drag to find (for instance) that sshd doesn't start properly on a reboot when you have no keyboard or monitor attached to the server under the desk. Been there, done that.
Once these precautions have been taken, do a make install in the SSH source directory, and it will put everything where it belongs.
Once SSH is installed, there are still a few configuration steps required. We're really not terribly strong with this, and in any case cannot provide a tutorial for SSH here. But we will note a few things that ought to be attended to after upgrading secure shell.
Insure that sshd gets launched properly at system startup. In our installation, the daemon is located in /usr/local/sbin instead of /usr/local/bin, and the /etc/rc.d/init.d/sshd script has to be modified to reflect the proper location.
The file we use lives in /etc/rc.d/init.d/sshd, and it contains:
#
# secure shell daemon
#
# chkconfig: 2345 90 05
# description: secure shell daemon
#
SSHD=/usr/local/sbin/sshd
[ -x $SSHD ] || exit 0
case "$1" in
start) echo "Starting secure shell daemon"
eval $SSHD
;;
esac
Once this file is in place (in /etc/rc.d/init.d/sshd), it has to be linked into the proper positions in the other runlevels so it actually: gets called. This is most easily done with the chkconfig program that manages the startup and shutdown locations in the various run levels.
# cd /etc/rc.d/init.d # chmod +x sshd # chkconfig --add sshd
If you prefer to do it yourself,
# cd /etc/rc.d # chmod +x init.d/sshd # ln -s ../init.d/sshd rc2.d/S90sshd # ln -s ../init.d/sshd rc3.d/S90sshd # ln -s ../init.d/sshd rc5.d/S90sshd
Important - do not forget the chmod step: the Secure Shell daemon won't start unless the init.d/sshd file is executable, and we've been burned by this more than once. It really stinks to be unable to get into your system remotely after a reboot because you forgot to chmod a file.
Make sure the sshd_config file is set correctly. In our installation we found that the file was installed to /usr/local/etc instead of /etc or /etc/ssh/, and that none of our customizations in the old version migrated to the new. In particular, we're careful to turn off PasswordAuthentication, though we are not sure this is a great idea if this is the first time you're using Secure Shell.
But in any case, be sure to disable the SSH version 1 protocol by setting Protocol 2 in the file. SSH1 is not secure and can be the source of all kinds of shenanigans.
We are not excited about allowing PasswordAuthentication for our secure shell daemons, because this provides limited system protection. It's true that it does forestall network sniffing -- and this is no small matter -- but it does not stop the bad guy from running a password guesser over the secure channel. In our work as penetration testers, we've gotten into systems via secure shell this way: by guessing or finding a password.
Much better is to use Public Key encryption, which has the potential for substantially better security as well as more convenience. We've been surprised how much we like public key support in SSH - much more than we expected.
In PK, each user creates a pair of keys: one public and one private, and the two are mathematically related. Both are very long strings of binary numbers, and they bear no relation to anything like a "password" or anything else human readable. The public key can be given away, but the private key is so important that it is further protected by a "pass phrase". This pass phrase actually encrypts the private key, and you'll need to type this key every time you need to use it.
The pass phrase should be something easy to type but impossible to guess: you'll be typing it a lot. At first it seems annoying to type this same phrase over and over, but quickly found that typing one long phrase for everything positively beats the pants off having to remember many individual passwords for all the sites we visit. This has turned out to be a big win.
Public key support revolves around the .ssh/ subdirectory in each user's UNIX/Linux home directory, and a couple of important files live here. These include (but not an exhaustive list):
IMPORTANT - Nothing will work right unless the ownership and permissions are set correctly. The directory and all files must be owned by the user in question, and must be unreadable by anybody but the user. The ssh server will simply ignore files that don't meet this stringent rule - it's a good security precaution. Just remember to chmod og= ~/.ssh ~/.ssh/* every now and then to be sure.
The ssh-keygen program is used to create a public key pair suitable for OpenSSH client and server use (there are other mechanisms for creating keys, we'll touch on the m later). It's possible to create both "user" and "host" keys, but since the host keys are usually created during the software installation process, we'll only touch on the user key.
There are many options for creating public key pairs, but we only care about making SSH version 2 RSA keys. This command creates the key pair, prompts for a pass phrase to encrypt the private key, and stores the two files in your ~/.ssh directory:
$ ssh-keygen -b 1024 -t rsa
-b 1024 specifies the number of bits to encrypt the key, and the consensus is that 1024 is fine: larger values just slow things down without adding appreciably to security. -t rsa specifies to use RSA encryption to produce an SSH version 2 key (the default is to create a version one key - bad news). It prompts for a filename to store the keys in (usually OK to keep the default), then asks for a pass phrase twice.
Choose a good pass phrase! Your private key is not very private if the very secret id_rsa file can be compromised.
Running the command should look like this:
$ ssh-keygen -b 1024 -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/home/friedl/.ssh/id_rsa): RETURN Enter passphrase (empty for no passphrase): passphrase + RETURN Enter same passphrase again: passphrase + RETURN Your identification has been saved in /home/steve/.ssh/id_rsa. Your public key has been saved in /home/steve/.ssh/id_rsa.pub. The key fingerprint is: 67:2c:06:ff:5a:cf:36:64:11:20:74:90:9f:97:14:8e steve@unixwiz.net
Now the files ~/.ssh/id_rsa.pub and ~/.ssh/id_rsa contain your public and private keys, respectively. We'll use them shortly. The public key file is one line of ASCII, and it looks like this (though it's actually all one line):
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEAx9aYnr7XwNw0Ei3X2ngxRjGg kQpK/EqpkveGCvvbMoH84zqu3Us8jSC0D392JZAEohGSoe0dWMBFm9Y41VGZ YmncwkfTQPFH1P0IvDw49aTAa2RJNFyVQANZCbSocDeTT0QpusuUj/v8hj7+ PqsUUpUXVQSDIhXBkWV+bJawc1c= steve@unixwiz.net
Again, there is no need to keep a public key secret.
When the ssh server is considering an incoming connection request via the public key mechanisms, it consults the ~/.ssh/authorized_keys for version-2 keys (we're specifically ignoring version-1 key support).
To enable a remote user to the local OpenSSH server, simply edit the user's ~/.ssh/authorized_keys file and append the user's public key -- one line -- to the file and save it. Do make sure that the file is owned by the user and is not readable by anybody else, but there is no need to restart the OpenSSH server to make it take effect.
Then, from the remote system, connect with the ssh command and the name of the remote host.
$ ssh www.example.com The authenticity of host 'www.example.com (10.1.1.1)' can't be established. RSA key fingerprint is 50:29:ac:84:60:42:31:f7:e0:4c:bc:d2:ff:de:ba:89. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'www.example.com,10.1.1.1' (RSA) to the list of known hosts. Enter passphrase for key '/home/steve/.ssh/id_rsa': passphrase + RETURN $ uname -a Linux www.example.com 2.2.12-20 #1 Mon Sep 27 10:25:54 EDT 1999 i586 unknown $
The first time you contact any host, you'll be warned that your local SSH installation doesn't know about the remote. When you confirm that you want to connect to it with yes, the name and fingerprint of the remote host is added to the ~/.ssh/known_hosts file. Then you won't be prompted again for this host.
After confirming the host, you'll enter your passphrase to decrypt your private key. If done correctly, the SSH client and server exchange your key information, and if all is well you're granted access to the remote system.
There is no reason that a single public key can't be put in multiple user accounts on the remote system: it's possible to store your public key in both /home/steve/.ssh and /root/.ssh directories. So logged in as a regular user, we can login as root on the remote system:
$ ssh -l root www.example.com
We use the fantastic telnet/serial/SSH client SecureCRT (from VanDyke Software) to connect from our Windows (ME/NT/2000) boxes to OpenSSH, and setup is straightforward.
After SecureCRT is installed on the Windows box, a pair of public/private keys must be created. From the menu, select Options:Global Options, and from this dialog box select SSH2, then click Create Identity File. Key creation is done just one time after installation. NOTE: these instructions are for SecureCRT v4.x.
Click Next in the Key Generation Wizard, then select an RSA - not DSA - key and click Next. In the next dialog box, enter your pass phrase (twice), then a comment that appears in the file. By convention this is your email address, but it can be any descriptive text that helps you identify the key. Then click Next.
In the next dialog box, select 1024 bits for the key and click Next. Then you are invited to move the mouse around: this provides important random inputs that help generate a key pair that will be hard to crack. This takes perhaps 30 seconds, and when finished you are asked to click Next.
In the final dialog box, you're asked to save your key files somewhere. It's not as important exactly where they go as long as you remember where that place is. You will need the public key file shortly. For our example we are using C:\identity.pub.
After the key has been generated, locate the public key file and get it over to the UNIX system. Since this file is small and ASCII, it's often easiest to open it in notepad, select the whole thing, and "cut" it into the Windows notepad. Then, logged into the remote server (perhaps with telnet or via ssh using a password), and pasting it into a file. For instance:
$ cat > /tmp/identity.pub "paste" the file here ^D <-- control-D $
Remember that the entry is one line, so you might need to use your favorite editor and join them together if pasting created multiple lines.
Once the file is over on the UNIX system, it has to be converted from the SecureCRT format (some kind of standard) to that used by OpenSSH. The ssh-keygen command is able to do this conversion for you:
$ ssh-keygen -i -f /tmp/identity.pub >> ~/.ssh/authorized_keys
Now your SecureCRT public key is installed, so you should be able to ssh into your UNIX box from Windows.
Once the public key mechanisms are known to be working correctly, including testing of proper restart after reboot, we recommend some additional steps.
#telnet stream tcp nowait root /usr/sbin/tcpd in.telnetd
# ps -fCinetd UID PID PPID C STIME TTY TIME CMD root 413 1 0 Feb01 ? 00:00:00 [inetd] # kill -1 413
We're quite sure that other options in the config file are important, but we've not really looked at them in much detail. Shame on us.
Once OpenSSH has been installed on a platform, it's reasonable to want to install it elsewhere without having to actually build it. So we've noted the files that are installed by the make install process so that they can be copied (with permissions!) to some other system. This listing is relative to /usr/local.
$ gtar -tzvf OpenSSH-3.1p1z.tgz -rws--x--x root/root 211396 2002-03-13 01:32:57 bin/ssh -rwxr-xr-x root/root 26520 2002-03-13 01:32:57 bin/scp -rwxr-xr-x root/root 75564 2002-03-13 01:32:57 bin/ssh-add -rwxr-xr-x root/root 42428 2002-03-13 01:32:57 bin/ssh-agent -rwxr-xr-x root/root 79640 2002-03-13 01:32:57 bin/ssh-keygen -rwxr-xr-x root/root 136784 2002-03-13 01:32:58 bin/ssh-keyscan -rwxr-xr-x root/root 46216 2002-03-13 01:32:58 bin/sftp lrwxrwxrwx root/root 0 2002-03-13 01:32:59 bin/slogin -> ssh -rwxr-xr-x root/root 230416 2002-03-13 01:32:58 sbin/sshd -rw-r--r-- root/root 182 2002-03-13 02:31:06 sbin/generate-ssh-keys -rw-r--r-- root/root 46525 2002-03-13 01:32:58 man/man1/ssh.1 -rw-r--r-- root/root 3261 2002-03-13 01:32:59 man/man1/scp.1 -rw-r--r-- root/root 4896 2002-03-13 01:32:59 man/man1/ssh-add.1 -rw-r--r-- root/root 6315 2002-03-13 01:32:59 man/man1/ssh-agent.1 -rw-r--r-- root/root 10081 2002-03-13 01:32:59 man/man1/ssh-keygen.1 -rw-r--r-- root/root 3890 2002-03-13 01:32:59 man/man1/ssh-keyscan.1 -rw-r--r-- root/root 7129 2002-03-13 01:32:59 man/man1/sftp.1 lrwxrwxrwx root/root 0 2002-03-13 01:32:59 man/man1/slogin.1 -> ssh.1 -rw-r--r-- root/root 43258 2002-03-13 01:32:59 man/man8/sshd.8 -rw-r--r-- root/root 2087 2002-03-13 01:32:59 man/man8/sftp-server.8 -rwxr-xr-x root/root 23712 2002-03-13 01:32:58 libexec/sftp-server -rw-r--r-- root/root 568 2002-03-13 01:32:56 share/Ssh.bin -rw-r--r-- root/root 88039 2002-03-13 01:32:59 etc/moduli -rw-r--r-- root/root 1144 2002-03-13 01:32:59 etc/ssh_config -rw-r--r-- root/root 2523 2002-03-13 01:32:59 etc/sshd_config
The only nonstandard file is sbin/generate-ssh-keys, which is a small shell script we have created: it simply creates the three key pairs that are required. Run this program one time after installation to create these keys.
The SSH daemon requires its own set of host keys, and these are created as part of the installation process. But when the binaries are copied to another machine, the keys must be generated manually. As with all keys, these are created with the ssh-keygen program, and the three sets of keys are created with:
# ssh-keygen -t rsa1 -f /usr/local/etc/ssh_host_key -N "" # ssh-keygen -t dsa -f /usr/local/etc/ssh_host_dsa_key -N "" # ssh-keygen -t rsa -f /usr/local/etc/ssh_host_rsa_key -N ""
![[O'Reilly SSH book]](../images/sshcover.gif)
This entire Tech Tip is very lame in comparison with the excellent O'Reilly book SSH, The Secure Shell: The Definitive Guide (2e), which covers everything in much more detail. Once you've gotten a handle on the basic configuration of OpenSSH, we recommend picking up this book to get to the finer points. We'll update our document as we pick them up ourselves.
There has been a problem reported in the ZLIB compression library that has prompted version 1.1.4. We've installed this and found it easy to do.
Unpack, build, and install the source:
# cd /home/source # gtar -xzvf zlib-1.1.4.tar.gz # cd zlib-1.1.4 # ./configure # make install
Edit the OpenSSH version file. In the OpenSSH directory is a file version.h, and we want to remind ourselves that we've made this fix even though we haven't actually patched OpenSSH directly. So we tack a z to the end:
#define SSH_VERSION "OpenSSH_3.0p1z"
Rebuild OpenSSH. We first do a "make clean" to force the changes to take effect:
# vi version.h # make clean # make
The resulting sshd file is left in the current directory.
Install sshd. Doing make install updates everything, and we're not sure this is really necessary. So we just manually copy the file to the final place. Then we look for the "parent" SSH daemon (the one with the PPID of 1) and send it the hangup signal. Then restart the daemon.
# strip sshd # cp sshd /usr/local/sbin # ps -fCnamed UID PID PPID C STIME TTY TIME CMD root 4509 1 0 07:46 ? 00:00:00 /usr/local/sbin/sshd root 4510 4509 0 07:46 ? 00:00:00 /usr/local/sbin/sshd root 4534 4509 0 07:49 ? 00:00:00 /usr/local/sbin/sshd # kill -1 4509 # /usr/local/sbin/sshd
We've managed to build OpenSSH on several other platforms with more or less the same instructions for Linux, but a few notes might be helpful here.
# cd openssh-3.2.3p1 (with gcc) # ./configure --with-ipv4-default
# mkdir -m 700 /var/empty # chown sshd /var/empty # chgrp sshd /var/empty
# cd openssh-3.2.2p1
# ./configure --with-ipv4-default --with-pam
Starting with OpenSSH 3.3.2, the privilege separation feature was introduced. This is a security provision that goes a long way to forestall full system compromise even if a vulnerability is found. The details are covered elsewhere, and our favorite is here.
But before SSHD is installed, both a sshd user and group must be added, and it looks in the password and group files like this:
/etc/group sshd:*:27: /etc/passwd sshd:*:27:27:sshd privsep:/var/empty:/sbin/nologin
Under Red Hat Linux 7.2, these commands do the trick:
# groupadd sshd # useradd -M -g sshd -c 'sshd privsep' -d /var/empty -s /sbin/nologin sshd # passwd -l sshd
The -M parameter suppresses creation of the home directory, as this is done by the "make install" procedure for OpenSSH.
Note further that the UsePrivilegeSeparation yes option must be set in the sshd_config file - this may be the default for new installations, but upgrades don't usually touch current values. Make sure this option is set.
Feedback on this document is very much welcomed by ![[Email address]](../images/steve-email.gif){kind=small}
.
![[Secure CRT]](../images/scrt-1.gif)
![[Steve's Email]](/images/steve-email.gif)
![[RSS Feed available]](/images/feed-icon-14x14.png){kind=icon}