Does this site look plain?

This site uses advanced css techniques

Systems exposed to the internet are heavily challenged to keep the bad guys out, and keeping up with the latest security patches is not always easy. So, the wise admin will attempt to institute systemic steps to limit the damage should a compromise occur, and one excellent method is the use of a chroot() jail.

Table of Contents

A chroot jail presents a dramatically restricted view of the filesystem to an application, and usually far fewer system privileges, and this all intends to limit the damage should the application go awry or be subverted by the bad guy.

This document touches on how chroot works and discusses some best practices that developers and administrators can use to make their installations more secure.


Background on chroot

The chroot system call changes the root directory of the current and all child processes to the given path, and this is nearly always some restricted subdirectory below the real root of the filesystem. This new path is seen entirely as "/" by the process, and we refer to this restricted environment as the "jail". It's not possible to escape this jail except in very limited circumstances.

The chroot system call is found in all versions of UNIX that we know of, and it serves to create a temporary root directory for a running process, and it's a way of taking a limited hierarchy of a filesystem (say, /chroot/named) and making this the top of the directory tree as seen by the application.

How to break out of jail

There are well-known techniques used to escape from jail, but the most common one requires root privileges inside the jail. The idea is for the program to do a chroot to a subdirectory, leaving the current directory outside the jail.

We'll add more notes on ways to break out of a jail - which is meant more to show what must be protected against than it is as a how-to for jailbreakers -- but we've found a good article on chroot in general here.

Almost all jail breaking requires root privileges.

General chroot principles

We have presented these in no particular order, and no one site will use them all. In particular, some tips apply to developers at the source code level, while others apply to administrators trying to jail an existing system.

Many of these points may end up being overly petty in practice, in that there are only so many layers of defense that a workable system can use, but we'll present all we can think of and let you pick and choose. An overriding principle is "What if the bad guy somehow does X? How can we limit our exposure".

Our general concern is mostly about remote buffer overflows, and this can give the bad guy complete control over our CPU: all our steps are designed to limit the damage should this unfortunate circumstance arise.

Run in the jail as a non-root user
A chroot jail is not impervious to escape, but it not easy and requires root permission in the jail itself, so we must take steps to limit this possibility. By running the jail as a non-root user, it's as secure as we know how to make it. It may be necessary for the daemon to launch as root in order to do a few tasks that require these permissions (say, binding to a low-numbered port), but the program must "give up" its root permissions after doing so.
We believe that this single factor is the most important one in setting up a jail properly.
"Give up" permissions correctly
We've seen situations where on some operating systems, a program can jump back and forth between a non-root user and root by use of a "saved" uid, and this has been exploited by the bad guy who get root.
The details of how to do this correctly are much more tricky when OS differences are taken into account: the variants are setresuid() seteuid(), setreuid(), and setuid() — it's likely that this does not exhaust the options. The right one depends on the OS you're running.
The best resource by far we've found on this is the outstanding Usenix 2002 paper Setuid Demystified, by Hao Chen, David Wagner, and Drew Dean: it is precisely on point, and we'll direct the reader to section 5.2 "Comparison among Uid-setting System Calls".
Explicitly chdir into the jail
The chroot call itself does not change the working directory, so if the new root is below the current directory, the application can still have access outside resources.
The application should explicitly change to a directory within the jail before running chroot: 
...
chdir(dir);
chroot(dir);
setXXuid(nonroot);      // give up root permissions correctly.
...
This closes a trivial escape route from the jail (but we'll note that you must use the proper setuid-esqe calls as noted in the previous item).
An alternate order of chdir and chroot: 
...
chroot(dir);
chdir("/");
...
appears to be equivalent.
Keep as little in the jail as possible
This limits what can be compromised should a vulnerability be discovered. Often this requires development support to do some "preloading" of non-jailed files before the chroot operation itself is performed (we'll touch on this a bit more later). But we're quite ruthless in removing things from the jail when possible.
Limit non-jail running of jailed binaries
For systems that do not have a command-line option for running chroot, the only alternative is to create a wrapper program. This wrapper will perform the key chroot operation, give up root permission, and then execute the jailed binary.
The wrapper must be run as root (only chroot can perform this operation), but the wrapper itself must not be found in the jail. Otherwise an intruder could quietly compromise the wrapper, and the next time the system is launched, the intruder's program would be run as root in a non-jailed environment. This is complete compromise.
Have root own as many jailed files as possible
This limits the ability of the intruder to make changes should a compromise occur. Our feeling is that the most likely cause of penetration will be the buffer overflow exploit in which the intruder executes arbitrary code in environment, and for files that the jailed system need not ever write to, making them readonly and owned by root means that the penetration can't chmod the file before writing to it. This rule applies to directories as well.
Drastically limit all permissions of files and directories
Our feeling is that if a permission bit is not required, it should not be set. For instance, the jailed "/dev/" directory should be of mode d--x--x--x with owner = root. Even though the only thing in the directory is /dev/null, forbidding searching of a directory strikes us as prudent practice across the board when it's known to work.
Create a permissions-setting script
When first setting up the jail, many of the permission-related knobs are tweaked by hand as we gradually tighten things up, looking for things to break (at which point the knob is eased back a bit). This research is intricate, and the knowledge gained really ought to be represented in source code somewhere.
We typically create a small shell script — living outside the jail — that sets the owner, group, and permissions mode on every file in the jailed environment. It always starts with a few recursive change-everything options to hardcode everything to very tight permissions, then relaxes the settings on the files that can tolerate this. It's important to include documentation in the script on why particular permissions are relaxed, as well as describing why certain files are found in the jail in the first place.
Once this script is created, we typically make all of our permissions-related changed here and then re-run the script to make them take effect. This is the only way that we can be sure that our script matches the running environment. A great side benefit of the permission script is that it serves as documentation to the next person setting up a similar environment.
A sample permission script that we use for one of our projects (running BIND in a chroot jail). The specific details aren't really important, but this gives an idea 
cd /chroot/named

# by default, root owns /everything/ and only root can write
# but directories have to be executable too.

chown -R root.named .

find .         -print | xargs chmod u=rw,og=r      # *all* files
find . -type d -print | xargs chmod u=rwx,og=rx    # directories

# the "secondaries" directory is where we park files from
# master nameservers, and named needs to be able to update
# these files and create new ones.

find conf/secondaries -type f -print | xargs chown named.named
find conf/secondaries -type f -print | xargs chmod ug=r,o=
chown root.named conf/secondaries
chmod ug=rwx,o=   conf/secondaries

# the var/run business is for the PID file
chown root.root  var
chmod u=rwx,og=x var

chown root.named  var/run
chmod ug=rwx,o=rx var/run
Try to do the chroot operation inside the daemon itself
... rather than rely on the explicit chroot command (this requires source code modifications). A daemon that has its own internal chroot can often park the executable located outside the jail: this is a big win because an intruder is not able to ever infect the binary directly.
But the more immediate benefit is that shared libraries and other startup files can be automatically loaded from the full system and need not be located inside the jail. This not only makes the system safer — less exposure to the outside — but also makes it easier to set up.
In many cases, even configuration files can be loaded from outside the jail, though this won't usually work if the daemon includes any kind of "reread config files" option.
Preload dynamically loaded objects
For developers adding chroot support to programs, consider operations that require access to full-system resources and perform them before closing the jail door. These steps are often not entirely obvious at first and require some trial and error, but we've found several that qualify.
Many systems load nameservice resolver clients dynamically at runtime, and they are not included in the shared objects bound to the executables. We have found that simply calling gethostbyname one time before the jail door is closed will load all the appropriate libraries required, so that later nameservice requests are handled properly: 
(void) gethostbyname("localhost");
We believe that syslogging operations fall in this category too, as many systems uses UNIX domain sockets for this and require access to the socket that syslogd is listening on. We've not done the modifications required for syslog support and cannot offer any specific suggestions. We believe that Solaris -- with its use of "doors" -- is an added complication.
For daemons that permit cmdline parameters to select the runtime users and group (after giving up root), the mapping of name to UID and GID must be done before the chroot operation so that the system-wide /etc/passwd and related files are used, not the one inside the jail. See the next section for the rationale.
This bit of C code shows the idea of how the user lookup should be performed separately from the user ID changing: 
if ( geteuid() == 0 )
{
struct passwd *userent = 0;

    if ( (run_as_user != 0) && (userent = getpwnam(run_as_user)) == 0 )
    {
        /* ERROR */
    }

    chroot( working_dir );

    if ( userent )
        setXXuid(userent);    // use the proper call!

    ...
Avoid using the jailed /etc/passwd file
...particularly for name to UID mapping used to determine the runtime user ID of the daemon. The mapping involves scanning the passwd file for the given name (say, named) and finding the user ID associated with it. If the bad guy somehow manages to compromise the jailed passwd file, it's possible that the UID for the runtime user could be changed to zero, which is root. This will take effect the next time the daemon restarts.
The bad guy shouldn't be able to compromise this file in the first place, because it should not be writable by the running user, but it's not out of the question that the daemon could somehow retain a writable file descriptor that the buffer overflow could use to modify the file: we believe we have seen this happen before. As is so common, a bug in one area of the system can have surprising impacts on security.
Close file descriptors aggressively before chrooting
We don't wish to leave handles open to non-jailed resources because these can all be exploited by those living inside the jail. Some file descriptors are required (say, to the syslog daemon), but developers should make a point to close anything that is not strictly required.
Update 7/2013: a great resource for this can be found here. (tip o' the hat to Tim Kuijsten for this)
Link config files from the outside
Some systems (such as BIND) share the configuration file between the jailed daemon and other utilities that are run from user mode. In this case, the config file simply must live inside the jail so that the daemon can access it, but the other utils from user mode still need to access this file. Rather than rebuild these utilities to use the special path (say, /chroot/named/etc/named.conf), instead go to the "regular" place for this file and create a symbolic link from the outside to the inside of the jail: 
# ln -s /chroot/named/etc/named.conf /etc/named.conf
This allows most of the tools to operate "normally", though one has to be a little more careful that users editing /etc/named.conf realize that they're affecting a jailed system.
This doesn't go the "other" direction, though it's not always obvious at first. Symbolic links from inside the jail to the outside will work for the administrator but will not work for the system running inside the jail.
Update environment variables to reflect the new root
Once in the jail, some environment variables might need to be updated to reflect the new jail-ness of the application. In particular, many shells maintain a $PWD variable reflecting the current working directory, and the getcwd(3) library function consults it as an optimization step.
Calling putenv("PWD=/") after the successful chrooting synchronizes this variable with the new directory.
Other environment variables might need to be altered (or removed) depending on how much it matters that environment leakage from the non-jailed environment might impact the jailed one.

Other Resources