GRUB

The first modifications that I needed to make were in GRUB. On CentOS the GRUB config file can be found at: /boot/grub/menu.lst. Below you’ll find the changes I made:

default=0
timeout=5
#splashimage=(hd0,0)/grub/splash.xpm.gz
serial --unit=0 --speed=115200
terminal --timeout=10 console serial
hiddenmenu

I’m sure if you compare that snippet with your current configuration you’ll see the changes that I’ve implemented. Primarily the commenting of the splashimage, and the addition of the serial and terminal lines.

In addition to these changes you’ll also need to append some console configurations to each ‘kernel’ line. Here is an example:

title CentOS (2.6.18-164.15.1.el5)
root (hd0,0)
kernel /vmlinuz-2.6.18-164.15.1.el5 ro root=/dev/SLIM/root console=tty0 console=ttyS0,115200n8


Again, compare this to your current configuration. You’ll notice I’ve added console=tty0 console=ttyS0,115200n8

/etc/inittab

Finally you’ll need to make a change to the /etc/inittab file, which will allow login via the serial console connection:

S0:2345:respawn:/sbin/agetty ttyS0 115200 linux


Once these changes have been applied you should be able to reboot and connect to the console:

virsh console <guest> should then display the GRUB menu, boot output and finally a login.

The solution I found was a tool called “JailKit“, which allows you to create chroot jails is a very simple way. Below I’ll outline the requirements for installing JailKit and using it to create an SSH chrooted “Jail”.

Step 1: Install JailKit

I was unable to find an up to date package for JailKit on CentOS, so I did the following to install from source:

yum install gcc make
wget -c http://olivier.sessink.nl/jailkit/jailkit-2.7.tar.bz2
tar xf jailkit-*.tar.bz2
cd jailkit-*
./configure
make
make install


Step 2: Configuration

Once you have JailKit built and installed you’ll need to configure it to provide the needed libraries and devices for your chroot SSH setup. Below is an example configuration I used for a JailKit setup on CentOS 5 x86_64 (using JailKit 2.5 at the time). Note: This is not the entire configuration. These three sections, uidbasics, ssh and basicshell required minor changes and this was the end-product of those changes:

[uidbasics]
comment = common files for all jails that need user/group information
libraries = /lib/libnsl.so.1, /lib64/libnsl.so.1, /lib/libnss*.so.2, /lib64/libnss*.so.2
regularfiles = /etc/nsswitch.conf
emptydirs = /home

[ssh]
comment = ssh secure shell
executables = /usr/bin/ssh
includesections = netbasics, uidbasics
devices = /dev/urandom, /dev/tty, /dev/null

[basicshell]
comment = bash based shell with several basic utilities
executables = /bin/sh, /bin/bash, /bin/ls, /bin/cat, /bin/chmod, /bin/mkdir, /bin/cp, /bin/cpio, /bin/date, /bin/dd, /bin/echo, /bin/egrep, /bin/false, /bin/fgrep, /bin/grep, /bin/gunzip, /bin/gzip, /bin/ln, /bin/ls, /bin/mkdir, /bin/mktemp, /bin/more, /bin/mv, /bin/pwd, /bin/rm, /bin/rmdir, /bin/sed, /bin/sleep, /bin/sync, /bin/tar, /bin/touch, /bin/true, /bin/uncompress, /bin/zcat, /usr/bin/id
regularfiles = /etc/motd, /etc/issue, /etc/bash.bashrc, /etc/bashrc, /etc/profile
directories = /usr/lib/locale/en_US.utf8
users = root
groups = root
includesections = uidbasics


Step 3: Populate The Jail Environment

Once these changes are made JailKit should be configured to provide the needed libraries and devices to your SSH chroot. You can now generate the chroot by using these two commands:

jk_init -v -f /home/chrootusers/ ssh
jk_init -v -f /home/chrootusers/ basicshell


Step 4: Generate Users

The final step in an SSH-chroot Jail setup is the creation of users. This step is a bit more complicated due to the fact that the users need to be created within the chroot setup and on the host system itself. The users will never have access to the host system, but that is where the initial authentication happens. Once they are authenticated they are pushed into the chroot jail and have limited permissions.

This script assumes two things that you should be aware of. If your configuration doesn’t match you’ll need to update the script accordingly:

1. This script assumes your chrooted jail home directories will be in /home/chrootusers/.
2. This script assumes group-based SSH restrictions using a system group called sshaccess.
3. This script also attempts to create storage restrictions using filesystem quotas, but succeeds if quotas are not enabled. (See Appendix for information on enabling quotas.)

Here is the script:

#!/bin/bash
#
# This code is released under the Public Domain.
# create jailed ssh accounts
# christer.edwards@gmail.com
# usage: ./chroot-jail username 'passwdhash'
#

## sanity checks
if [ $# -ne 2 ]; then
echo "This application requires two arguments: username 'passwdhash' (single-quotes required)"
exit 1
fi

if [ ! -d /home/chrootusers/./home/ ]; then
echo "This application assumes installation to /home/chrootusers"
exit 1
fi

getent passwd $1 &>/dev/null
if [ $? -eq 0 ]; then
echo "User $1 already exists on host system. Please remove and re-create using this tool."
exit 1
fi

getent group sshaccess &>/dev/null
if [ $? -ne 0 ]; then
echo "This application assumes 'AllowGroups sshaccess' restriction in /etc/ssh/sshd_config. sshaccess group missing"
exit 1
fi

## creation
echo
echo "Creating User and Jailed Home Directory"
echo
useradd -m -d /home/chrootusers/./home/$1 -s /usr/sbin/jk_chrootsh -p $2 $1

echo "Adding User to 'sshaccess' Group"
echo
gpasswd -a $1 sshaccess &>/dev/null

JAILUID=$(id -u $1)
JAILGID=$(id -g $1)

echo "Creating Required Internal Jail Passwd and Group Files"
echo
echo "$1:x:${JAILUID}:${JAILUID}::/home/$1:/bin/bash" >> /home/chrootusers/etc/passwd
echo "$1:x:${JAILGID}:" >> /home/chrootusers/etc/group

echo "Applying Quota Restrictions"
setquota -u $1 10240 15360 0 0 /home || echo "Unable to Apply Quota Restrictions. Are Quotas enabled?"


If you create all of your users using this script you should save yourself a fair amount of headache. Between the user-creation script and the JailKit tools, creating SSH chrooted jails becomes a very simple task.

Appendix: Enabling Quotas

When I first implemented this JailKit setup in production I invited a few of the users to do their worst to break out of the jail or otherwise interfere with the system and other users. One of the potential problems that we found was that of drive space. Because there were no storage limitations for the chrooted users, one user could create large files and interfere with the others (or the host system) by filling up the drive. Below I’ve outlined how to activate filesystem quotas on Red Hat Enterprise or CentOS machines and apply a 15M hard limit to your chrooted users:

Configure usrquota

Quotas are a filesystem-based feature that needs to be activated within the fstab. To activate the use of persistent quotas append the term “usrquota” to the mount options of /home/ in your fstab. Example:

/dev/LocalVol/HOME    /home    ext3    defaults,usrquota    1 2

Activate usrquota

Once this is configured the following four commands will install, configure and activate quotas.

yum install quota
mount -o remount /home/
quotacheck -cmu /home/
quotaon /home/

At this point you aren’t applying any limits to the user (yet), but the quota management system is on and active. If you create your users using the script above they will be given a 10M soft limit (after 10M they will be given a warning) and a 15M hard limit (absolutely no more disk writes). You can change these values (last line in the script) based on your needs.

Mrepo is a tool developed by Dag Wieers of rpmforge fame. If you’ve ever run an RPM based distribution (Red Hat, Fedora, CentOS, etc) you’ve very likely heard of Dag Wieers. He keeps the package machine running and provides those packages not otherwise available in the core repositories. I read that he developed this tool to help streamline the creation and deployment of RPM mirrors and network install servers, which he so often needed in dealing with clients.

I used this exact setup to create and maintain an internal repository for our company which supplies network-based installation capabilities for Red Hat Enterprise 3, 4 and 5, CentOS 3, 4 and 5, Fedora 10 and 11, and openSUSE. This allows us to more quickly deploy new servers as well as saving bandwidth when applying errata to hundreds of servers and desktops.

Before I dive into the nuts and bolts of this operation there are a few things that are assumed with this setup. You should be aware of these so that you know what to expect and/or you can update your preferences accordingly.

1. The export and storage location I prefer is /var/ftp/pub, with the DVD images stored in a subdirectory of /var/ftp/pub/.iso/. This allows client access via FTP without any FTP server customizations, but also HTTP by way of symbolic links and NFS with basic configuration.
2. The export location also takes into consideration the fact that many people put /var/ on its own partition. A mirror will require decent amount of space, and this allows it to be separate from the rest of the file system.
3. If you want your mirror available via FTP, HTTP and NFS you’ll need to install the vsftpd and httpd packages.
4. Steps 3 and 4 are only required if you plan on mirroring Red Hat Network errarta. This requires a valid RHN account and available entitlement!
5. The mrepo wiki page on my personal wiki is available to the public for additional reference and discussion.

With those details out of the way let’s get started!

Step 1: Build and Install The Package

The latest codebase for mrepo is not available in package format (at least that I was able to find), so we’ll have to build the package from svn. I have done this a few times and it is very painless. Simply follow the below commands and you’ll end up with an installed mrepo RPM.

mkdir -p ~/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
svn co http://svn.rpmforge.net/svn/trunk/tools/mrepo/ ~/mrepo-0.8.6svn/
tar cjvf ~/rpmbuild/SOURCES/mrepo-0.8.6svn.tar.bz2 ~/mrepo-0.8.6svn/
echo "%_topdir ~/rpmbuild/" > ~/.rpmmacros
cd ~/mrepo-0.8.6svn/
rpmbuild -ba mrepo.spec
yum localinstall ~/rpmbuild/RPMS/noarch/mrepo-0.8.6svn*.noarch.rpm

If everything goes according to plan you should end up with an mrepo-0.8.6svn*.noarch.rpm in your ~/rpmbuild/RPMS directory, and it should also be installed. If you don’t have these, or you ran into other problems leave a comment and we’ll see what we can figure out.

Step 2: Create The Public Directory Structure

I’m guessing the reason that you want to create and export an RPM repository is that you want to share packages with others, or perhaps you want the ability to do network-based installations. This means you’ll want to make this content public. You can choose a different location for your export if you’d like, but I prefer to put my content in /var/ftp/pub/, so here are the directories I’d create if I wanted to make available a CentOS, Red Hat Enterprise and Fedora repository, including the latest version and both major architectures.

mkdir -p /var/ftp/pub/.iso/{rhel,centos}5-{i386,x86_64}
mkdir -p /var/ftp/pub/.iso/fedora11-{i386,x86_64}

Step 3: Generate System ID For Use With RHN (Optional)

If you’re going to be exporting Red Hat Enterprise images and packages you’re going to need to generate a System ID for each release and architecture you want to gather updates for. This will allow your machine (even a non-Red Hat Enterprise machine) to connect to and gather updates from the Red Hat Network. Note: This does require a valid RHN username, password and entitlement!

gensystemid -u RHN_USERNAME -p RHN_PASSWORD --release=5Server --arch=i386 /var/ftp/pub/.iso/rhel5-i386
gensystemid -u RHN_USERNAME -p RHN_PASSWORD --release=5Server --arch=x86_64 /var/ftp/pub/.iso/rhel5-x86_64


Note: Be sure to change the paths above to match your own if you did not follow step two exactly.

Step 4: Minor Tweaks Needed if Run On CentOS (Optional)

I have found there are a few minor things that need to happen in order for a CentOS based machine to mirror updates from the Red Hat Network. They are fairly simple, but are required in order for the CentOS machine to properly and securely connect to the Red Hat Network. Note: If you are not planning on mirroring Red Hat Enterprise packages you can skip this step.

echo "up2date default" >> /etc/sysconfig/rhn/sources
echo "uuid[comment]=Universally Unique Identifier for this server" >> /etc/sysconfig/rhn/up2date-uuid
echo "rhnuuid=$(uuidgen)" >> /etc/sysconfig/rhn/up2date-uuid
mkdir /usr/share/rhn
cp RHN-CA-CERT /usr/share/rhn/

Note: The last command will not work out-of-the-box because the RHN-CA-CERT file is not found by default on a CentOS machine. You will need to copy this from a Red Hat Enterprise machine (located in /usr/share/rhn/RHN-CA-CERT) and place it on your CentOS machine.

Step 5: Download The .iso Images

We’re now ready to download and install images we’re going to want to export and use. Below are a few example commands for CentOS and Fedora. After these are downloaded be sure to place them in their appropriate directory, located in /var/ftp/pub/.iso/. The full path should match that found in step 6 below for “iso=”

CentOS 5.3:

wget -c http://mirrors.kernel.org/centos/5.3/isos/i386/CentOS-5.3-i386-bin-DVD.iso
wget -c http://mirrors.kernel.org/centos/5.3/isos/x86_64/CentOS-5.3-x86_64-bin-DVD.iso


Fedora 11:

wget -c http://mirrors.kernel.org/fedora/releases/11/Fedora/i386/iso/Fedora-11-i386-DVD.iso
wget -c http://mirrors.kernel.org/fedora/releases/11/Fedora/x86_64/iso/Fedora-11-x86_64-DVD.iso

Your milage may vary with those mirrors, but the basic idea is to download (or if you already have downloaded, copy) the DVD images and place them into your previously created export directory of /var/ftp/pub/.iso/{$distro$releasever-$basearch}.

Step 6: Create The Config File

We’re almost done! I hope you’ve stayed with me so far. The last thing we need to do before we can start downloading our updates is create the configuration file. This file will detail three main things:

1. Where are the DVD images we want to export?
2. Where do we want to export them?
3. What mirror(s) do we want to pull our updates/errata from?

Once we’ve defined these items we’ll be ready to kick off mrepo and populate our mirror.

The mrepo package that we built earlier should come with a default config file. To make things simple we’ll just customize the existing file and then we’ll be ready to go. The config file can be found at /etc/mrepo.conf, and here is an example config with entries for CentOS, Fedora and Red Hat Enterprise:

[main]
srcdir = /var/ftp/pub/.iso
wwwdir = /var/ftp/pub
confdir = /etc/mrepo.conf.d
arch = i386 x86_64
mailto = root@localhost
smtp-server = localhost
rhnlogin = username:password

[centos5-i386]
name = CentOS $release ($arch)
release = 5.3
arch = i386
iso = /var/ftp/pub/.iso/centos5-i386/CentOS-$release-$arch-bin-DVD.iso
updates = rsync://mirrors.kernel.org/centos/$release/updates/i386/

## centos 5 x86_64
[centos5-x86_64]
name = CentOS $release ($arch)
release = 5.3
arch = x86_64
iso = /var/ftp/pub/.iso/centos5-x86_64/CentOS-$release-$arch-bin-DVD.iso
updates = rsync://mirrors.kernel.org/centos/$release/updates/x86_64/

## rhel 5 i386
[rhel5-i386]
name = Red Hat $release Server ($arch)
release = 5.3
arch = i386
iso = /var/ftp/pub/.iso/rhel5-i386/rhel-$release-server-$arch-dvd.iso
updates = rhns:///rhel-$arch-server-5

## rhel 5 x86_64
[rhel5-x86_64]
name = Red Hat $release Server ($arch)
release = 5.3
arch = x86_64
iso = /var/ftp/pub/.iso/rhel5-x86_64/rhel-$release-server-$arch-dvd.iso
updates = rhns:///rhel-$arch-server-5

## fedora 11 i386
[fedora11-i386]
name = Fedora $release ($arch)
release = 11
arch = i386
iso = /var/ftp/pub/.iso/fedora11-i386/Fedora-$release-$arch-DVD.iso
updates = rsync://mirrors.kernel.org/fedora/updates/$release/$arch/

## fedora 11 x86_64
[fedora11-x86_64]
name = Fedora $release ($arch)
release = 11
arch = x86_64
iso = /var/ftp/pub/.iso/fedora11-x86_64/Fedora-$release-$arch-DVD.iso
updates = http://mirrors.kernel.org/fedora/updates/$release/$arch/

Once you have made the needed changes to your configuration you’re ready to kick off the mirror and export process. Based on what we have in the configuration the next step will do the following:

• Loop-Mount and export the DVD images to /var/ftp/pub/. It will create subdirectories using the bracketed title we used in the config file (ie; /var/ftp/pub/fedora11-x86_64/)
• Mirror the public updates contents locally from the locations defined by “updates =”
• Create two subdirectories within our wwwdir path for RPMS.os and RPMS.updates.

Step 7: Create The Repositories

If you’ve made it through each of the previous steps without hitting any roadblocks you’re now ready to initiate your mirror creation.

mrepo -guvvv

Note: The command above suggests very verbose output (vvv) for the first run. This way you can see what it is doing and the progress it is making. For future runs you can omit the verbose options.

Installation

yum install tftp-server

DBAN Configuration

After following my previous instructions on how to extract the DBAN PXE images, copy the two files (kernel.bzi and initrd.gz) to the /tftpboot/ directory.

mkdir -p /tftpboot/dban
cp kernel.bzi /tftpboot/
cp initrd.gz /tftpboot/


PXE Configuration

You’ll also need to configure the PXE system to know what to launch/run when a specific command is given at boot.  In this case we’ll only have the DBAN option, which we could run by using the ‘dban’ command:

/tftpboot/pxelinux.cfg/default

display msgs/boot.msg
prompt 1
## dban
label dban
kernel dban/kernel.bzi
append load_ramdisk=1 initrd=dban/initrd.gz root=/dev/ram0 init=/rc quiet nuke=”dwipe” –autonuke –method dodshort”

The first line in that file, “display” tells it to display a file as a menu prompt at boot time.  The second step in this section is creating that menu prompt:

/tftpboot/msgs/boot.msg

**WARNING** TYPE ‘dban’ TO AUTO-WIPE THE PXE BOOTED MACHINE.  **WARNING**

Start The Service

Now that we have everything configured we’ll go ahead and start the service:

service xinetd restart