Zelut’s Blog

I’ve been spending a lot of time lately researching FreeBSD security solutions. Topics such as pf (packet filter), system hardening, etc. One of the tips that I thought I’d share here is a method of configuring your system to use blowfish encryption for passwords as opposed to the default MD5. If you are on a system that has a lot of users it might be prudent to update the encryption type to make your stored passphrase safer.

Configure /etc/login.conf

The password format is stored within the /etc/login.conf file. The default value being “md5″. To update your configuration to use blowfish instead make the following change to your file:

-       :passwd_format=md5:\

+       :passwd_format=blf:\

After you’ve made this change you’ll need to rehash the login database, which is done using the cap_mkdb command:

cap_mkdb /etc/login.conf

At this point any newly assigned passwords will be created using blowfish instead of MD5. Any existing accounts will retain their MD5 password hash until the password is reset.

I’m looking for some good pf (packet-filter) documentation if anyone has anything to suggest. I’ve added a second FreeBSD machine to my public-facing network, and may be adding a third soon. I want to make sure that my firewall is setup properly but I’m not very familiar with pf. Can anyone suggest a book, online document, etc that has good examples for setting up pf?

A friend of mine shot me this article today (What’s cooking for FreeBSD 8?), which is a pretty detailed outline of the things coming up for FreeBSD 8. I have to say, some of the things have got be excited! Some of the highlights that I want to see:

• Parallel Ports Build (announcement)
• Jails v2 (commit message)
• Xen Dom-U support (wiki page)

There is actually quite a bit more, but those are the immediate ones that look like they’d noticeably affect me. I’d really love to have FreeBSD setup on a VPS via Xen, with Jails for each of my services. Currently I have that setup at home (jails), but its on old hardware that takes forever to compile. Parallel Ports Build would sure be a nice improvement, assuming the hardware can handle it (which my current hardware cannot).

To any of you other FreeBSD users out there, what are you excited for in the list and why?

I don’t know how many times I have had to look up this information in the past. In hopes of learning-through-writing I’ve decided to publish these methods of recovering or resetting the MySQL root password.

Set Initial Password

If you’ve just installed MySQL and have never set the root password a password is not yet needed. Until one is defined you should be able to access your database(s) without the password. To set a password for the first time you can use:

mysqladmin -u root password NEWPASSWORD

Update Root Password
If you want to update or change the existing root password you can use:

mysqladmin -u root -p'oldpassword' password newpass

You should note that this will require that you know the current password.

Recover Lost Password

If you have completely lost the MySQL root password and need to reset it you can. This will require short downtime on your database(s). Below are the steps for:

1. stopping the service.
2. restarting with bypassed security.
3. logging into mysql without authentication
4. inserting a SQL statement, resetting the password.
5. restarting the database.

Stop the database using one of these commands (Linux vs FreeBSD installations):

/etc/init.d/mysql stop
/usr/local/etc/rc.d/mysql-server stop

Start MySQL in safe mode, bypassing security:

mysqld_safe --skip-grant-tables &

Login to MySQL as the root user. No authentication required:

mysql -u root

Provide the following SQL commands to reset the password. Change “NEWPASSWORD” to your password of choice:

mysql> use mysql;
mysql> update user set password=PASSWORD("NEWPASSWORD") where User='root';
mysql> flush privileges;
mysql> quit

Restart MySQL with new authentication in place:

/etc/init.d/mysql restart

I hope this not only helps me remember these steps but helps others who run into the same problem. I have written about Ubuntu specific instructions for resetting the mysql password. These instructions should be global to any distribution or MySQL installation.

Not long ago we rebuilt a number of servers within our production environment at work, migrating from FreeBSD to CentOS. One of the requirements in this migration was that we needed a server that would act as an SSH-only Jail on the new platform. This meant I needed to setup a system where users could SSH into a machine but the only thing they could do on that machine was SSH back out again. We used this as our “bounce” machine. Users would have outside access to their Desktops or other machines only by way of this secured SSH-only machine.

The solution I found was a tool called “JailKit“, which allows you to create chroot jails is a very simple way. Below I’ll outline the requirements for installing JailKit and using it to create an SSH chrooted “Jail”.

Step 1: Install JailKit

I was unable to find an up to date package for JailKit on CentOS, so I did the following to install from source:

yum install gcc make
wget -c http://olivier.sessink.nl/jailkit/jailkit-2.7.tar.bz2
tar xf jailkit-*.tar.bz2
cd jailkit-*
./configure
make
make install


Step 2: Configuration

Once you have JailKit built and installed you’ll need to configure it to provide the needed libraries and devices for your chroot SSH setup. Below is an example configuration I used for a JailKit setup on CentOS 5 x86_64 (using JailKit 2.5 at the time). Note: This is not the entire configuration. These three sections, uidbasics, ssh and basicshell required minor changes and this was the end-product of those changes:

[uidbasics]
comment = common files for all jails that need user/group information
libraries = /lib/libnsl.so.1, /lib64/libnsl.so.1, /lib/libnss*.so.2, /lib64/libnss*.so.2
regularfiles = /etc/nsswitch.conf
emptydirs = /home

[ssh]
comment = ssh secure shell
executables = /usr/bin/ssh
includesections = netbasics, uidbasics
devices = /dev/urandom, /dev/tty, /dev/null

[basicshell]
comment = bash based shell with several basic utilities
executables = /bin/sh, /bin/bash, /bin/ls, /bin/cat, /bin/chmod, /bin/mkdir, /bin/cp, /bin/cpio, /bin/date, /bin/dd, /bin/echo, /bin/egrep, /bin/false, /bin/fgrep, /bin/grep, /bin/gunzip, /bin/gzip, /bin/ln, /bin/ls, /bin/mkdir, /bin/mktemp, /bin/more, /bin/mv, /bin/pwd, /bin/rm, /bin/rmdir, /bin/sed, /bin/sleep, /bin/sync, /bin/tar, /bin/touch, /bin/true, /bin/uncompress, /bin/zcat, /usr/bin/id
regularfiles = /etc/motd, /etc/issue, /etc/bash.bashrc, /etc/bashrc, /etc/profile
directories = /usr/lib/locale/en_US.utf8
users = root
groups = root
includesections = uidbasics


Step 3: Populate The Jail Environment

Once these changes are made JailKit should be configured to provide the needed libraries and devices to your SSH chroot. You can now generate the chroot by using these two commands:

jk_init -v -f /home/chrootusers/ ssh
jk_init -v -f /home/chrootusers/ basicshell


Step 4: Generate Users

The final step in an SSH-chroot Jail setup is the creation of users. This step is a bit more complicated due to the fact that the users need to be created within the chroot setup and on the host system itself. The users will never have access to the host system, but that is where the initial authentication happens. Once they are authenticated they are pushed into the chroot jail and have limited permissions.

This script assumes two things that you should be aware of. If your configuration doesn’t match you’ll need to update the script accordingly:

1. This script assumes your chrooted jail home directories will be in /home/chrootusers/.
2. This script assumes group-based SSH restrictions using a system group called sshaccess.
3. This script also attempts to create storage restrictions using filesystem quotas, but succeeds if quotas are not enabled. (See Appendix for information on enabling quotas.)

Here is the script:

#!/bin/bash
#
# This code is released under the Public Domain.
# create jailed ssh accounts
# christer.edwards@gmail.com
# usage: ./chroot-jail username 'passwdhash'
#

## sanity checks
if [ $# -ne 2 ]; then
echo "This application requires two arguments: username 'passwdhash' (single-quotes required)"
exit 1
fi

if [ ! -d /home/chrootusers/./home/ ]; then
echo "This application assumes installation to /home/chrootusers"
exit 1
fi

getent passwd $1 &>/dev/null
if [ $? -eq 0 ]; then
echo "User $1 already exists on host system. Please remove and re-create using this tool."
exit 1
fi

getent group sshaccess &>/dev/null
if [ $? -ne 0 ]; then
echo "This application assumes 'AllowGroups sshaccess' restriction in /etc/ssh/sshd_config. sshaccess group missing"
exit 1
fi

## creation
echo
echo "Creating User and Jailed Home Directory"
echo
useradd -m -d /home/chrootusers/./home/$1 -s /usr/sbin/jk_chrootsh -p $2 $1

echo "Adding User to 'sshaccess' Group"
echo
gpasswd -a $1 sshaccess &>/dev/null

JAILUID=$(id -u $1)
JAILGID=$(id -g $1)

echo "Creating Required Internal Jail Passwd and Group Files"
echo
echo "$1:x:${JAILUID}:${JAILUID}::/home/$1:/bin/bash" >> /home/chrootusers/etc/passwd
echo "$1:x:${JAILGID}:" >> /home/chrootusers/etc/group

echo "Applying Quota Restrictions"
setquota -u $1 10240 15360 0 0 /home || echo "Unable to Apply Quota Restrictions. Are Quotas enabled?"


If you create all of your users using this script you should save yourself a fair amount of headache. Between the user-creation script and the JailKit tools, creating SSH chrooted jails becomes a very simple task.

Appendix: Enabling Quotas

When I first implemented this JailKit setup in production I invited a few of the users to do their worst to break out of the jail or otherwise interfere with the system and other users. One of the potential problems that we found was that of drive space. Because there were no storage limitations for the chrooted users, one user could create large files and interfere with the others (or the host system) by filling up the drive. Below I’ve outlined how to activate filesystem quotas on Red Hat Enterprise or CentOS machines and apply a 15M hard limit to your chrooted users:

Configure usrquota

Quotas are a filesystem-based feature that needs to be activated within the fstab. To activate the use of persistent quotas append the term “usrquota” to the mount options of /home/ in your fstab. Example:

/dev/LocalVol/HOME    /home    ext3    defaults,usrquota    1 2

Activate usrquota

Once this is configured the following four commands will install, configure and activate quotas.

yum install quota
mount -o remount /home/
quotacheck -cmu /home/
quotaon /home/

At this point you aren’t applying any limits to the user (yet), but the quota management system is on and active. If you create your users using the script above they will be given a 10M soft limit (after 10M they will be given a warning) and a 15M hard limit (absolutely no more disk writes). You can change these values (last line in the script) based on your needs.

Today I have finally found some time to catch up on my blogging. I’ve got a whole slew of backlogged information that I want to post, so hopefully my fingers (and my brain!) can keep up long enough to get it all published. The topic I wanted to start with today is mrepo: RPM Repository Mirroring made easy.

Mrepo is a tool developed by Dag Wieers of rpmforge fame. If you’ve ever run an RPM based distribution (Red Hat, Fedora, CentOS, etc) you’ve very likely heard of Dag Wieers. He keeps the package machine running and provides those packages not otherwise available in the core repositories. I read that he developed this tool to help streamline the creation and deployment of RPM mirrors and network install servers, which he so often needed in dealing with clients.

I used this exact setup to create and maintain an internal repository for our company which supplies network-based installation capabilities for Red Hat Enterprise 3, 4 and 5, CentOS 3, 4 and 5, Fedora 10 and 11, and openSUSE. This allows us to more quickly deploy new servers as well as saving bandwidth when applying errata to hundreds of servers and desktops.

Before I dive into the nuts and bolts of this operation there are a few things that are assumed with this setup. You should be aware of these so that you know what to expect and/or you can update your preferences accordingly.

1. The export and storage location I prefer is /var/ftp/pub, with the DVD images stored in a subdirectory of /var/ftp/pub/.iso/. This allows client access via FTP without any FTP server customizations, but also HTTP by way of symbolic links and NFS with basic configuration.
2. The export location also takes into consideration the fact that many people put /var/ on its own partition. A mirror will require decent amount of space, and this allows it to be separate from the rest of the file system.
3. If you want your mirror available via FTP, HTTP and NFS you’ll need to install the vsftpd and httpd packages.
4. Steps 3 and 4 are only required if you plan on mirroring Red Hat Network errarta. This requires a valid RHN account and available entitlement!
5. The mrepo wiki page on my personal wiki is available to the public for additional reference and discussion.

With those details out of the way let’s get started!

Step 1: Build and Install The Package

The latest codebase for mrepo is not available in package format (at least that I was able to find), so we’ll have to build the package from svn. I have done this a few times and it is very painless. Simply follow the below commands and you’ll end up with an installed mrepo RPM.

mkdir -p ~/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
svn co http://svn.rpmforge.net/svn/trunk/tools/mrepo/ ~/mrepo-0.8.6svn/
tar cjvf ~/rpmbuild/SOURCES/mrepo-0.8.6svn.tar.bz2 ~/mrepo-0.8.6svn/
echo "%_topdir ~/rpmbuild/" > ~/.rpmmacros
cd ~/mrepo-0.8.6svn/
rpmbuild -ba mrepo.spec
yum localinstall ~/rpmbuild/RPMS/noarch/mrepo-0.8.6svn*.noarch.rpm

If everything goes according to plan you should end up with an mrepo-0.8.6svn*.noarch.rpm in your ~/rpmbuild/RPMS directory, and it should also be installed. If you don’t have these, or you ran into other problems leave a comment and we’ll see what we can figure out.

Step 2: Create The Public Directory Structure

I’m guessing the reason that you want to create and export an RPM repository is that you want to share packages with others, or perhaps you want the ability to do network-based installations. This means you’ll want to make this content public. You can choose a different location for your export if you’d like, but I prefer to put my content in /var/ftp/pub/, so here are the directories I’d create if I wanted to make available a CentOS, Red Hat Enterprise and Fedora repository, including the latest version and both major architectures.

mkdir -p /var/ftp/pub/.iso/{rhel,centos}5-{i386,x86_64}
mkdir -p /var/ftp/pub/.iso/fedora11-{i386,x86_64}

Step 3: Generate System ID For Use With RHN (Optional)

If you’re going to be exporting Red Hat Enterprise images and packages you’re going to need to generate a System ID for each release and architecture you want to gather updates for. This will allow your machine (even a non-Red Hat Enterprise machine) to connect to and gather updates from the Red Hat Network. Note: This does require a valid RHN username, password and entitlement!

gensystemid -u RHN_USERNAME -p RHN_PASSWORD --release=5Server --arch=i386 /var/ftp/pub/.iso/rhel5-i386
gensystemid -u RHN_USERNAME -p RHN_PASSWORD --release=5Server --arch=x86_64 /var/ftp/pub/.iso/rhel5-x86_64


Note: Be sure to change the paths above to match your own if you did not follow step two exactly.

Step 4: Minor Tweaks Needed if Run On CentOS (Optional)

I have found there are a few minor things that need to happen in order for a CentOS based machine to mirror updates from the Red Hat Network. They are fairly simple, but are required in order for the CentOS machine to properly and securely connect to the Red Hat Network. Note: If you are not planning on mirroring Red Hat Enterprise packages you can skip this step.

echo "up2date default" >> /etc/sysconfig/rhn/sources
echo "uuid[comment]=Universally Unique Identifier for this server" >> /etc/sysconfig/rhn/up2date-uuid
echo "rhnuuid=$(uuidgen)" >> /etc/sysconfig/rhn/up2date-uuid
mkdir /usr/share/rhn
cp RHN-CA-CERT /usr/share/rhn/

Note: The last command will not work out-of-the-box because the RHN-CA-CERT file is not found by default on a CentOS machine. You will need to copy this from a Red Hat Enterprise machine (located in /usr/share/rhn/RHN-CA-CERT) and place it on your CentOS machine.

Step 5: Download The .iso Images

We’re now ready to download and install images we’re going to want to export and use. Below are a few example commands for CentOS and Fedora. After these are downloaded be sure to place them in their appropriate directory, located in /var/ftp/pub/.iso/. The full path should match that found in step 6 below for “iso=”

CentOS 5.3:

wget -c http://mirrors.kernel.org/centos/5.3/isos/i386/CentOS-5.3-i386-bin-DVD.iso
wget -c http://mirrors.kernel.org/centos/5.3/isos/x86_64/CentOS-5.3-x86_64-bin-DVD.iso


Fedora 11:

wget -c http://mirrors.kernel.org/fedora/releases/11/Fedora/i386/iso/Fedora-11-i386-DVD.iso
wget -c http://mirrors.kernel.org/fedora/releases/11/Fedora/x86_64/iso/Fedora-11-x86_64-DVD.iso

Your milage may vary with those mirrors, but the basic idea is to download (or if you already have downloaded, copy) the DVD images and place them into your previously created export directory of /var/ftp/pub/.iso/{$distro$releasever-$basearch}.

Step 6: Create The Config File

We’re almost done! I hope you’ve stayed with me so far. The last thing we need to do before we can start downloading our updates is create the configuration file. This file will detail three main things:

1. Where are the DVD images we want to export?
2. Where do we want to export them?
3. What mirror(s) do we want to pull our updates/errata from?

Once we’ve defined these items we’ll be ready to kick off mrepo and populate our mirror.

The mrepo package that we built earlier should come with a default config file. To make things simple we’ll just customize the existing file and then we’ll be ready to go. The config file can be found at /etc/mrepo.conf, and here is an example config with entries for CentOS, Fedora and Red Hat Enterprise:

[main]
srcdir = /var/ftp/pub/.iso
wwwdir = /var/ftp/pub
confdir = /etc/mrepo.conf.d
arch = i386 x86_64
mailto = root@localhost
smtp-server = localhost
rhnlogin = username:password

[centos5-i386]
name = CentOS $release ($arch)
release = 5.3
arch = i386
iso = /var/ftp/pub/.iso/centos5-i386/CentOS-$release-$arch-bin-DVD.iso
updates = rsync://mirrors.kernel.org/centos/$release/updates/i386/

## centos 5 x86_64
[centos5-x86_64]
name = CentOS $release ($arch)
release = 5.3
arch = x86_64
iso = /var/ftp/pub/.iso/centos5-x86_64/CentOS-$release-$arch-bin-DVD.iso
updates = rsync://mirrors.kernel.org/centos/$release/updates/x86_64/

## rhel 5 i386
[rhel5-i386]
name = Red Hat $release Server ($arch)
release = 5.3
arch = i386
iso = /var/ftp/pub/.iso/rhel5-i386/rhel-$release-server-$arch-dvd.iso
updates = rhns:///rhel-$arch-server-5

## rhel 5 x86_64
[rhel5-x86_64]
name = Red Hat $release Server ($arch)
release = 5.3
arch = x86_64
iso = /var/ftp/pub/.iso/rhel5-x86_64/rhel-$release-server-$arch-dvd.iso
updates = rhns:///rhel-$arch-server-5

## fedora 11 i386
[fedora11-i386]
name = Fedora $release ($arch)
release = 11
arch = i386
iso = /var/ftp/pub/.iso/fedora11-i386/Fedora-$release-$arch-DVD.iso
updates = rsync://mirrors.kernel.org/fedora/updates/$release/$arch/

## fedora 11 x86_64
[fedora11-x86_64]
name = Fedora $release ($arch)
release = 11
arch = x86_64
iso = /var/ftp/pub/.iso/fedora11-x86_64/Fedora-$release-$arch-DVD.iso
updates = http://mirrors.kernel.org/fedora/updates/$release/$arch/

Once you have made the needed changes to your configuration you’re ready to kick off the mirror and export process. Based on what we have in the configuration the next step will do the following:

• Loop-Mount and export the DVD images to /var/ftp/pub/. It will create subdirectories using the bracketed title we used in the config file (ie; /var/ftp/pub/fedora11-x86_64/)
• Mirror the public updates contents locally from the locations defined by “updates =”
• Create two subdirectories within our wwwdir path for RPMS.os and RPMS.updates.

Step 7: Create The Repositories

If you’ve made it through each of the previous steps without hitting any roadblocks you’re now ready to initiate your mirror creation.

mrepo -guvvv

Note: The command above suggests very verbose output (vvv) for the first run. This way you can see what it is doing and the progress it is making. For future runs you can omit the verbose options.

After typing out that title I wonder if I’ve been spending too much time on forums, hehe. In any case, I ran into a problem today regarding my database tables being read-only so I thought I’d post this for the good of the internets. I had to dig pretty deep to find a solution to it. Hopefully some additional Google fodder will make life a bit easier for the next guy.

If you’re a regular reader here you know that I had some database problems this morning with this site. I ended up needing to import some databases from backups and manually tweak a few things to restore the lost data. Well it turns out that after a database import or a change of users and permissions your database tables can become read-only. I’m not entirely sure the reasoning behind this, but I found some discussion about it on a MySQL developers mailing list thread.

The solution that I found was to export (read: backup) all of my MySQL databases, and magically things become read-write again. I did this through the phpmyadmin web interface:

1. Login to /phpmyadmin/ on your site.
2. Click “Export” in the first column, near the bottom.
3. Click “Select All” atop your list of databases.
4. Select any special export options you want (optional)
5. Click “Save as file” and a compression type. Click Go.

This will export and (optionally) compress your databases into an archive and present it to you as a download through your browser. Tuck it away for safe keeping somewhere and then check your database(s) again. Are they still read-only? If they are you are probably experiencing a different problem than I was, and I’m sorry this didn’t help. If they are now read-write, congratulations!

If anyone can shed any light on the reasoning behind this I’d be very interested to know. Until then, I’ve learned not to question the oddities like this but to simply share the method to the madness. Enjoy.

I apologize for the trouble on the site today. I made a minor little mistake with a database and spent the entire day trying to solve it. It appears that everything is working as expected now and I want to thank Stephen Shaw for helping me out, just as I was about to give up. Great work–good eye!

I’m going to make sure I have some healthy backups at this point, and perhaps I’ll find a little time to blog about the whole situation. Mainly what to avoid and what tips I’ve found in the process.

Thank you again for your patience. Let me know if you find anything out of place on the site.

One of the things that has long bugged me about Mac OS X (one of the few things!) is the different way that the Home and End keys function inside a document. I have long been used to Home moving to the beginning of the current line, and End moving to the end of the current line (both on Windows and Linux machines), but on OS X they seem to move to the end of the document and the beginning of the document. After months of working on OS X I have yet to get used to this functionality so today I set out to find a solution. This is what I found.

I quickly found that there are a lot of people with this same gripe. Google is ripe with suggestions. Some of them manual keyboard re-mapping tricks, others suggesting the “Mac way” to achieve the same, and some with third-party software solutions. I went with the latter in this case.

There is a free program available called DoubleCommand, which allows you to remap the way your keyboard works with a number of pre-defined checkboxes. All you need to do in order to gain the “normal” Home and End functionality is to click a box and Activate the changes. Here are five steps you can follow to achieve Home and End normalfication.

1. Download DoubleCommand
2. Install by double-clicking on the .pkg file and following the prompts.
3. Launch DoubleCommand via: Apple > System Preferences > DoubleCommand
4. Select “PC style Home and End keys”
5. Save (System wide or User) and Activate

Next time you’re in a document and need to hit the beginning or end of a line, simply press fn-home/end and you’ve got the functionality the way you might expect it to work. Enjoy!

If you have any other solutions to this problem please share them in the comments. As they say, there is more than one way to skin a cat!

I’ve been using my Mac more and more while I’m at work these days and one of the things that I quickly started to miss was the Linux keyboard shortcut for locking your screen (ctrl-alt-l) when you’re going to be away from your desk. The first few times I simply resigned myself to trust my office mate, but that wasn’t going to cut it long-term. I set out to find a solution to locking my screen while I was going to be away from my Mac. This is what I found.

There are actually a number of solutions that I found. Some require third-party software. Others require creating dock icons to launch command-line tools. The solution I came up with requires no third-party software, custom dock icons or arthritis-inducing keyboard shortcuts. This uses easily configurable options that can also easily be undone if desired. I used Hot Corners.

If you’re not familiar with Hot Corners, it is the ability to use your mouse and the four corners of your desktop to achieve additional functionality. I use my upper-right hand corner to launch Spaces. The lower-right for Expose (All Windows). I’ve now added the lower-left to launch my Screensaver. By configuring my machine to activate the screensaver when the mouse is in the lower-left corner of the screen I am easily able to lock my screen when I need to be away from the computer. I simply drag the mouse down, screensaver turns on (which requires a password to unlock, of course) and I’m golden.

To activate this feature, do the following:

1. Click Apple > System Preferences > Desktop and Screensaver
2. In the lower left corner of that window you’ll find a button that says “Hot Corners”. Select that and then select one of the four corners to “Start Screensaver”.

As I mentioned before, there are other solutions to this problem, but this is the solution that I found the least complicated or difficult to setup. Do you have another solution that you consider easier? Let us know in the comments.