Steven's Technical Blog

I’m currently installing a new sandbox for our DSL customers.

The idea is quite simple: Once a customer is virus infected or doing something nasty put him into a sandbox using some firewall forwading (IPFW) and squid magic. I’ve created such a sandbox about 3 years ago – but I have new ideas and some updates I want to bring in. I will probably show how the whole system works in a later post.

But before going live with the new sandbox I just wanted to test the basics and make sure i have some proof of concept that everything is working as planned.
So I took one of my PCengines Alix board (alix2d3) and decided to install FreeBSD 7.2 on it.

Sounds easier as it is but here are the steps what I did:

Installing FreeBSD to have a PXEboot Environment using a serial console

Actually, installing FreeBSD over the network is quite simple and consists of following tasks:

1. configure a DHCP server
2. configure a TFTP server
3. configure a NFS server
4. prepare the data for the installation
5. modify some stuff on the nfs host
6. boot the alix box and install everything needed
7. reboot alix box and enjoy

So, but some stuff is really tricky…

Using a Virtual Machine as Host

To not fuck up any hardware installation on my side I decided to use my macbook pro and parallels to start a virtual-machine for the FreeBSD host. Just create a new virtual machine and do some basic FreeBSD installation – dont forget the ports collection and all sources –  and also install ‘rsync’ (out of the ports).

Make sure you have connectivity to the internet (put the network adapter into bridged ethernet mode) to download sources from the internet.

By the way: You can download an already prepared VM from parallels with FreeBSD 7.1. It will not work to get over step 5. Just as an info…

Step 1: Configure a DHCP Server

Just install the ISC DHCP Server

 # cd /usr/ports/net/isc-dhcp30-server
 # make install

Since I just need a basic DHCP server I disabled all options. Once the DHCP server is installed you need to change the configuration (/usr/local/etc/dhcpd.conf). Mine looks like:

option domain-name "mrmouse.ch";
option domain-name-servers 193.239.21.21, 193.239.21.20;
default-lease-time 600;
max-lease-time 7200;
ddns-update-style ad-hoc;
log-facility local7;

subnet 192.168.1.0 netmask 255.255.255.0 {
}

host newbox.mrmouse.ch {
      hardware ethernet 00:0d:b9:17:2d:ac;
      fixed-address 192.168.1.100;
      next-server 192.168.1.1;
      filename "freebsd7/boot/pxeboot";
      option root-path "/usr/local/freebsd7";
}

Note: 192.168.1.1 is the IP address of my FreeBSD host running the DHCP/TFTP/NFS server. the ‘filename’ and root-path will become obvious when configuring the data for tftp and nfs.

Before being able to start the DHCP server you have to activate it in the /etc/rc.conf.
Just add dhcpd_enable=”YES” to the configuration file.

Now you may start the dhcp server:

# /usr/local/etc/rc.d/isc-dhcpd start

Step 2: Configure a TFTP Server

It’s great that FreeBSD has already a TFTP server built in. Just activate it in the /etc/inetd.conf file and change the path to /usr/local:

 tftp    dgram   udp     wait    root    /usr/libexec/tftpd      tftpd -l -s /usr/local

Hopefully you’ve already allowed inetd to be started (/etc/rc.conf -> inetd_enable=”YES”)

Now restart inetd:

# /etc/rc.d/inetd restart

Step 3: Configure a NFS Server

Add following lines to /etc/rc.conf:

 rpcbind_enable="yes"
 mountd_enable="yes"
 nfs_server_enable="yes"

Now we have to export the directoy. Edit (or create if not existing) /etc/exports and add:

 /usr/local/freebsd7     -network 192.168.1 -mask 255.255.255.0

Since this directory is not yet existing – lets create it:

 # mkdir /usr/local/freebsd7
 # chmod 755 /usr/local/freebsd7

Now we’re ready to start the NFS services:

 # /etc/rc.d/rpcbind start
 # /etc/rc.d/mountd start
 # /etc/rc.d/nfsd start

Verification can be done by typing

 # showmount -e
 Exports list on localhost:
 /usr/local/freebsd7                192.168.1.0

Step 4: Prepare the Data for the Installation

Mount the ISO file you’ve downloaded and used to setup your virtual machine within your VM and copy all files to your newly created directories:

 # rsync -avH /cdrom/ /usr/local/freebsd7/
 # cp -pR /cdrom/* /usr/local/freebsd7

Step 5: Modify some stuff

The most important thing now is that we increase the speed of the serial console for the installation. The default speed of the serial console is 9600bps. Setting some variables will not not change it.
Within your VM change the pxeboot to set the higher speed:

 # cd /sys/boot
 # make clean
 # make BOOT_COMCONSOLE_SPEED=115200

Note: Dont dare to make a ‘make install’! We don’t want to modify the bootstrap of the VM.

After we’ve done this we my copy the new pxeboot for our tftp client and clean up the mess:

 # cd /sys/boot/i386/pxeldr
 # cp pxeboot /usr/local/freebsd7/boot
 # cd /sys/boot
 # make clean

After this step we need to configure the new loader.conf.
Update /usr/local/freebsd7/boot/loader.conf and just add those 3 lines to it – leave the rest untouched:

 comconsole_speed="115200"
 console="comconsole"
 vfs.root.mountfrom="ufs:/dev/md0c"

It seems that there’s some bug for the mfs_root in FreeBSD when loading a compressed file during boot. Workaround is to not use a compressed file, therefore:

 # cd /usr/local/freebsd7/boot
 # gzip -d mfsroot.gz

Step 6: Boot your alix board and install FreeBSD

Attach your alix board to your host and boot it. It might be that you need to activate PXE boot in the BIOS (press ‘s’ during memory check).
Change your console speed to 115200bps.

During startup you see the client MAC address. Remember it and modify the “hardware ethernet”-line in the /usr/local/etc/dhcpd.conf and restart the DHCP server.

If everything is going fine you should see the ‘Welcome to FreeBSD’ banner and get sysinstall ready to install your small box.

During installation watch out:

• I used “FreeBSD system console (monochrome)” for installation. Was the best ,-)
• use the “Standard” Boot loader – not the FreeBSD BootMgr
• add a user (or you will not be able to ssh to the box)
• set a root Password. Mine is “123” … (just a joke ,-))
• enable ssh
• enable TTY for serial console (motify /etc/tty so you get:

  ttyd0   "/usr/libexec/getty std.9600"   vt100 on secure)

• disable all other TTYs (ttyv0 – 8 -> put ‘off’)

Note: For FreeBSD 8.x ttyd0 has to be replaced by ttyu0.

Step 7: Finish installation and reboot

Once everything is finished you can reboot the box (actually after installation the box will reboot). Just take out the LAN cable otherwise you start another installation.
Don’t forget to set back your terminal to 9600bps. If you want to change this you have to rebuild the boot block.
And now: Have fun now and do whatever your want with the alix box.