Make Your Own Bootable Thumbdrive with Buildroot

From Sfvlug

Current revision as of 07:18, 7 April 2012

At the meeting on Saturday, March 31, 2012, I got the opportunity to show off the fruits of my latest project to a few members. Kurt and Mark asked me some more specific details on how to reproduce this, so here it is for them and any others who may be interested in making their own bootable thumbdrives.

A little bit of background, first. This project will actually result in a build more similar to an embedded Linux project like OpenWrt or similar because it uses many of the same tools. So it is important to understand what we are doing here. We are going to build a cross-compiling tool chain then populate a file system with specially built software. This means two things. First, we don't have to build for our native processor type if we so choose, and second, binaries from our native operating system will not work because they do not link against the cross-compiled libraries. Also, this project will result in a system containing uClibc instead of the standard glibc against which your regular operating system binaries will be linked, and busybox which will replace coreutils, util-linux, and several other base packages.

Setting Up the Cross-Compiling Environment

Make sure you have what you will need installed first. Even though my final image is only about 50MB, unpacking all the source code and compiling has taken over 5GB, so make sure you have plenty of disk space. Additionally, you will need make and gcc to build your cross-compiler. It also helps to have patch and sed already installed. RPM based distros should have @development-tools installed, and DEB based distros should have build-essentials.

Browse on over to http://www.buildroot.org/ and click the link on the left for Download. Pick either a daily snapshot or the latest stable release, which is 2012.02 at this time. New stable releases are every three months, invariably on the last day of that month, so the next one will be on May 31.

Pick a directory where to unpack the downloaded tarball, preferably somewhere under your home directory. I used linux-uclibc, and unpacking results in a subdirectory like buildroot-2012.02.

During my build I chose to use a look-aside build directory so I could use the same sources for multiple builds if I want to. This means buildroot will build outside of the directory created by unpacking the tarball. The default behavior is to put all the resultant files into a subdirectory called output. Instead I told it to do something else for output, put it in ../i686-2012.02. Now I can also make another output directory called ../arm-2012.02 or x86_64-2012.02 if I want to. You might want to skip this for your first build but if you're ambitious enough to want to try multiple projects, go ahead and follow my lead. I started with a default config with the following:

make defconfig O=../i686-2012.02

Now I can work exclusively from my look-aside directory. I only customized a few items at first.

cd ../i686-2012.02
make menuconfig

If you have ever built the Linux kernel from source then you are probably familiar with the make menuconfig operation. Buildroot uses the same interface ported from the kernel. It also has xconfig and gconfig targets available if you prefer Qt or Gtk to the ncurses interface.

Some specifics to turn on for the first run. As you can see, defconfig has defaulted to i386 for the target architecture, and i686 for the target variant. You could now select a different processor type. I am building on a quad-core processor, so under Build Options I set the "number of jobs to run simultaneously" to 5. Under Target Optimizations, I turned on "enable large file support," "enable IPv6 support," "enable RPC support," "enable WCHAR support," and "enable C++ support." The latter will build a cross-compiling g++ as well as gcc. Under System Configuration, I set "/dev management" to "udev." Doing so now avoids having a bunch of /dev entries which are hard to get rid of in the final product. Also, I set the "port to run a getty on" to "tty1." Leaving it as a serial port doesn't work well for the thumbdrive. Finally, under Kernel, turn on "Linux Kernel" and set "defconfig name" to "i386." Also select "install kernel image to /boot in target." Pick "exit" a few times until it returns to the shell prompt.

make

Now just let it build. This could take a few hours, so go to sleep or work or lunch or whatever. We have left most of the options very basic, so there should be no problems building.

Customizing a Target Image

Believe it or not, we are now very close to having a bootable image available. Only a few options remain but first I want to discuss a few more customizations. Busybox, uClibc, and the Linux kernel are all customizable using the same menuconfig option.

make busybox-menuconfig
make uclibc-menuconfig
make linux-menuconfig

You may choose to change some options in any of these. Indeed, you probably want to add some drivers for some network cards to the kernel. The Intel e1000 is useful for running in qemu or VirtualBox. I didn't really need to alter uClibc beyond its defaults, but I turned on a few more applets in busybox and added bzip2 and xzip support. After customizing these things, run make again. I prefer to do incremental changes and run make after small sets of changes to make sure if I turn on any incompatible options, I can easily back them out.

Inside your output or look-aside directory, there is a subdirectory named target. This is just about everything that will be built into your final image. You can customize anything in here now, particularly files in target/etc. You probably want to fix up the /etc/fstab so that /dev/sdb1 gets mounted as root in read-write mode.

Now, to build a final image, enter into menuconfig again. Under Filesystem images, choose "tar the root filesystem." I left the "compression method" as "no compression." Also, you might like to play around with the "ext2 root filesystem." Under Bootloaders, I chose to use grub and I will explain how to use it below. At one point I tried to use syslinux but I could not successfully build it, so if you get it working, let me know. Choose "exit" a few more times and get back to the shell and run make again.

Building A Bootable Thumbdrive

If all has gone well, you now have a rootfs.tar under output/images. You also have rootfs.ext2 if you chose that option. You can loopback mount this and chroot to it if you like.

We're going to set buildroot aside for a moment and prepare the thumbdrive. Insert it into a USB socket. Check /proc/partitions and see what new partitions exist. Mine came up as /dev/sdb. Do the following as root.

fdisk /dev/sdb

Use 'd' to delete any existing partition from the drive and 'c' to create a new one. I accepted the defaults and created a partition of maximum size on the drive. Make sure it is type 0x83 - Linux. Use 'w' to write the new partition table to the drive and exit fdisk. Make a new filesystem on the drive, mount it, and unpack the rootfs.tar onto it.

mkfs /dev/sdb1
mount /dev/sdb1 /mnt
cd /mnt
tar xvf $OUTPUT/images/rootfs.tar

Next we will make sure to install the grub which is now on the thumbdrive onto the boot sector of the thumbdrive.

grub-install --root-directory=/mnt /dev/sdb

You will probably need an initrd in order to boot properly. My distro uses dracut to produce this and I found it worked perfectly.

dracut -k /mnt/lib/modules /mnt/boot/initrd.img 3.2.6

Finally, you need to tell grub how to boot the kernel. Make the following in /mnt/boot/grub/menu.lst:

default=0
timeout=10

title		Linux
	root	(hd0,0)
	kernel	/boot/bzImage ro root=/dev/sdb1
	initrd	/boot/initrd.img

Going Further

Now that you have your first bootable thumbdrive and have booted it a few times to make sure it works, you have probably noticed it is quite boring at this point. It has vi and an /etc/passwd file and not much else. You could probably use it as rescue media for another computer but that's it.

If you made a module for your network card, modprobe it and run udhcpc eth0. If you built busybox with ping you can test your network and if you added nc you can probe some ports.

Get back into menuconfig and add some more packages under Package Selection For The Target. I made my initial thumbdrive into a general purpose firewall. I added ebtables, iptables, ipset, dropbear, and dnsmasq. These are enough to give me at least the functionality of OpenWrt or others. I also added net-snmp and nfs-utils. Just run make after selecting the packages and unpack the tarball as above. You will need to run dracut again if you added to your kernel but it is not necessary to mess with grub any more.

Adding A Custom Package

Here's the most advanced thing I have done with this. I added a new package to buildroot. In order to add more advanced features to my firewall, I added xtables-addons. Check out http://xtables-addons.sf.net/ for more information about what this does. In short, it's some extensions for iptables which add portknocking, tarpitting, and portscan detection to name a few.

Adding a package consists of just a few steps. First, I added the xtables-addons subdirectory under my buildroot directory's package subdirectory. In this case it was buildroot-2012.02/package/xtables-addons. You will see that all available packages have a subdirectory under package. Inside here, I added Config.in with the following contents:

config BR2_PACKAGE_XTABLES_ADDONS
        bool "xtables-addons"
        help
          Extensions targets and matches for iptables

          http://xtables-addons.sourceforge.net/

Before "bool" and "help" that's one tab. The other two indented lines are one tab and two spaces. I think this is important. Second, I added another file to the same location, xtables-addons.mk with the following:

#############################################################
#
# xtables-addons
#
#############################################################

XTABLES_ADDONS_VERSION = 1.41
XTABLES_ADDONS_SOURCE = xtables-addons-$(XTABLES_ADDONS_VERSION).tar.xz
XTABLES_ADDONS_SITE = http://$(BR2_SOURCEFORGE_MIRROR).dl.sourceforge.net/sourceforge/xtables-addons
XTABLES_ADDONS_INSTALL_STAGING = NO
XTABLES_ADDONS_INSTALL_TARGET = YES
XTABLES_ADDONS_CONF_OPT = --with-kbuild=$(LINUX_DIR) --with-xtlibdir=/lib/xtables/
XTABLES_ADDONS_DEPENDENCIES = host-pkg-config iptables linux

$(eval $(call AUTOTARGETS))

And finally, I added a line to package/Config.in:

source "package/xl2tp/Config.in"
source "package/xtables-addons/Config.in"

This final addition makes it show up in menuconfig. But don't select it in menuconfig because I still have a problem with it. Instead, from the shell just run this:

make xtables-addons

That will build the package and install it in output/target. When you next run make, it will be added to your images. But when it is built initially, it fails because it tries to alter some files on your host system, files which only root may change, which is why it is important to build as a normal user and not root. Once I figure out how to avoid this package doing this, I will update this document, and perhaps submit the package to the buildroot team as an addition.

I also might add mini_snmpd from http://freecode.com/projects/minisnmpd as well because net-snmp is a lot bigger than is necessary for these needs.

Jeff 23:57, 6 April 2012 (UTC)