Imageless VM builds with libguestfs
If you're running a VM cluster and regularly deploying new VMs, you'll have noticed that using a base image to build new VMs upon is just a fucking pain in the ass. It is indeed such a huge pain in the ass, that I had to spell this out three times in two sentences, just to clarify how much of a pain in the ass this is. Because you'll build the image, deploy it three times and notice that there's something you fucked up, that you'd like to change, and that gleefully involves just doing the whole thing over without fucking up anything else. Hence, you'll live with the buggy piece of crap because that's still easier than rebuilding the whole thing from scratch and you've got shit to do. At least, that's what I did. Until I got fed up with it and started building vmaker to get rid of the pain.
The awesome part about vmaker is that I came across libguestfs, a library that allows you to do lots of crazy stuff with a VM image without having to make those image's devices show up anywhere on your host system. No loop devices, no kpartx, no stuff being mounted that is still leftover from a build that crashed halfway through and now has to be cleaned up (by you), you just run guestfish and go "hey, make me an MSDOS partition table, gimme a partition, turn that into a PV and VG, create a 5 gig root partition formatted with ext4, and a 2 gig partition for /var/log also formatted with ext4 and mounted to /var/log."
Or, in guestfish speak:
guestfish -a "$IMAGEFILE" run \ : part-disk /dev/sda mbr \ : pvcreate /dev/sda1 \ : vgcreate vmsys /dev/sda1 \ : lvcreate root vmsys 5120 \ : mkfs ext4 /dev/vmsys/root \ : mount /dev/vmsys/root / \ : mkdir-p /var/log \ : lvcreate varlog vmsys 2048 \ : mkfs ext4 /dev/vmsys/varlog
Then you can mount the whole thing under /mnt:
guestmount -a "$IMAGEFILE" \ -m /dev/vmsys/root \ -m /dev/vmsys/varlog:/var/log \ --rw -o dev /mnt
And it'll just work. Now you can send debootstrap along to install the system of your choice, chroot into it and configure it.
Booting
I guess you'd also like that shiny new VM to be bootable. Sadly, installing Grub to a VM image isn't all that much fun and involves fiddling with device maps, Grub complaining that it can't find all kinds of crazy stuff, and strange things happening when you try to actually boot the damn thing. libguestfs can come in quite handy there, too.
First of all, still inside the chroot, you need to install Grub:
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get install -y linux-image-generic grub2
This will cause Apt to install the packages, but not run any configuration. For the config to work, you'll need a sane /dev, and this is exactly what guestfish emulates for you. So, after you're done with the basics, unmount the image and run this baby:
guestfish -a "$IMAGEFILE" run \ : mount /dev/vmsys/root / \ : mkdir-p /boot/grub \ : write /boot/grub/device.map "(hd0) /dev/sda" \ : copy-in "parts/etc-default-grub" "/tmp" \ : mv "/tmp/etc-default-grub" "/etc/default/grub" \ : command "grub-install /dev/sda" \ : command "update-grub"
Guestfish will then mount your image again, create a dummy device.map that tells Grub to look for /dev/sda, and run the installation. And the awesome part is that this actually works without pain.
Apart from the fact that libguestfs uses Fuse and is therefore a bit slow (a single vmaker run takes about 20 minutes), this completely frees me of having to carry a multi-gig base image around that is outdated by the time it's finished. Changing the partition layout that vmaker creates involves changing a coupl'a lines in the script, instead of having to manually walk through the installer again -- and the configuration I need to give to guestfish is actually pretty much the same as what I'd do if I'd create the partition table manually because it uses the same tools. So I can basically write that down off the top of my head, which I can't do with a debconf preseeding file. This is really, really awesome. I love this thing.
VM deployment is finally fun!
(Albeit a bit slow. I'll see if I can find a way around that.)