Yesterday at work, a fellow PhD-student came to me because he had messed up his Linux installation on his laptop. He had misspelled a point for a slash and typed chown -R someuser / as root. He did not realise that something was wrong and only aborted the command after quite a while.

Since all the sysadmin tools now belonged to a user and not to root anymore, the system would not boot but into a rescue shell from where it was not even possible to change back permissions, due to the messed up system.

The normal and best solution for such cases is to ...boot from one of the fabulous Linux Live-CDs. Those give you a full-fledged Linux that boots from CD only, without touching the harddisk. Popular examples are Knoppix or grml.

Even if that would have been possible (which it was not, see below), I would have only used it to copy his data to another machine and then install a new Linux on it, for two reasons: It is not trivial to restore the old file ownerships and, of course, not everything may belong to root. On Debian, one could have done a apt-get --reinstall install `dpkg --get-selections | grep -v deinstall | cut -f1` which would reinstall all your packages. But the machine ran a really old Red Hat Linux that I did not know and that was worth being annihilated for security reasons alone. And because of it being Red Hat and not Debian, of course :)

But the first problem was to get the thing to boot and that proved to be non-trivial. The internal CD-drive was not there any more and the box had no floppy drive. We tried external CD-drives, one connected via USB and another via PCMCIA, but the BIOS would not boot from them. While tweaking the BIOS' boot-parameters, I saw that the thing could boot from network, but I had never tried that before.

An alternative would have been to use the intact Windows that was installed on another partition. But first, I do not know how good the Ext3 driver for Windows is and I am very doubtful about it handling file ownerships and permissions correctly. Second, it was a matter of principle. In lack of choice, I began a quick read about PXE and setting up the boot evironment.

There are quite a few ingredients since the booting box has to first get its IP-address via DHCP and then be supplied with the kernel to boot. Since I already have a DHCP server in my network, I had to use the second network interface card (eth1) in my own box and open a private subnet behind it, in which I would offer the services for the laptop to boot from.

Luckily, a friend that frequently gives me a hand with Linux things on IRC, pointed me to this site which mentions that Knoppix has a built-in server for that. That seemed a very good solution, so I aborted my efforts to set up the stuff by hand and downloaded the latest Knoppix ISO-image. I quickly created a new virtual PC with VMware and made sure that its virtual network card is bridged to the host's eth1 and that the Knoppix ISO is included as a virtual CD-drive.

Then I booted the Knoppix in the virtual PC and simply started what they call Terminal Server. After that, I connected the laptop with a crossed network cable to eth1 on my machine and turned it on. I was totally surprised and amazed to watch it find the dhcp and to see the Knoppix boot prompt coming up! I did not expect this to work out of the box - but it did.

What I had practically done, is to abuse this virtual PC over the network as a CD-drive for the laptop, so it now booted Knoppix, just as if it would from a CD. You can have a look at the figure to see a sketch of the network setup that I just described.

The first thing I did was to take images from the laptop's harddisk - after all my collegue had been sloppy with his backups. I did it the simple way: dd if=/dev/hdaX | ssh <TheHostsIP> "cat >/path/to/big/disk/filename.img" . Since it had become late and it takes a few hours to back up 40G that way, we decided to continue the next day and went for a beer.

Today, when I returned, I found the laptop frozen. That was due to a funny network glitch: When the Knoppix in the virutal PC booted, it got its IP-address from a DHCP server running on the host's eth1. Of course, I had to turn that off before starting Knoppix' Terminal Server which includes an own DHCP. Therefore Knoppix' DHCP lease expired after a while and its network went offline which, of course, made the laptop's heart stop, since it still ran from the net. After configuring Knoppix' network manually and restarting the Terminal Server, I could reboot the laptop, too. I was lucky that the backup finished before the network went down, so I did not have to redo it.

The next step was to get a new Debian installed. I wiped the old filesystem, created a new one (ReiserFS) and mounted it. Then I (for the first time, although I had known about it before) used debootstrap to install a new minimal Debian from the Swedish Debian-mirror into the new partition. With chroot, I made the new system to my root-filesystem. After that, it went the usual way: installing more packages (e.g. a kernel), setting passwords, fixing /etc/fstab, /etc/hostname, /etc/hosts and the network configuration, configuring the X-server and so on and so forth. The installation of the new bootloader (Grub) has to be done outside the chroot, using the --root-directory= option of grub-install. Just now, while writing this, I came upon this HOWTO which describes in more detail how to install a new Debian system with debootstrap from Knoppix.

After the first successful reboot of the laptop on its own, all that remained was to copy back the things from the backup (especially /home) and to install the remaining packages that the laptop's owner requested. I gave him a quick introduction to apt, Debian's wonderful package mangagement system, and to the Debian way of network configuration - then I could release the two to live happily ever after (I hope).