This procedure should be considered highly experimental. While it has been tested to work, it may break in unexpected ways, destroy data, eat your dog, break the economy, or have other adverse effects. The pv-grub kernel used is experimental and may vanish or change behavior without warning, leaving your BSDNode unbootable. Use at your own risk. Discontinue use and consult a doctor immediately if any rash appears.
With the recent addition of pv-grub in the kernel list (as 'z - testing' for now), it's now possible to boot BSD kernels on a Linode.
I (bd_) have made images for NetBSD, sized for a Linode 360. Here's how to install them.
Create a /boot image (ext2, 64mb) and a netbsd image (raw, 12224mb). Create finnix and netbsd config profiles as described in the Roll your own section below. Then boot into finnix and type:
# curl http://fushizen.net/~bd/bsdnode/netbsd/install.sh|sh
Once the script is done, you can reboot using pv-grub as described below. Once you boot, there's a bit of final setup you need to do. Once you're dropped dropped into single user mode in lish:
/etc/rc.conf is not configured. Multiuser boot aborted. Enter pathname of shell or RETURN for /bin/sh: (just hit enter) # mount -o rw / # passwd Changing password for root. New Password: Retype New Password: # export TERM=xterm # vi /etc/rc.conf (uncomment rc_configured=YES, comment rc_configured=NO) # sync # reboot
Once you're dropped into lish, you can reboot again normally.
Roll your own
What you'll need:
- A Xen linode (old UML linodes will not work - go ahead and add a new one just for this)
- A fastish internet connection (upstream) from your home/work/other machine.
- Ideally a kvm-capable CPU on your home/work machine; otherwise, some other x86 virtualization system from which you can extract raw whole-disk images.
anonymous: dont think you need to virtualize to do this if you have a spare disk lying around & dd ? hmm.
First, we'll set up the images on the linode. Create two disk images; one small (64mb is plenty) /boot partition, formatted ext2, and another 'netbsd' image of whatever size (at least 2G recommended), formatted raw
Create a new configuration profile - name it 'finnix' or similar. Set kernel to recovery - finnix, xvda to finnix, xvdb to /boot, xvdc to your netbsd image, .and the initrd to recovery - finnix. Disable all helper options. Boot it and make sure you can log into lish.
Once you're in, set a root password and start up sshd.
root@hvc0:~# passwd Enter new UNIX password: Retype new UNIX password: passwd: password updated successfully root@hvc0:~# /etc/init.d/ssh start [*] Generating SSH1 RSA host key... done [*] Generating SSH2 RSA host key... done [*] Generating SSH2 DSA host key... done [*] Starting OpenBSD Secure Shell server... sshd done
Now format and mount /boot:
root@hvc0:~# mke2fs /dev/xvdb root@hvc0:~# mkdir /mnt/boot root@hvc0:~# mount /dev/xvdb /mnt/boot
We'll also format and mount the netbsd partition; we'll be using it as scratch space for now:
root@hvc0:~# mke2fs /dev/xvdc root@hvc0:~# mkdir /mnt/tmp root@hvc0:~# mount /dev/xvdc /mnt/tmp
And set up some temporary swap space:
root@hvc0:~# dd if=/dev/zero of=/mnt/tmp/swap bs=1M count=256 root@hvc0:~# mkswap /mnt/tmp/swap root@hvc0:~# swapon /mnt/tmp/swap
Building the kernel
The NetBSD 5.0 i386 XEN3PAE_DOMU kernel configuration works out of the box with Linode Xen. All you need to do is pull the source down into your finnix temporary setup, setup a build environment, use that to build the tool chain for cross-compiling, and build the kernel using the XEN3PAE_DOMU conf.
Note: I (entrosca) had some difficulty finding information in documentation regarding available kernel configurations. I dissected build.sh to find that bsd.kernobj.mk is useful and will point you to /src/sys/arch/$arch/conf/ for your relevant architecture (as, e.g., passed to build.sh with '-m i386').
The path for the relevant kernel conf is: ftp://ftp.netbsd.org/pub/NetBSD/NetBSD-current/src/sys/arch/i386/conf/XEN3PAE_DOMU
Also, I (entrosca) haven't tested, but I suspect a 64-bit build using '-m amd64' and 'kernel=XEN3_DOMU' for build.sh in the kernel build step will work too (you'll also need the amd64 cross-compile toolchain).
First, download and extract the NetBSD source. Do the following on your linode (lish or through ssh, doesn't matter):
root@hvc0:~# cd /mnt/tmp root@hvc0:/mnt/tmp# wget ftp://ftp.netbsd.org/pub/NetBSD/NetBSD-current/tar_files/src/*.tar.gz root@hvc0:/mnt/tmp# for i in *.tar.gz; do tar xzf $i; done root@hvc0:/mnt/tmp# rm *.tar.gz
Update Source from CVS
We want the latest version, so update it further from CVS:
root@hvc0:/mnt/tmp# apt-get update root@hvc0:/mnt/tmp# apt-get install cvs root@hvc0:/mnt/tmp# cd src root@hvc0:/mnt/tmp/src# cvs -z3 update -dP
Say yes when prompted by ssh.
Setup Build Environment
Now that we've downloaded and updated the kernel source, all that remains is to build it. First, setup the linux build environment:
root@hvc0:/mnt/tmp/src/sys# aptitude install build-essential zlib1g-dev libncurses5-dev root@hvc0:/mnt/tmp/src/sys# rm /var/cache/apt/archives/* # Finnix's default path doesn't sit well with build.sh, so fix it: root@hvc0:/mnt/tmp/src/sys# PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11:/usr/games
Build Cross-Compile Toolchain
root@hvc0:/mnt/tmp/src/sys# cd .. root@hvc0:/mnt/tmp/src# ./build.sh -O /mnt/tmp/obj -m i386 -U -u tools
This step will take a while, so it's a good chance to get started on setting up the image, below.
Build i386 XEN3PAE_DOMU BSD Kernel
root@hvc0:/mnt/tmp/src# ./build.sh -O /mnt/tmp/obj -m i386 -U -u kernel=XEN3PAE_DOMU
Copy your new kernel to /boot:
root@hvc0:/mnt/tmp/src# cp /mnt/tmp/obj/sys/arch/i386/compile/XEN3PAE_DOMU/netbsd /mnt/boot
Now unmount your scratch space and move on to the image setup.
root@hvc0:/mnt/tmp/src# cd root@hvc0:~# swapoff /mnt/tmp/swap root@hvc0:~# umount /mnt/tmp
If you wish to use a older kernel see the below directions. See NetBSD Kernel Instructions for further information.
Although NetBSD 4.0 comes with a Xen kernel, this won't work on linode out of the box; Linode uses 64-bit dom0s (so PAE support is needed and must be added by patching), and it also uses a debug build of the hypervisor, which means one of the PTE bits that bsd wants is unusable. We'll need to download the very latest version, and patch it to be usable.
To begin, follow the steps outlined in the Current Kernel Get Source/Update section above, but use an older version, e.g. ftp://ftp.netbsd.org/pub/NetBSD/NetBSD-4.0.1/
Next up, we'll need to apply a small patch to the kernel. As mentioned before, hypervisor debugging steals a PTE bit for its own use, so we need to make sure NetBSD doesn't use the same bit for its own purposes.
root@hvc0:/mnt/tmp/src# cd sys root@hvc0:/mnt/tmp/src/sys# curl http://fushizen.net/~bd/netbsd_gnttab_compat.patch|patch -p0
When you're done with that, you can continue through the sames steps as outlined in the Current Kernel section above.
Note: I (entrosca) got a 404 when trying to pull the patchfile mentioned above on 2010-03-04, so this option may no longer be available.
Preparing the image
You'll need to prepare a disk image by running the NetBSD installer locally, then uploading the resulting image whole. I'll assume you have kvm installed and working already; you'll need to adjust these instructions a bit if you're using something else, or if your local OS isn't linux.
First, grab a netbsd ISO from one of the mirrors at http://www.netbsd.org/mirrors/#iso.
Once you have that, create a disk image of size equal to the netbsd image you created on your linode:
$ touch netbsd-img $ perl -e 'truncate "netbsd-img", SIZE*1024*1024'
And fire up the installer:
$ kvm -hda netbsd-img -cdrom i386cd-4.0.iso -boot d -no-acpi
Proceed through the install normally. Make sure to set a root password, and use a secure password hash like MD5. The bootblock doesn't particularly matter; you'll be booting from pv-grub.
Once the install's finished, reboot into the installed system:
$ kvm -net nic,model=ne2k_pci -net user -hda netbsd-img -boot c -no-acpi
Edit /etc/fstab using vi, replacing 'wd' with 'xbd'. Also edit /etc/rc.conf, adding:
And comment out every line in /etc/ttys except for the 'console' line.
Finally shut down with:
Now compress your disk image for upload:
$ bzip2 -9 netbsd-img
And actually do the upload (make sure you've umounted the scratch mount first!):
$ cat netbsd-img.bz2 | ssh root@finnixip 'bzip2 -dc > /dev/xvdc'
You're now ready to reboot into NetBSD. Shutdown finnix, and set up a new config profile:
Kernel: z - testing xvda: your netbsd image xvdb: /boot
Again, disable helper options.
Once you've created the new profile, reboot into it. You should soon see a grub prompt on lish. Type:
grubdom> kernel (hd1)/netbsd grubdom> boot
And you should be in BSD!
A few caveats:
- Currently the boot process requires that you manually point GRUB at the kernel; pv-grub as currently deployed won't look for a menu.lst. (Note: This appears to no longer be true. The PV-GRUB page says it looks for /boot/grub/menu.lst)
- BSD ssh is configured to disable password root logins by default, so you'll need to useradd -G wheel -m <username> to create a login user before logging in. Remember that BSD 'su' requires that you be in the wheel group!
- This should probably be considered firmly in the 'use at your own risk' category - don't expect too much support from linode proper if you do do this :)