NetBSD howto

This procedure should be considered highly experimental. While it has been tested to work, it may break in unexpected ways, destroy data, eat your dog, break the economy, or have other adverse effects. The pv-grub kernel used is experimental and may vanish or change behavior without warning, leaving your BSDNode unbootable. Use at your own risk. Discontinue use and consult a doctor immediately if any rash appears.

With the recent addition of pv-grub in the kernel list (as 'z - testing' for now), it's now possible to boot BSD kernels on a Linode.

Premade images

I (bd_) have made images for NetBSD, sized for a Linode 360. Here's how to install them.

Create a /boot image (ext2, 64mb) and a netbsd image (raw, 12224mb). Create finnix and netbsd config profiles as described in the Roll your own section below. Then boot into finnix and type:

# curl http://fushizen.net/~bd/bsdnode/netbsd/install.sh|sh

Once the script is done, you can reboot using pv-grub as described below. Once you boot, there's a bit of final setup you need to do. Once you're dropped dropped into single user mode in lish:

/etc/rc.conf is not configured.  Multiuser boot aborted.
Enter pathname of shell or RETURN for /bin/sh: (just hit enter)
# mount -o rw /
# passwd
Changing password for root.
New Password:
Retype New Password:
# export TERM=xterm
# vi /etc/rc.conf
(uncomment rc_configured=YES, comment rc_configured=NO)
# sync
# reboot

Once you're dropped into lish, you can reboot again normally.

Roll your own

What you'll need:

• A Xen linode (old UML linodes will not work - go ahead and add a new one just for this)
• A fastish internet connection (upstream) from your home/work/other machine.
• Ideally a kvm-capable CPU on your home/work machine; otherwise, some other x86 virtualization system from which you can extract raw whole-disk images.

anonymous: dont think you need to virtualize to do this if you have a spare disk lying around & dd ? hmm.

Preparation

First, we'll set up the images on the linode. Create two disk images; one small (64mb is plenty) /boot partition, formatted ext2, and another 'netbsd' image of whatever size (at least 2G recommended), formatted raw

Create a new configuration profile - name it 'finnix' or similar. Set kernel to recovery - finnix, xvda to finnix, xvdb to /boot, xvdc to your netbsd image, .and the initrd to recovery - finnix. Disable all helper options. Boot it and make sure you can log into lish.

Once you're in, set a root password and start up sshd.

root@hvc0:~# passwd
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
root@hvc0:~# /etc/init.d/ssh start
[*] Generating SSH1 RSA host key... done
[*] Generating SSH2 RSA host key... done
[*] Generating SSH2 DSA host key... done
[*] Starting OpenBSD Secure Shell server... sshd done

Now format and mount /boot:

root@hvc0:~# mke2fs /dev/xvdb
root@hvc0:~# mkdir /mnt/boot
root@hvc0:~# mount /dev/xvdb /mnt/boot

We'll also format and mount the netbsd partition; we'll be using it as scratch space for now:

root@hvc0:~# mke2fs /dev/xvdc
root@hvc0:~# mkdir /mnt/tmp
root@hvc0:~# mount /dev/xvdc /mnt/tmp

And set up some temporary swap space:

root@hvc0:~# dd if=/dev/zero of=/mnt/tmp/swap bs=1M count=256
root@hvc0:~# mkswap /mnt/tmp/swap
root@hvc0:~# swapon /mnt/tmp/swap

Building the kernel

Current Kernel

The NetBSD 5.0 i386 XEN3PAE_DOMU kernel configuration works out of the box with Linode Xen. All you need to do is pull the source down into your finnix temporary setup, setup a build environment, use that to build the tool chain for cross-compiling, and build the kernel using the XEN3PAE_DOMU conf.

Note: I (entrosca) had some difficulty finding information in documentation regarding available kernel configurations. I dissected build.sh to find that bsd.kernobj.mk is useful and will point you to /src/sys/arch/$arch/conf/ for your relevant architecture (as, e.g., passed to build.sh with '-m i386').

The path for the relevant kernel conf is: ftp://ftp.netbsd.org/pub/NetBSD/NetBSD-current/src/sys/arch/i386/conf/XEN3PAE_DOMU

Also, I (entrosca) haven't tested, but I suspect a 64-bit build using '-m amd64' and 'kernel=XEN3_DOMU' for build.sh in the kernel build step will work too (you'll also need the amd64 cross-compile toolchain).

Get Source

First, download and extract the NetBSD source. Do the following on your linode (lish or through ssh, doesn't matter):

root@hvc0:~# cd /mnt/tmp
root@hvc0:/mnt/tmp# wget ftp://ftp.netbsd.org/pub/NetBSD/NetBSD-current/tar_files/src/*.tar.gz
root@hvc0:/mnt/tmp# for i in *.tar.gz; do tar xzf $i; done
root@hvc0:/mnt/tmp# rm *.tar.gz

Update Source from CVS

We want the latest version, so update it further from CVS:

root@hvc0:/mnt/tmp# apt-get update
root@hvc0:/mnt/tmp# apt-get install cvs
root@hvc0:/mnt/tmp# cd src
root@hvc0:/mnt/tmp/src# cvs -z3 update -dP

Say yes when prompted by ssh.

Setup Build Environment

Now that we've downloaded and updated the kernel source, all that remains is to build it. First, setup the linux build environment:

root@hvc0:/mnt/tmp/src/sys# aptitude install build-essential zlib1g-dev libncurses5-dev
root@hvc0:/mnt/tmp/src/sys# rm /var/cache/apt/archives/*
# Finnix's default path doesn't sit well with build.sh, so fix it:
root@hvc0:/mnt/tmp/src/sys# PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11:/usr/games

Build Cross-Compile Toolchain

root@hvc0:/mnt/tmp/src/sys# cd ..
root@hvc0:/mnt/tmp/src# ./build.sh -O /mnt/tmp/obj -m i386 -U -u tools

This step will take a while, so it's a good chance to get started on setting up the image, below.

Build i386 XEN3PAE_DOMU BSD Kernel

root@hvc0:/mnt/tmp/src# ./build.sh -O /mnt/tmp/obj -m i386 -U -u kernel=XEN3PAE_DOMU

Wrapup

Copy your new kernel to /boot:

root@hvc0:/mnt/tmp/src# cp /mnt/tmp/obj/sys/arch/i386/compile/XEN3PAE_DOMU/netbsd /mnt/boot

Now unmount your scratch space and move on to the image setup.

root@hvc0:/mnt/tmp/src# cd
root@hvc0:~# swapoff /mnt/tmp/swap
root@hvc0:~# umount /mnt/tmp

If you wish to use a older kernel see the below directions. See NetBSD Kernel Instructions for further information.

Older Kernels

Although NetBSD 4.0 comes with a Xen kernel, this won't work on linode out of the box; Linode uses 64-bit dom0s (so PAE support is needed and must be added by patching), and it also uses a debug build of the hypervisor, which means one of the PTE bits that bsd wants is unusable. We'll need to download the very latest version, and patch it to be usable.

To begin, follow the steps outlined in the Current Kernel Get Source/Update section above, but use an older version, e.g. ftp://ftp.netbsd.org/pub/NetBSD/NetBSD-4.0.1/

Next up, we'll need to apply a small patch to the kernel. As mentioned before, hypervisor debugging steals a PTE bit for its own use, so we need to make sure NetBSD doesn't use the same bit for its own purposes.

root@hvc0:/mnt/tmp/src# cd sys
root@hvc0:/mnt/tmp/src/sys# curl http://fushizen.net/~bd/netbsd_gnttab_compat.patch|patch -p0

When you're done with that, you can continue through the sames steps as outlined in the Current Kernel section above.

Note: I (entrosca) got a 404 when trying to pull the patchfile mentioned above on 2010-03-04, so this option may no longer be available.

Preparing the image

You'll need to prepare a disk image by running the NetBSD installer locally, then uploading the resulting image whole. I'll assume you have kvm installed and working already; you'll need to adjust these instructions a bit if you're using something else, or if your local OS isn't linux.

First, grab a netbsd ISO from one of the mirrors at http://www.netbsd.org/mirrors/#iso.

Once you have that, create a disk image of size equal to the netbsd image you created on your linode:

$ touch netbsd-img
$ perl -e 'truncate "netbsd-img", SIZE*1024*1024'

And fire up the installer:

$ kvm -hda netbsd-img -cdrom i386cd-4.0.iso -boot d -no-acpi   

Proceed through the install normally. Make sure to set a root password, and use a secure password hash like MD5. The bootblock doesn't particularly matter; you'll be booting from pv-grub.

Once the install's finished, reboot into the installed system:

$ kvm -net nic,model=ne2k_pci -net user -hda netbsd-img  -boot c -no-acpi           

Edit /etc/fstab using vi, replacing 'wd' with 'xbd'. Also edit /etc/rc.conf, adding:

dhclient=YES
sshd=YES

And comment out every line in /etc/ttys except for the 'console' line.

Finally shut down with:

# poweroff

Now compress your disk image for upload:

$ bzip2 -9 netbsd-img

And actually do the upload (make sure you've umounted the scratch mount first!):

$ cat netbsd-img.bz2 | ssh root@finnixip 'bzip2 -dc > /dev/xvdc'

You're now ready to reboot into NetBSD. Shutdown finnix, and set up a new config profile:

Kernel: z - testing
xvda: your netbsd image
xvdb: /boot

Again, disable helper options.

Once you've created the new profile, reboot into it. You should soon see a grub prompt on lish. Type:

grubdom> kernel (hd1)/netbsd
grubdom> boot

And you should be in BSD!

A few caveats:

• Currently the boot process requires that you manually point GRUB at the kernel; pv-grub as currently deployed won't look for a menu.lst. (Note: This appears to no longer be true. The PV-GRUB page says it looks for /boot/grub/menu.lst)
• BSD ssh is configured to disable password root logins by default, so you'll need to useradd -G wheel -m <username> to create a login user before logging in. Remember that BSD 'su' requires that you be in the wheel group!
• This should probably be considered firmly in the 'use at your own risk' category - don't expect too much support from linode proper if you do do this :)