caker's User Mode Linux tips and notes
Christopher S. Aker - Last modified 2004-11-28 14:55 CDT
NOTE (2004-11-28): This document is fairly out of date and contains some errors.
You can find the latest version of this document here.
HOST Kernel Compile and SKAS patch
The latest SKAS patches are available from Blaisorblade here:
http://www.user-mode-linux.org/~blaisorblade/
http://www.user-mode-linux.org/~blaisorblade/patches/skas3-2.4/
http://www.user-mode-linux.org/~blaisorblade/patches/skas3-2.6/
HOST SKAS Kernel Module (waaaay outdated)
Roger Binns has provided skas as a kernel module for RH: http://www.rogerbinns.com/modskas3/. I downloaded and compiled the module, and then installed with 'insmod /path/to/skas.o'.
UML Kernel Compile
As of 2.6.9, the UML source is in the vanilla kernel (from kernel.org). An UML patch is not required...
make menuconfig ARCH=um
make linux ARCH=um
mv linux /usr/bin/linux
UML Kernel Modules install
The most recent modules (and stuff) from this kernel compile must be installed into the rootfs image of any UML instance that will run under this version of 'linux'. Try mounting the rootfs over loop back, then running this
make modules ARCH=um
mkdir mnt
mount root_fs mnt -o loop
make modules_install INSTALL_MOD_PATH=mnt/ ARCH=um
HOSTFS Issues
We don't want the UML virtual machine to be able to mount the host's filesystem, so that means we can't compile hostfs support into the UML Kernel. Unfortunately, someone could still compile and install hostfs support as a kernel module, so that means we must build our UML Kernel without module support. Don't do the Kernel Modules install routine.
HOST Startup script (rc.local or likewise)
# turn on ip_forward
echo 1 > /proc/sys/net/ipv4/ip_forward
# install the skas host kernel module
insmod /usr/lib/uml/skas.o
# install the tun kernel module
insmod tun
Networking using Bridging
This is the correct way to network UMLs
Please follow the UML Bridging How To:
http://edeca.net/articles/bridging/
Networking using IP Forwarding (outdated/don't use)
This is what the UML net helper was doing. I wanted to do it manually, so I scripted two crappy bash scripts: tap_up and tap_down.
Host Main IP: 192.168.1.112 (this is used as every tap's host-side IP address)
UML IP: 192.168.1.200
bring up
tunctl -u caker -t tap0
* ifconfig tap0 192.168.1.112 netmask 255.255.255.255 up
* bash -c echo 1 > /proc/sys/net/ipv4/ip_forward
* route add -host 192.168.1.200 dev tap0
* bash -c echo 1 > /proc/sys/net/ipv4/conf/tap0/proxy_arp
* arp -Ds 192.168.1.200 eth1 pub
* arp -Ds 192.168.1.200 eth1 pub
bring down
* route del -host 192.168.1.200 dev tap0
* bash -c echo 0 > /proc/sys/net/ipv4/conf/tap0/proxy_arp
* arp -i eth1 -d 192.168.1.200 pub
* arp -i eth1 -d 192.168.1.200 pub
::::::::::::::
/etc/rc.d/tap-up
::::::::::::::
#!/bin/bash
# quick utility to configure a TAP device, and do some host setup
# by Christopher S. Aker
# tap-up user ifname privateIP externalIP extIF
ARGS=5
E_BADARGS=65
test $# -ne $ARGS && echo -e "Usage: tap-up USERNAME IFNAME PRIVIP EXTIP EXTIF\n tap-up caker tap0 10.1.1.200 192.168.1.200 et
h1" && exit $E_BADARGS
user=$1
device=$2
privateIP=$3
extIP=$4
extIF=$5
# delete the tap if it's already there
tunctl -d $device
tunctl -u $user -t $device
ifconfig $device $privateIP netmask 255.255.255.255 up
echo 1 > /proc/sys/net/ipv4/ip_forward
route add -host $extIP dev $device
echo 1 > /proc/sys/net/ipv4/conf/tap0/proxy_arp
arp -Ds $extIP $extIF pub
arp -Ds $extIP $extIF pub
echo "tap-up: Interface $device configured with $privateIP local, $extIP remote, dev $extIF"
::::::::::::::
/etc/rc.d/tap-down
::::::::::::::
#!/bin/bash
# Quick script to bring down a TAP device
# by Christopher S. Aker
# tap-down DEVICE
ARGS=1
E_BADARGS=65
test $# -ne $ARGS && echo -e "Usage: tap-down DEVICE\n tap-down tap0" && exit $E_BADARGS
device=$1
extIP=`/sbin/route -n | grep $device | awk '{print $1}'`
if [ -z "$extIP" ]; then
echo "Can't determine external IP address of $device, exiting." >&2
exit 1
fi
route del -host $extIP dev $device
echo 0 > /proc/sys/net/ipv4/conf/$device/proxy_arp
arp -i eth1 -d $extIP pub
arp -i eth1 -d $extIP pub
tunctl -d $device
echo "tap-down Interface $device down"
Filesystem Stuff
dd if=/dev/zero of=empty.2048 count=1 bs=1M seek=2048
cp -p --sparse=always empty.2048 rh.new
mkfs.ext3 -j rh.new
Can loop mount a COW file: NO
Can mount partitioned disk image: Unknown
This patch by McMechan patches the UML Kernel source code to vastly improve COW file creation times. He reduced the number of system calls to generate a 512GB from ~32,000,000 down to one. It's MUCH faster.
Runing UML!
Create the tap Device:
/etc/rc.d/tap-up caker tap0 192.168.1.112 192.168.1.200 eth1
Execute linux!
linux ubd0=rootfs ubd7=swapfs mem=64M con=null con0=fd:0,fd:1 eth0=tuntap,tap0,<optional MAC address>,<host side IP?>
This is what my HOST's routing table looks like:
[root@testbed /]# /sbin/route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.1.200 0.0.0.0 255.255.255.255 UH 0 0 0 tap0
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
0.0.0.0 192.168.1.30 0.0.0.0 UG 0 0 0 eth1
This is what the UML's routing table looks like:
[root@localhost root]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
0.0.0.0 192.168.1.112 0.0.0.0 UG 0 0 0 eth0
Swap, tmp and tmpfs
How much swap space? Host RAM+swap must be greater than the total amount of memory allocated to UMLs.
Example Config:
Host Machine RAM: 1024MB
Host Machine SWAP: 1024MB
I want to run 16 UMLs each with 256MB
That's a total of 4096MBs allocated to UMLs
My host machine needs at least 2GB more swap
|
What about tmp space? User Mode Linux createas a temp file as big as the RAM you allocate the UML instance. It's placed in your environment's TMPDIR path. Once useful strategy is to use the tempfs filesystem for your TMPDIR.
silug on IRC writes:
mkdir /tmp/uml
mount -t tmpfs -o mode=1777,size=512M none /tmp/uml
TMPDIR=/tmp/uml linux mem=512M ...<rest of linux command>
UML unlinks the file right after creating it. So, an ls -al /tmp/uml is going to show an empty directory. Check out "lsof" - a tool for viewing open files.
lsof - Lsof is a Unix-specific diagnostic tool. Its name stands for LiSt Open Files, and it does just that. It lists information about any files that are open by processes currently running on the system. It can also list communications open by each process.
./Configure linux
make
The preferred method is to have your normal /tmp direcrory mounted as tmpfs. Modify your /etc/fstab file:
tmpfs /tmp tmpfs defaults,size=7168M 0 0
Change size=7168M to no less than host ram+host swap, and no larger than the total of UML's memory
Resizing a Filesystem
Warning: Although this has been tested, you should make a backup in case something goes wrong.
If your filesystem image is 2048M, and you want to increase it to 4096M (4GB), run this
dd if=/dev/zero of=root_fs bs=1M conv=notrunc count=1 seek=4096
losetup /dev/loop0 root_fs
ext2resize /dev/loop0
losetup -d /dev/loop0
You will need ext2resize from Source Forge. (./configure && make && make install)
devfs options
You can use the devfs kernel option. If you decide not to use it, include "devfs=nomount" in your linux command line options.
Make sure you have the /dev/ubd devices created in your rootfs
for i in 0 1 2 3 4 5 6 7; do mknod ubd$i b 98 $i; done
chmod 660 ubd*
chgrp disk ubd*
ls -alt ubd*
Also, make sure to change your UML's /etc/fstab and /etc/inittab to reference the normal /dev/ devices.
NOTE: I've recently abandoned the old /dev/ filesystem in favor of devfs.
Consoles and Stuff
Launch linux like this:
linux con=null con0=fd:0,fd:1 .......
This tells linux that console "co" will be on standard in/standard out. The UML's /dev/vc/0 device will refer to stdin/out (the shell you launched linux from). If you are using devfs you need to modify your UML's /etc/inittab file. Comment out all getty the lines, and add:
/dev/ttys/ is a symlink to /dev/vc/
co:2345:respawn:/sbin/mingetty vc/0
You must also modify the UML's /etc/securetty and add vc/0.
Search Engine Stuff: howto, how-to, user mode linux, tips and tricks, cheat sheet, quickstart, bootstrap, hosting, virtual private server