► How to migrate Xen from version 3.2 to version 4 on GNU/Debian Lenny

La version française est disponible ici: Migration de Xen version 3 vers la version 4 sous GNU/Debian Lenny

Introduction

Xen is an opensource hypervisor. Version 4 has been released on the 7th of april 2010.

One of the main new feature concerns high availability. With the intergation of project Remus code, Xen now supports fault tolerance for virtual machines, using live transactional synchronization of VM state between physical servers.

Unfortunatly, Debian is only shipped with Xen version 3.2 for current stable Lenny, and version 3.4 for next stable Squeeze. Let's see how to install Xen 4 on Lenny.

Current system inventory

The target Xen platform we want to upgrade is discribed here (sorry for my non french readers, theses 2 documents are not translated yet):

Despite the use of cluster stack in the second case, installation will follow the same steps for both architectures.

But we have to take care about some things in order to complete migration:

  • We are about to upgrade Xen hypervisor. We will also have to upgrade Dom0 kernel (during my tests, Debian 2.6.26 kernel crashed while booting with Xen4 and I don't have time to investigate it for now).
  • Because we update Dom0 kernel, we must build a new DRBD module accordingly
  • Finally, because we'll build Xen from sources, we have to clean Debian install in order to avoid libraries conflicts.

All compilations will be done on a specific computer. For obvious security reasons, it's of course out of question to compile Xen on the Dom0. Remember ? "Never install dev tools on productions servers" ;-)

Environment setup

Inside our dedicated VM, tou must install needed packages for compilation:

Dev environment setup 
aptitude install bcc bin86 gawk bridge-utils iproute libcurl3 libcurl4-openssl-dev bzip2 module-init-tools transfig tgif texinfo
             texlive-latex-base texlive-latex-recommended texlive-fonts-extra texlive-fonts-recommended pciutils-dev mercurial
             build-essential make gcc libc6-dev zlib1g-dev python python-dev python-twisted libncurses5-dev patch libvncserver-dev
             libsdl-dev libjpeg62-dev iasl libbz2-dev e2fslibs-dev git-core uuid-dev gcc-multilib

Xen compilation

Xen compilation steps 
mkdir ~/Xen
cd ~/Xen
wget http://bits.xensource.com/oss-xen/release/4.0.0/xen-4.0.0.tar.gz
tar -xzf xen-4.0.0.tar.gz
cd xen-4.0.0/
make xen
make tools

As we'll use Debian provided Dom0 kernel, we don't need to build it. The make stubdom command is therefore not required.

Binaries have copied into dist directory. You'll find a script called install.sh which will take care of copying files in destination system tree. System files are located dist/install. In order to make deployment easier, we'll build a tar.gz archive from direcory dist.

But first, we have to modify some files to respect Debian's usages for configuration files stroage: we have to move etc/sysconfig to etc/default. Then, we have to adapt init scripts.

Adaptation de l'installation aux normes Debian 
cd dist/install/etc
mv sysconfig default
cd init.d

Replace sysconfig with default (line 23) in xend.

Replace sysconfig with default (line 38) in xendomains.

Tar.gz archive creation for Xen 4
tar -czf xen4.tar.gz dist

Then, you can copy this archive on each dom0 you want to update. Let's deal now with DRBD module.

DRBD compilation

Our next dom0 kernel will be 2.6.32 from backports repos. We have to build it. As our dev VM does not use it, we must specify kernel version module-assistant will have to use to build against.

I only write a summary of needed commands, details are already here (still only in french, sorry): Installation de DRBD > 8.3.2 sur Debian GNU/Linux Lenny

DRBD packages build
apt-get install linux-headers-2.6.32-bpo.4-xen-amd64 module-assistant
dpkg -i drbd8-module-source_8.3.7-0_all.deb
m-a build -l 2.6.32-bpo.4-xen-amd64 drbd8-module

Dom0 configuration

Before beginning, ensure all your VMs have been migrated (or stopped, depending on you configuration).

First install new kernel from backports:

Kernel 2.3.32 install
echo "deb http://www.backports.org/debian lenny-backports main" >> /etc/apt/sources.list
aptitude update
aptitude install debian-backports-keyring
aptitude update
aptitude install linux-image-2.6.32-bpo.4-amd64 linux-image-2.6.32-bpo.4-xen-amd64

Now reboot the server. If everything is fine, you can uninstall all xen related packages as well as the old kernel. As usual, do not forget to backup your Xen configuration !

Once done, you can install DRBD module and Xen4 archive:

DRBD install 
dpkg -i drbd8*.deb
Xen4 install 
tar -xzf xen4.tar.gz
cd dist
./install.sh
echo "none /proc/xen xenfs defaults 0 0" >> /etc/fstab

Without the last command, xend process will be unable to communicate with hypervisor and therefore, nothing will work. You may have to add some others modules in /etc/modules:

Adding other Xen modules 
blktab
xen-evtchn

Now it's time to restore Xen's config files. Finally, setup xen related services start xend and xendomains.

Xen4 services setup 
update-rc.d xend defaults 20 21
update-rc.d xendomains defaults 20 21

When done and before rebooting on hypervisor, you should install some libraries. They are needed for Xen to work, but have been uninstalled by Debian (because you remove packages).

Xen4 needed librairies install
aptitude install libpci3 libsdl1.2debian libx11-6 libxext6 libgl1-mesa-glx

Now you can reboot and start Xen4 hypervisor. Let's check Xen status with command xm list.

Et voilà, now you have a completly new Xen4 dom0 running Debian Lenny. Have fun !

Sources and references

Xen
DRBD
GNU/Debian Linux
via publications.jbfavre.org

Good doc - Notes