XenServer + DRBD (Update 4) The Guide

drbd guide

*** Is you are using XenSever 5.6 FP1 please see http://joekane.eu/xenserver-56-fp1-drbd-working first.

Here is the DRBD + XenServer install guide, Any issues post a comment and I'll try my best to resolve.

###### DRBD + Xenserver Guide: JoeKane.eu ########

Create DRBD RPM for installation on XenServer

Download the Driver Development Kit (ISO) from http://www.citrix.com/lang/English/lp/lp_1688621.asp
Mount or Extract the ISO and import using XenCenter - Be sure to select XML as file type
$ mkdir /drbd/
$ cd /drbd/
$ wget http://oss.linbit.com/drbd/8.3/drbd-8.3.7.tar.gz
$ tar drbd-8.3.7.tar.gz
$ cd /drbd-8.3.7
$  ./configure --enable-spec --with-km
$ make tgz
$ cp drbd*.tar.gz `rpm -E _sourcedir`
$ rpmbuild -bb drbd.spec
$ rpmbuild -bb drbd-km.spec

RPM is located in /usr/src/redhat/RPMS/i386/

See: http://www.drbd.org/users-guide/s-build-rpm.html

If you want to skip this step I have attached the RPM's needed!

Download RPM's:

DRBD 8.3.7 + XenServer 5.5 U2 RPM: http://bit.ly/dmvv0g

DRBD 8.3.7 + XenServer 5.6 Beta RPM: http://bit.ly/9hHNEt

DRBD 8.3.7 + XenServer 5.6 Final RPM: http://bit.ly/diYtbY

DRBD 8.3.8.1 + XenServer 5.6 Final RPM: http://bit.ly/cNboke

If you are using XenSever 5.6 FP1 please see http://joekane.eu/xenserver-56-fp1-drbd-working first.

---------------------------------------------------------------------------

Install XenServer 5.5 U2 on each server – Don’t configure local storage - Connect the servers together using a crossover cable.
My setup has 1x 80GB drive for XenServer install and 1x 250GB drive for the DRBD storage

1.    Install DRBD (Both Servers)

Copy drbd-km-2.6.18_128.1.6.el5.xs5.5.0.505.1024xen-8.3.7-12.i386.rpm & drbd-utils-8.3.7-1.i386.rpm to your XenServer /tmp directories – Use Winscp for this

$ rpm -ivh drbd-utils-8.3.7-1.i386.rpm
$ rpm -ivh drbd-km-2.6.18_128.1.6.el5.xs5.5.0.505.1024xen-8.3.7-12.i386.rpm

2.     Configure another Management Nic's for each XenServer

The IP addresses I’m using are 10.1.1.1 & 10.1.1.2 (255.255.255.0 netmask)

3.    Fdisk the Drives (Both Servers)
Be sure to configure your Drive using Fdisk, you can see your partition table using
$ Fdisk –l
I won’t go into too much detail here; fdisk is very easy to use, in my setup I have the 250GB as /dev/sda and a single partition /dev/sda1

4.    Configure DRBD (Both Servers)

 Nano /etc/drbd.conf

# ----------------------
# You can find an example in  /usr/share/doc/drbd.../drbd.conf.example

#include "drbd.d/global_common.conf";
#include "drbd.d/*.res";

resource drbd-sr1 {
protocol C;
startup {
}

disk { max-bio-bvecs 1;
}

net {
allow-two-primaries;
cram-hmac-alg "sha1";
shared-secret "password";
after-sb-0pri discard-zero-changes;
after-sb-1pri consensus;
after-sb-2pri disconnect;
}

syncer {
rate 1G;
}

on xenserver-drbd1 {
device /dev/drbd1;
disk /dev/sda1;
address 10.1.1.1:7789;
meta-disk internal;
}
on xenserver-drbd2 {
device /dev/drbd1;
disk /dev/sda1;
address 10.1.1.2:7789;
meta-disk internal;
}
}
# --------------------

4. Configure Notifications in case of Split brain. (Both Servers)

nano /usr/lib/drbd/notify.sh

# Add to the bottom of the file

esac

#echo "$BODY" | mail -s "$SUBJECT" $RECIPIENT
HOST_UUID=`xe host-list --minimal`
xe message-create body="$BODY" host-uuid=$HOST_UUID name=DRBD_ATERT priority=10

#I also changed the "BODY" message because it was received empty.

case "$0" in
*split-brain.sh)
SUBJECT="DRBD split brain on resource $DRBD_RESOURCE"
BODY=" Split brain detected, Manual split brain recovery is necessary! "
# BODY="
#DRBD has detected split brain on resource $DRBD_RESOURCE
#between $(hostname) and $DRBD_PEER.
#Please rectify this immediately.
#Please see http://www.drbd.org/users-guide/s-resolve-split-brain.html for details on doing so."
;;

-------------------------------

5.    Allow port 7788 through Iptables (Both Servers)

Get your interface alias using $ ifconfig

(Thanks Sam)

Ifconfig


-------------------------------
$ iptables -I INPUT 1 -i xenbr2 -p tcp --dport 7788:7799 -j ACCEPT
$ /etc/init.d/iptables save
-------------------------------

6.    Configure LVM to see your Drive (Both Servers)

nano /etc/lvm/lvm.conf

# By default we accept every block device:

Add your block device "r|/dev/sda.*|" to the filter,

e.g. filter = [ "r|/dev/xvd.|", "r|/dev/sda.*|" ]

#Set the write cache state to 0 default is 1

write_cache_state = 0

---------------------------------

7. Remove LVM Cache (Thanks PF4)

rm -f /etc/lvm/cache/.cache

--------------------------------


8.    Create Xenserver pool using XenCenter

---------------------------------

9.    Enable DRBD Storage and Replicate

On both XenServers execute the following

$ drbdadm create-md drbd-sr1
$ modprobe drbd
$ drbdadm up drbd-sr1

Now ONLY on the XenServer that will be your primary

$ drbdadm -- --overwrite-data-of-peer primary drbd-sr1
$ cat /proc/drbd

After Full Sync - Make pool secondary drbd primary "$ drbdadm primary drbd-sr1"

$ cat /proc/drbd will not show Primary:Primary

10.    Create Storage Repository

$ xe sr-create shared=true device-config:device="/dev/drbd1" name-label="DRBD-SR1" type=lvm

$ chkconfig drbd on (on both XenServers)

That’s it - install your VM’s and test VM migration.
Any Questions post a comment.

Thanks go to,
Florian Haas from linbit.com (The Makers of DRBD): http://fghaas.wordpress.com/2007/09/03/drbd-806-brings-full-live-migration-fo...
Natanael Mignon of http://natanael-mignon.blogspot.com/ - Gave the starting steps to get everything working.
Citrix for providing XenServer and the Citrix Forums
Loads of website related to DRBD configs and the DRBD Mailing list.

*** Small Update - Dont forget to add the names of your servers to /etc/hosts using your crossover IP addresses