Search Ali's Blog

Thursday, December 09, 2010

Xen Installation on CenOS-5.5

As part of my work, I had to bring up Xen Servers setup, and install some virtual machines.
I've been working on VMware ESX Servers for a while, and I must admit that VMware products are way more mature than Xen products. Below step by step installation of Xen over CentOS-5.5 and how to bring up and configure DomU Virtual Machines.

Dom0 Installation:
1. Install CentOS-5.5 OS
   ISO is downloadable from here

2. Install Xen kernel and Tools:
# yum install kernel-xen kernel-xen-devel xen

# yum install virt-manager
# yum install virt-viewer
3. Edit /etc/grub.conf and select xen kernel

title CentOS (2.6.18-194.26.1.el5xen)
        root (hd0,0)
        kernel /xen.gz-2.6.18-194.26.1.el5
        module /vmlinuz-2.6.18-194.26.1.el5xen ro root=LABEL=/ rhgb quiet
        module /initrd-2.6.18-194.26.1.el5xen.img
title CentOS (2.6.18-194.el5)
        root (hd0,0)
        kernel /vmlinuz-2.6.18-194.el5 ro root=LABEL=/ rhgb quiet
        initrd /initrd-2.6.18-194.el5.img
4. Reboot the machine
5. Check kernel name, it will have xen substring:

# uname -a
Linux 2.6.18-194.26.1.el5xen ...
6. On my machine, yum installed Xen version 3.0.3:

# rpm -qa|grep -i ^xen-[0-9]

DomU Installation:
1. run virt-manager from dom0
2. select server name in the table, and select "new"

3. configure the virtual machine, when you get to the installation media, you better have the installation dir (extracted iso) available through on your network (check this link if you want to bring up your own apache) note that for full Linux installation (e.g., for RHEL-5.5) you need more than 4GB virtual disk space (I use 8GB).
4. when installation is done, machine will reboot and come up with bridged network connection
5. If you configured your VM to use DHCP, then you're ready, go ahead and ping the WAN.
- For more info on how to use virt-manager to install DomU instance (with snapshots) check this link.

DomU Network Configuration:
Xen supports three modes for network virtualization:
Bridging, Routing w/o NAT, Routing with NAT, each one of them need to be configured differently.

By default, the network is set up in bridging mode.
To change the network configuration, there are two main configuration files:
# /etc/xen/xend-config.sxp (hypervisor conf)

# /etc/xen/VM-NAME (virtual machine conf)

DomU in Bridging mode:
1. Make sure that xend-config.sxp has the following lines:
   (network-script network-bridge)

   (vif-script vif-bridge)
   To use different netdev (rather than eth0) use the line:
   (network-script 'network-bridge netdev=eth1')

   To create multiple bridges, you must create your own script.
   For example, in 
xend-config.sxp add:
   (network-script network-bridge-multi)
   The cerate the file (with +x permissions):   # cat /etc/xen/scripts/network-bridge-multi
   dir=$(dirname "$0")
   "$dir/network-bridge" "$@" vifnum=0 netdev=eth0 bridge=xenbr0
   "$dir/network-bridge" "$@" vifnum=1 netdev=eth1 bridge=xenbr1
   "$dir/network-bridge" "$@" vifnum=2 netdev=eth2 bridge=xenbr2
   check this link for more details.
2. Configure DomU virtual interfaces,
   Edit /etc/xen/VM-NAME and examine the following line:

   vif = [ "mac=00:16:36:69:0d:0b,script=vif-bridge,bridge=xenbr2" ]
   where xenbr2 is an example of the bridge name
3. Note that if the netdev of the bridge your going to use was created after machine reboot, then you need to restart the xen network script, for example:
# /etc/xen/scripts/network-bridge stop 
# /etc/xen/scripts/network-bridge start
If the pethX interface you expect does not come up, try to reboot the machine (I noticed that in Xen 3.0, the network-bridge may fail to create the pethX interface if you restart the script few times, reboot solved the issue)
4. Make sure that the bridge as appear in brctl is output, is configured as expected:
#brctl show xenbr2
bridge name     bridge id  STP enabled  interfaces
xenbr2          8000.00..  no           vif2.0
If you configured your system in the past to work in routing mode, you may see virbr0 instead of xenbr0, to revert these changes, run:
# service libvirtd stop; chkconfig libvirtd off
If for some reason the peth2 that coresponds to your netdev is not attached to the bridge, then find your peth:
# ethtool -i peth2
and then add the interface to the bridge:
#  brctl addif xenbr2 peth2
If for some reason, peth2 link is down, run:
# ip link set peth2 up

At this point you can run the VM, configure the IP, and heck ping-pong, you can tcpdump the bridge (xenbr2) and its slaves to check the packets flow.

DomU in Routed mode:
Note that most of the tips of this section are based on this link.
1. Update your grub entry to include the following parameters:

title Kernel-XXX-xen
    root (hd0,5)
    kernel /boot/xen.gz 
    module /boot/vmlinuz-XXX-xen root=/dev/sda6 vga=0x31a resume=/dev/sda5 splash=silent showopts
    module /boot/initrd-XXX-xen
2. Set netloop driver module parameter, edit /etc/modprobe.conf and include the line (to top netloop from creating 8 useless vifs):
options netloop nloopbacks=0
3. Reboot Dom0
4. Configure Dom0 networking, edit /etc/xen/xend-config.sxp and include the line:

(network-script network-route)
(vif-script     vif-route)
5. Configure DomU networking, edit /etc/xen/VM-NAME and include the line:
vif = [ 'mac=00:16:3e:33:22:11, ip=, vifname=eth0' ]
Where the IP address, is the IP address you plan to give to vifname interface within the VM.
6. Power on the VM, and conifgure (for example) the eth0 interface to use the IP=
7. Ping the IP address of Dom0 (run ifconfig from Dom0 and check your netdev IP, must start with 192.x.x.x [depends on your mask]).
8. To ping other machines outside the GW, need to configure the default GW in the VM, this can be done in many different ways, for example:
route add default gw eth0

- To install KVM on CentOS, check this link

- If you get the error message:
ERROR    unable to connect to 'localhost:8000': Connection refused
You probably don't have xen-http-server enabled, edit:
and include the line:
(xend-http-server yes)
then restart xend:
/etc/init.d/xend restart


Adrián y Eli said...

I don't understand very much about computers but it's a very nice blog...
Would you visit me and give me your opinion?

hamza khanzada said...

i read the blog the blog is very informative
i liked it
hamza khanzada