CentOS 6.4, QEMU+KVM
It’s summer, it’s hot as hell, I am back home from mountains and I’ve plenty of free time.
Between a barbecue and the next one I spend my time playing with and learning new stuff: this week new stuff is called QEMU-KVM.
Yesterday I also tried XenServer but to be honest I wasn’t impressed, it just look like to be an old version of CentOS minimal install with some custom repos and a fancy GUI.
I played with it for just a bunch of hours, but the thing that just performing an installation on a software RAID-1 turned out to be a PITA to say the least is a clear sign that it’s not the best tool for my needs.
I swapped a couple of HDDs and in 2 minutes I went back to the already installed CentOS 6.4 with QEMU+KVM.
The client machine, for what it matters, is my Fedora 19 x86_64 workstation, virsh and virt-manager the tools I use for remote administration tasks.
Installing QEMU-KVM is just a matter of typing yum install libvirtd qemu-kvm bla bla bla, chkconfig libvirtd on and doing a system reboot (better safe than sorry).
The tricky part at least for me was setting up a damn bridged network interface, luckily I found this great writeup.
I am going to report here what I did to setup a couple of bridged network interfaces on my setup.
.:. Host side configuration
Here’s the desired setup:
eth0 192.168.0.254 which is the network interface/IP of the physical machine
eth0:0 192.168.0.252 which is mapped to VM-1 192.168.0.2
eth0:1 192.168.0.253 which is mapped to VM-2 192.168.0.3
First of all, on the physical machine, create 2 files in /etc/sysconfig/network-scripts/:
# FILE NAME: ifcfg-eth0:0
DEVICE=eth0:0
BOOTPROTO=static
ONBOOT=yes
IPADDR=192.168.0.252
NETMASK=255.255.255.0
# FILE NAME: ifcfg-eth0:1
DEVICE=eth0:1
BOOTPROTO=static
ONBOOT=yes
IPADDR=192.168.0.253
NETMASK=255.255.255.0
Once done creating the two files restart the network service…
service network restart
…and add a couple of iptables rules:
iptables -A PREROUTING -d 192.168.0.252 -i eth0 -p tcp -j DNAT --to-destination 192.168.0.2
iptables -A PREROUTING -d 192.168.0.253 -i eth0 -p tcp -j DNAT --to-destination 192.168.0.3
service iptables save && service iptables restart
Since every time libvirtd service starts it overwrite the iptables rules to make the settings permanent we have to create a new service to be runned after libvirtd startup or edit the file /etc/rc.local.
I edited rc.local adding what follow:
touch /var/lock/subsys/local
iptables -I FORWARD -i eth0 -o virbr0 -p tcp -m state --state NEW -j ACCEPT
iptables -I FORWARD -i eth0 -o virbr1 -p tcp -m state --state NEW -j ACCEPT
Do another full restart (better safe than sorry :: 2) and everything should work.
Last thing to do is tell the VMs to use the bridged network adapter; to do it from virt-manager, select a virtual machine, edit -> Virtual Machine Details, NIC and select the preferred eth0:x : macvtap device from the drop down menu.
In the same page, as Device model select virtio, this way after the next shutdown (not reboot) the virtual machine will be executed with the extra parameter vhost=on which will boost network performance.
Check that vhost_net module is correctly loaded with the command:
[root@server ~]# lsmod | grep vhost
vhost_net 30520 1
macvtap 9948 3 vhost_net
tun 17063 2 vhost_net
If vhost_net module is loaded, check if it’s actually used for the VMs with:
[root@server ~]# ps -ef | grep vhost
qemu 4721 1 4 17:51 ? 00:03:03 /usr/libexec/qemu-kvm -name torrent -S -M rhel6.4.0 -enable-kvm -m 256 -smp 1,sockets=1,cores=1,threads=1 -uuid e32dbfd9-fe79-fd4a-b7a8-58697db41a56 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/torrent.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/home/virtual-machines/torrent.img,if=none,id=drive-virtio-disk0,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net-pci,__com_redhat_macvtap_compat=on,netdev=hostnet0,id=net0,mac=52:54:00:62:52:18,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 127.0.0.1:0 -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
root 4723 2 1 17:51 ? 00:00:47 [vhost-4721]
root 5746 3701 0 18:56 pts/0 00:00:00 grep vhost
The string paramater vhost=on should be present in every QEMU-KVM related process (in my case the one with PID 4721).
.:. Guest side configuration :: static IP address
First of all, my virtual machines are all based on CentOS 6.4 minimal netinstall.
I like them to have a static IP address (VM-1 192.168.0.2 and VM-2 192.168.0.3), which is a pretty easy task.
[root@VM-1 ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth2
# add/edit the following lines and comment BOOTPROTO="dhcp" if present
IPADDR=192.168.0.2
GATEWAY=192.168.0.1
NETMASK=255.255.255.0
[root@VM-1 ~]# service network restart
Now the VM-1 should have a static IP address.
.:. Guest side configuration :: troubleshooting
If you happen to had deleted the virtual network adapter from virt-manager, when you add a new one it will have a different random MAC address and the guest OS will not see it anymore (CentOS prints an error during bootup).
The solution consists of two steps:
- edit the file
/etc/sysconfig/network-scripts/ifcfg-eth*with the new MAC address (the one shown in virt-manager) and the new device name (if it was eth0, write eth1, if it was eth1, write eth2).
There should be a way to force the new virtual network adapter to the use good old eth0, but I don’t know how to do it. - delete the file
/etc/udev/rules.d/70-persistent-net.rules. - ok, I lied, the third step is restart the virtual machine.
This is it