Network Virtualization

Improve networking performance in Virtual Machine environments. The larger context for this project is to provide the systems support for conducting large-scale network experiments using a PRIME-based virtualized network and systems testbed.

• Sajib Kundu
• Raju Rangaswami
• Jason Liu

• 01/20/09: A new problem – with unsynchronized packet and notification counts
• 01/07/09: CTA proposal discussion w/ Geoff and Jason
• 01/06/09: Problem (supposedly) identified - xen_recvmsg packet corruption
• 12/29/08: Xen-profiling results and prime_rx revision
• 12/15/08: Oprofile brief discussion
• 11/07/08: Oprofile findings and next steps
• 10/27/08: Project Update

The goal of this project is to explore various optimizations to improve inter-VM communication, specifically exploring the use of XenSockets and psuedo-devices for Linux VMs.

Current direction:

• Profile network data-path for XenSocket based communication using “oprofile”
• Optimize kernel-to-user-space copying of data between PRIME and the Linux kernel for packets forwarded by the simulation engine.

1. By default, xen kernel may not be enabled for profiling. So before installation, compile the kernel with profiling option enabled.
2. Once you are booted with the new kernel, install oprofile using the command (if you are using ubuntu version of linux)

apt-get install oprofile

  If you want to install oprofile from the sources, perform these folloeing steps:
  *   Download oprofile 0.9.3 source (oprofile-0.9.3.tar.gz) from  http://oprofile.sourceforge.net
  *   Download oprofile patch for Xen (oprofile-0.9.3-xen-rc2.patch) from http://xenoprof.sourceforge.net
  *   Compile and install oprofile modified for xen

> tar –zxvf oprofile-0.9.3.tar.gz

cd oprofile-0.9.3
patch –p1 < ../oprofile-0.9.3-xen-rc2.patch
./configure –with-kernel-support
make
make install

Type the command opcontrol –help. If successful the command should list –xen as a valid option.

Now your Domain0 is set up for use of oprofile. But you may have to perform the same steps for installing oprofile tool for virtual machines. The easiest way to install tools in VM is:
 * Dump the disk image of the VM under /mnt/
 * execute chroot /mnt/ /bin/bash
 * then type apt-get install oprofile. This steps will install oprofile in VMs.

Active profiling in Virtual Machines.

Imagine that you have dom1 and dom2 running along with dom0.
- Reset the machines
  dom0> opcontrol --reset
  dom1> opcontrol --reset
  dom2> opcontrol --reset
- Pass Parameters in Dom0
dom0> opcontrol --start-daemon --event=GLOBAL_POWER_EVENTS:1000000 --xen=/boot/xen-syms-3.0-unstable –vmlinux=/boot/vmlinux-syms-2.6.16.13-xen0 --active-domains=1,2
  --event specifies the activity you want to measure with oprofile.
  --xen specifies the xen image
  --vmlinux specifies the runnig kernel image in Dom0.
  --active-domains options asks for the list of dom ids for the target VMs
- Start profiling in DomUs
  dom1> opcontrol --start --event=GLOBAL_POWER_EVENT:1000000 --xen=/boot/xen-syms-3.0-unstable --vmlinux=/boot/vmlinux-syms-2.6.16.13-xenU
  Remember to copy the linux image and kernel image from Dom0 to /boot/ directory of Dom1. Also the events should be the same for virtual machine and dom0.
- Start Profiling in Dom0
  dom0> opcontrol --start
- Run the Benchmark
- Shutdown oprofile
  dom0> opcontrol --shutdown
  dom1> opcontrol --shutdown
  dom2> opcontrol --shutdown
- To get the statistics you need to perform opreport -l in each domain.
- To get the symbols for kernel modules that are inserted dynamically, perform this command
   opreport -l -p /home/xprime (If your modules are located under /home/xprime/)

[Sajib: start systematically recording your experiments and results here.]