Xen and Co.: Communication-Aware CPU Scheduling and Accounting in Xen


Recent advances in software and architectural support for server virtualization have created interest in using this technology in the design of consolidated hosting platforms. Since virtualization enables easier and faster application migration as well as secure co-location of antagonistic applications, higher degrees of server consolidation are likely to result in such virtualization-based hosting platforms (VHPs). We identify two shortcomings in existing virtual machine monitors (VMMs) that prove to be obstacles in operating hosting platforms, such as Internet data centers, under conditions of such high consolidation: (i) CPU schedulers that are agnostic to the communication behavior of modern, multi-tier applications and (ii) inadequate or inaccurate mechanisms for accounting the CPU overheads of I/O virtualization. We develop a new communication-aware CPU scheduling algorithm and a CPU usage accounting mechanism and implement these in the Xen 3.0.2 kernel.

Related publications:

Xen and Co.: Communication-aware CPU Scheduling for Consolidated Xen-based Hosting Platforms, Govindan, Nath, Das, Urgaonkar, and Sivasubramaniam, In Proceedings of the Third International ACM SIGPLAN/SIGOPS Conference on Virtual Execution Environments (VEE), June 2007 San Diego, CA.

Communication-aware CPU Management for Consolidated Virtualization-based Hosting Platforms”, Govindan, Nath, Das, Urgaonkar, and Sivasubramaniam, Technical Report CSE-06-017, Department of Computer Science and Engineering, The Pennsylvania State University, October 2006.


Sriram Govindan
Arjun R. Nath
Amitayu Das
Bhuvan Urgaonkar
Anand Sivasubramaniam

Evaluation summary (or what Xen and Co. can do for you):

Our experimental evaluation with realistic Internet server applications and benchmarks, reported in the paper above, demonstrates the performance/cost benefits and the wide applicability of our algorithms. For example, the TPC-W benchmark exhibited improvements in average response times between 20%-35% for a variety of consolidation scenarios. A streaming media server hosted on our prototype VHP was able to satisfactorily service up to 3.5 times as many clients as one running on the default Xen.

Download Xen and Co.:
Xen patch: xen_coord_patch

Xen 3.0.2 Code Changes: xen_coord_changes.tar

Instructions for using the code:

Use the above patch file (xen_coord_patch). Or else, download the above tar, untar it to extract the seven files that were modified (listed below). Replace these files in Xen 3.0.2 with those extracted from our tar. Compile the resulting Xen kernel, and you are ready!

Notes on our implementation:

The following files were modified :

Scheduler modifications:


Device driver modifications:


Code sections modified are enclosed within comments
// Xen-CoOrd changes start
modified code
// Xen-CoOrd changes ends

Questions or comments:

In case if you have any doubts or questions, kindly write to: sgovinda@cse.psu.edu