Summer 2009 Research
The purpose of this page is to list a research plan for the summer and a number of papers that are relevant to the plan.
Overview
Virtual machines are very popular now for "solving" all sorts of administrative and management problems. Unfortunately, nobody really knows exactly how they affect performance. We are attempting to quantify and qualify the performance impact of VM execution according to machine-level events (like translation lookaside buffer misses, L2 cache misses, etc.). Essentially, we will execute some applications a number of times, and gather data about events that may affect application performance.
We recently submitted a paper to MASCOTS09 detailing a little bit of the work we have done on this. A lot more characterization needs to be done in order for us to use the data to make predictions about how well arbitrary applications will run inside VMs.
The Plan
The goal of this research is to be able to make predictions about how application performance changes when executing inside VMs. Basically, this entails gathering a lot of machine-level events, and determining the significant factors that affect application performance. By and large, the analysis of the events is statistical. Here is a short list of data we want.
- Event Mean per Application
- Event Variance/Standard Deviation per Application
- Mean Deviation
- Correlation between environments
- Correlation between events
- Correlation between applications
- Confidence level to 95%
- All of the above, done for application performance
In addition to these statistics, we also want things like
- Cycles Per Instruction
- TLB Miss/Hit Rate
- L1 and L2 Miss/Hit rate
These are an agglomeration of the events we already have. For example, Cycles Per Instruction = Number of Instructions Retired / Number of Unhalted CPU Cycles
We want data that compares these dimensions and tells us useful things about how VM execution affects the application's performance. For example, one result might be that the number of instructions executed has basically no use in predicting application performance. Another result might be that Nehalem VM performance is much better than Harpertown VM performance because TLB misses occur less often. What we have is a bunch of data with at least 4 dimensions to it (depending on how you partition things).
- Environment (currently Linux, Xen Domain 0, Xen Domain U)
- Application and problem size and spread (HPL with 1, 2, 4 cores, 2 nodes, 4 nodes, ...)
- Event (TLB misses, Instructions executed, ...)
- Machine Architecture (Nocona, Harpertown, Nehalem)
Assorted Papers
Classic Papers
Title : Formal Requirements for Virtualizable Third Generation Architectures
Authors : Gerald Popek and Robert Goldberg
Year : 1974
Place : Communications of the ACM
Title : Architecture of Virtual Machines
Authors : Robert Goldberg
Year : 1973
Place : Workshop on Virtual Computer Systems
Title : The Origin of the VM/370 Time-Sharing System
Authors : Robert J. Creasy
Year : 1981
Place : IBM Journal of Research and Development
Virtual Machines
Title :Xen and the Art of Virtualization
Authors :B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, I. Pratt, A. Warfield, P. Barham, and R. Neugebauer
Year : 2003
Place : ACM Symposium on Operating Systems Principles
Title : AMD64 Virtualization: Codename Pacifica Technology
Authors :
Year : 2005
Place : AMD
Title : Intel Virtualization Technology: Hardware Support for Efficient Processor Virtualization
Authors : Gil Neiger, Amy Santoni, Felix Leung, Dion Rodgers, and Rich Uhlig
Year : 2009
Place : Intel Technology Journal
Title : Optimizing network virtualization in xen
Authors : Aravind Menon, Alan L. Cox, and Willy Zwaenepoel
Year : 2006
Place : USENIX Annual Technical Conference
Title : QEMU, a Fast and Portable Dynamic Translator
Authors : Fabrice Bellard
Year : 2005
Place : USENIX 2005 Annual Technical Conference
Title : Virtual Machine Monitors: Current Technology and Future Trends
Authors : Mendel Rosenblum and Tal Garfinkel
Year : 2005
Place : Computer
Performance Measurement and Analysis
Title : Diagnosing Performance Overheads in the Xen Virtual Machine Environment
Authors : Aravind Menon, Jose Renato Santos, Yoshio Turner, G. (John) Janakiraman, and Willy Zwaenepoel
Year : 2005
Place : VEE '05: 1st ACM/USENIX International Conference on Virtual Execution Environments
Title : How Well Can Simple Metrics Represent the Performance of HPC Applications?
Authors : Laura C. Carrington, Michael Laurenzano, Allan Snavely, Roy L. Campbell, Larry P. Davis
Year : 2005
Place : SC '05: Proceedings of the 2005 ACM/IEEE conference on Supercomputing
Title : Modeling and predicting application performance on parallel computers using HPC challenge benchmarks
Authors : W. Pfeiffer and N.J. Wright
Year : 2008
Place : IEEE International Symposium on Parallel and Distributed Processing
Title : A simulation based study of TLB performance
Authors : J. Chen, Anita Borg, and Norman Jouppi
Year : 1992
Place : SIGARCH Computer Architecture News
Title : Automatically characterizing large scale program behavior
Authors : Timothy Sherwood, Erez Perelman, Greg Hamerly, and Brad Calder
Year : 2002
Place : ASPLOS-X: Proceedings of the 10th international conference on Architectural support for programming languages and operating systems
Title : Measuring Cache and TLB Performance and Their Effect on Benchmark Runtimes
Authors : Alan J. Smith and Rafael H. Saavedra
Year : 1995
Place : IEEE Transactions on Computers
Virtual Machines in Clusters
Title : Dynamic Virtual Clustering
Authors : Wesley Emeneker and Dan Stanzione
Year : 2007
Place : IEEE International Conference on Cluster Computing