Oh, goodness! It seems that Greg Shields’ attempt at “How to Correctly Explain the Architectural Differences Between Hyper-V and ESX” isn’t all that correct. Oh, he starts out pretty well, classifying both Hyper-V and ESX as Type-1 hypervisors, which is correct. Where he goes astray is when he claims that Hyper-V utilizes “paravirtualization” and ESX relies on “hardware emulation” – wrong!
As a matter of fact, both Hyper-V and ESX use paravirtualization. Both Hyper-V and ESX, can run non-paravirtualized operating systems because they offer a set of device drivers that emulate some very popular physical devices (VGA adapter, network adapter, disk controller) that are supported – out of the box – by most operating systems. Once the guest operating system is up and running, you have the option to install the paravirtualized drivers – Enlightenments for Hyper-V and VMware Tools for ESX. So, the assertion that Hyper-V derives an advantage over ESX through the use of paravirtualization is totally bogus.
Where Hyper-V does require paravirtualization is in the processor. Hyper-V cannot work without having “virtualization support” (i.e. Intel VT or AMD-V) baked into the CPU. ESX will take advantage of these technologies when they are available, but also has the ability to use binary translation to run VMs on hosts that do not have CPUs that provide hardware assisted virtualization.
Can we please lay this paravirtualization myth to rest?
Next, we have the argument that Hyper-V is “microkernelized” because its device drivers are implemented in the administrative OS. Well, that makes it a distributed OS, not a microkernel based OS. Take a look at how Wikipedia defines “Microkernel”
In computer science, a microkernel is a computer kernel that provides the mechanisms needed to implement an operating system, such as low-level address space management, thread management, and inter-process communication. If the hardware provides multiple privilege levels, then the microkernel is the only software executing at the most privileged level (generally referred to as supervisor or kernel mode). Actual operating system services, such as device drivers, protocol stacks, file systems and user interface code are contained in user space.
Loading device drivers in the Primary Partition does not make Hyper-V a micro-kernel based OS (I believe it probably is a micro-kernel based OS, but not because of where it loads its device drivers!), nor does loading the device drivers into the vmkernel make ESX a monolithic OS. Both Hyper-V and ESX are modular operating systems, meaning that they extend their functionality through the use of “modules” that interface with the core OS via a well defined set of interfaces.
What Greg is trying to describe (or so I believe) is the difference in how Hyper-V and ESX manage their device drivers. With Hyper-V, the device drivers are loaded in the management partition (primary partition) and with ESX, they are loaded by the vmkernel into usermode space within the vmkernel. I think it’s time for a diagram to illustrate – see Figure 1:
Figure 1. Hyper-V & ESX - Architectural Differences
Notice that Hyper-V has device drivers living within the Primary Partition and ESX has device drivers that get loaded by the vmkernel. Not a huge deal, unless you consider that, in the case of Hyper-V, if the Primary Partition (Windows 2008) fails, your VMs lose the ability to talk to the outside world (if they survive at all), whereas with ESX, if the service console (modified Red Hat Linux) fails, your VMs continue to operate without problem.
As for the comment:
Ever wonder why it takes VMware forever to add support for new hardware? That’s because any new driver or hardware support has to be specifically encoded, tested, and integrated into that hypervisor.
Again, wrong! The primary reason it takes “so long” to provide new hardware support is because ESX restricts the HCL to improve reliability. Every component on the HCL has been thoroughly tested and is certified to be compatible with ESX. There is some integration work that has to be done, but no more so than with any other operating system (including Windows!). The difference is that VMware has tighter control over the device driver eco-system than does Microsoft. If you look at Windows hosts, a frequent cause of outages is device drivers developed by third parties that have undergone third party certification. Yes, you get support quickly, but do you get robust support? Sometimes yes, sometimes no.
Then we have this interesting series of statements:
1. It doesn’t matter how big your administrative OS is.
2. With local disk drives so big these days, your administrative OS (called the “primary partition” in Hyper-V and Xen) can be 10K in size or it can be 10G.
3. That administrative OS is merely there so you can manage the rest of the box.
4. Further, because Hyper-V’s administrative OS (“primary partition”) has all this nice and already-existing support for drivers built into the OS, you can argue that it will always have a greater level of driver support than ESX.
Let’s take these one at a time:
1. If it’s truly an administrative OS (as is the case with ESX), I could maybe buy this argument; however, with Hyper-V, the primary partition (that administrative OS) is a key and integral part of the virtualization solution (see item #4).
2. Waste not, want not.
3. That is true for ESX; it is not true for Hyper-V (see item #4)
4. With Hyper-V, the “administrative OS”/”primary partition” provides the device drivers that allow the virtual machines to interact with the underlying hardware. The size and complexity that Greg was disparaging a few paragraphs ago come up in spades in the primary partition. Every new device driver, feature enhancement, user interface modification, etc. that gets incorporated provides a new opportunity for a conflict or incompatibility – not to mention that there’s nothing to stop an administrator from installing whatever application they want in the primary partition – it is, after all, just “another Windows OS instance”.
While we’re on the topic of device drivers, let’s look a little deeper. What are some of the key cornerstones of virtualization? In my book, you have two fundamental aspects:
- Encapsulation. A virtual machine is encapsulated in a group of files and the boundaries are clearly known.
- Portability. The ability to move a virtual machine (running or not) from one host to another without concern about the underlying hardware.
Both Hyper-V and ESX do a great job with #1; however, let’s consider #2. In Hyper-V (per Ben Armstrong – Microsoft virtualization Program Manager), “while we do provide high performance drivers, we also provide emulated devices for unsupported operating systems. For systems using the high performance drivers there is clean abstraction at the VMBus layer.” This means that – for supported OSes – there should be no problem with VM portability. What I’m not clear on is whether unsupported VMs have the same degree of portability, or if they are tightly coupled to the underlying hardware…Ben, can you clarify, please?
The current version of Hyper-V is also restricted by its use of NTFS as a filesystem. Since NTFS is not a clustered filesystem (in its current encarnation – supposed to change in W2K8 R2), a single LUN cannot be “owned” by multiple hosts at the same time. This is the lynch pin in the “live migration” issue – since you can’t have two hosts sharing access to the same LUN, Hyper-V is unable to live migrate a running VM. This will change with Windows Server 2008 R2 when Hyper-V will offer live migration support.
So…let’s recap for everyone’s benefit:
- Hyper-V and ESX are both “Type-1” hypervisors: TRUE
- Hyper-V is a “microkernelized” hypervisor and ESX is a “monolithic” hypervisor: FALSE
- ESX drivers exist within its hypervisor: TRUE, but are dynamically loaded and run at a lower privilege level than the core hypervisor functions
- It doesn’t matter how big your administrative OS is.: TRUE, if it’s really an administrative OS. FALSE in the case of Hyper-V which relies on the primary partition for critical virtualization functionality
- Further, because Hyper-V’s administrative OS (“primary partition”) has all this nice and already-existing support for drivers built into the OS, you can argue that it will always have a greater level of driver support than ESX.: TRUE – but is that a good thing?
- That means Hyper-V will always tend to see faster raw performance.: FALSE In both cases (Hyper-V and ESX), there’s a degree of abstraction, which introduces some overhead. In neither case is the overhead “significant”. In no circumstance would I make the comment that Hyper-V is always faster; nor would I make the assertion that ESX is always faster. The performance for a particular device is dependent upon the device driver implementation as well as the entire software stack that deals with the device. I’d wager that Hyper-V will perform better under some circumstances and ESX will perform better under others.
I hope I’ve cleared up some of the confusion. I know it’s rampant and people are looking for some clear understanding and guidance. To be fair, I am biased toward VMware. I’ve made my living by using and supporting VMware technologies for a long time…