Nesting Hyper-V with VMware Workstation 8 and ESXi 5

As Veeam moves forward and starts to become focused two virtualization platforms, I thought to myself now is the time to work out how I’m going to introduce Microsoft Hyper-V in my lab environment. I wanted to know if it could be virtualized running inside a VMware virtual machine of some sorts.

Veeam has been very successful in presenting its solutions through the use of portable labs running for example on laptops nested under VMware Workstation. We would often have a nested ESX server, vCenter, DC and Veeam apps running on a single laptop and I wanted to know if Hyper-V could be thrown into the mix.

This post will function as a how-to guide and provide a step-by-step process to run Hyper-V virtual machines on either VMware Workstation 8 or ESXi 5.

For a long time, I was told it was impossible, and a few months ago, I heard a passing rumour that it would be possible when ESXi 5 came out. I’d also heard that ESXi 5 running on Intel Nehalem or Intel Core i7 was going to allow nested hypervisors to also contain running, nested 64bit virtual machines. So firstly, I went about making sure that when I was due for a new laptop, I would have this Intel architecture, or equivalent AMD, in my system. I also managed to build a lab with the same architecture that I plan to use on the road for events.

Through Twitter, someone sent me a link to a few blogs that helped me start my quest:

http://www.vcritical.com/2011/07/vmware-vsphere-can-virtualize-itself/#comment-12442

http://www.virtuallyghetto.com/2011/07/how-to-enable-support-for-nested-64bit.html

By the time I’d gotten a chance to give it a whirl, ESXi had gone GA. What I found from the blogs posted above was there were some great pointers, but when I followed either post’s instructions, all I got was a blank Black Screen like a few other people were reporting. No matter what additional information I gave, it just would not work. Undeterred, I decided to try something different.

I installed VMware Workstation 8 instead of ESXi 5 and managed to get a nested Hyper-VM working. At this point, I knew my hardware was going to be compatible with nesting Hyper-V. The blog posts above stated the key to making it work is a feature found in your CPU/BIOS called Intel EPT. Now, I read that if you have Nehalem/Core i7, you should have Intel EPT; the blog posts suggested that it should be enabled through the BIOS. However, I didn’t find this option in either of my system BIOS.

While testing what would work, I decided to use Windows 2008 R2 Standard and enable Hyper-V as a Role rather than install the standalone Hyper-V product. I only did this to make my life easier, but the standalone Hyper-V product is a fine option as well.

Nesting a Hyper-VM on VMware Workstation 8

So, following are the steps to create a Microsoft Hyper-V VM running in VMware Workstation 8, but later I’ll show you how to do it in ESXi 5 as well:

1. Create a New VM with version 8 hardware
   
2. Give it 4 GB RAM and 2 x vCPUs with about 80-100 GB disk space, depending upon how many VMs you wanted nested underneath Hyper-V.
3. The instructions lead you to believe that you should pick a VMware ESX option as the guest OS… STOP! DON’T!
   Select Windows 2008 R2 x64.
   
4. When you are finished, make sure you add another NIC to the VM used as the Hyper-V virtual network.
5. Under the settings of the VM > CPU, make sure you have the option to pass-through the Intel VT-x/EPT feature.
   
6. Make sure you have set the VM to boot from Windows 2008 R2 x64 media ISO.
7. Before booting, you should edit the config file .vmx and add the parameter: hypervisor.cpuid.v0 = “FALSE”
   
8. Now Boot and Install Windows 2008 R2 x64.
9. Once finished, open up Server Manager and click “Add Role”.
   
10. Select and install the Hyper-V option. At this point, you will know if your system is working correctly and passing the Intel EPT feature, because if it doesn’t, you won’t be able to go past this point.
    
11. You’ll also have to select the network adapter used for the virtual network.
    
12. Now install Hyper-V, which will need a reboot.
13. After it is completed, open Server Manager drill down to Hyper-V and connect to the local server.
    
14. Now create and install a virtual machine.
    

Once done, you should be able to use it as normal, albeit slow.

Nesting Hyper-VM running ESXi 5

Now, doing the same thing on ESXi 5 is a little trickier although some of the steps are the same.

1. 1. Before anything you need to place an entry in the /etc/vmware/config file found in the tech support mode on your ESXi 5. I enabled SSH through the security profile in the vSphere Client. Then used putty SSH into the ESXi system.
   2. From there I executed the following command which is needed to allow nested hypervisors :

      # echo 'vhv.allow = "TRUE" ' >> /etc/vmware/config

      Notice the use of single and double quotes in the command-line

   3. Now create a virtual machine using version 8 hardware, 4GB (or as much as you can spare), 2 x vCPUs, 2 or more vNICs and a 100GB virtual disk.
   4. Before booting up the VM and installing Hyper-V we need to add two lines the virtual machines config file .vmx
      You can try this through the vSphere Client in the settings of the virtual machine > Configuration Parameters, whereas I had better luck doing it from command-line
      
      
      To add them using command-line move back in SSH > change into the directory where you Hyper-V VM is installed

      # echo 'vhv.allow = "TRUE" ' >> /etc/vmware/config

      In my example the config file is called Hyper-V.vmx. Type the following commands:

      # echo 'monitor.virtual_exec = "hardware" ' >> Hyper-V.vmx

      # echo 'hypervisor.cpuid.v0 = "FALSE" ' >> Hyper-V.vmx

   5. Now back in the VM settings > Options > CPU/MMU Virtualization make sure you have the option to pass the Intel EPT feature.
      
   6. Now in the Options area > CPUID Mask click on Advanced
      
   7. Add the following CPU mask Level ECX: —- —- —- —- —- —- –H- —-
      
   8. Now Install Hyper-V or Windows 2008 R2 and enable the Hyper-V role.
   9. You are ready to roll.

Gotchas / Tips

Here are a few tips from Ricky to avoid any of the stopping points along the way:

1. 1. • On my system part way through install of Microsoft Hyper-V the OS requires a reboot. When you do this after Hyper-V has been installed it blue screens…DON’T PANIC because it doesn’t blue screen while actually using Hyper-V
      • With both my server and laptop I had no way of telling if my systems had Intel EPT it was a case of seeing if VMware passed Intel EPT without complaining and if Hyper-V spotted it. The give a way for me was when I’d tried VMware Workstation first and it worked straight off. I’ve read a million and 1 things about this subject, but my gut feeling is if you look for Nehalem or Core i7 and a motherboard that supports Intel VT. I think that is a safe bet. I read you need Intel VT –x2 which I don’t have (I don’t think) so that was misleading…If I can narrow the field of information I will update this post.
      • Remember nesting a hypervisor means it’s going to run very, very slow…however installing the nested hypervisor in a datastore that is on SSD disks helps big time.
      • The 2 blogs linked above presented 2 methods for creating the VM using version 4/7 hardware or version 8 hardware. I first went with version 8 hardware and no joy at all. All I got was a blank black screen. I actually tried a combination of the tweaks in both methods and that is what worked for me.
      • The port group that the nested Hyper-V machine resides on should be set to Promiscuous Mode: Accept
      • In one of the blogs there was a note suggesting making the entries manually in the config files (instead of using the GUI) was more stable. I found this too, so hence why I changed the config files in a shell session using putty.

Do you have any tips around installing Hyper-V virtual machines on ESXi 5 or VMware Workstation 8? Please share your comments below!

Read more:
How to create a Hyper-V VM template