When I moved the VMs connected to the Broadcom NICs to the Intel NICs they behaved normally and users can login much faster. I then tried to move the VMs on the Intel Cards to Broadcom and we got the poor performance. We concluded it must be something related to the Broadcom Network card.
Troubleshooting steps:
- Updating the Broadcom NICs to the latest driver. Still No change.
- We compared the options/Properties of Intel Card vs. Broadcom card and it was noticed that Transmit buffers in Broadcom was 200 vs 500 in the Intel cards, I tried changing it to match the Intel settings. Still No change.
Resolution:
After several investigations, the problem turned out to be in the VMQ setting which was enabled on both Broadcom and Intel. When it was disabled on the Broadcom card, the VM worked perfectly. Looks like the Broadcom card is not compatible with the VMQ feature.
Reference Links:
- High Ping Latency in Hyper-V Virtual Machines: http://www.flexecom.com/high-ping-latency-in-hyper-v-virtual-machines/
- Slow Network Access Within Virtual Machines - Broadcom and Hyper-V: http://fundamentallygeek.blogspot.com/2012/11/slow-network-access-within-virtual.html
- VMQ feature http://technet.microsoft.com/en-us/library/gg162704(v=ws.10).aspx