Comments on: The Great vSwitch Debate – Part 3

By: Mike Williams

Mike Williams — Thu, 10 Dec 2009 22:12:36 +0000

I think your point about needing 802.3ad (etherchanneling) the pNICs is a good one when using IP Hashing. An important implication in this is that this virtually eliminates the possibility to use the IP Hashing when running ESX on blade servers, as *no* blade server switching technology that I’m aware of would allow an etherchannel across 2 switch modules. If you were using passthrough modules to allow each blade access to external switches directly, then you could connect those to some upstream stacked 3750s, 6500s running VSS or Nexus 7000s (and soon 5000s) using VPCs.

Just like the hashing on the network side using etherchannels, it’s important (as you said) to choose the hashing that’s best for what that box does. It’s if a server that dumps files to one other box, then IP hashing isnt buying anything. If it’s an application server that handles thousands of connections from a wide distribution of client IP addresses, IP hashing is probably the best option.

I’m surprised ESX hasn’t taken a tip from Cisco and added another level which is source/dest IP *AND* TCP port hashing. This allows even 2 machines that talk alot between them to have multiple conversations balanced properly over multiple links.

By: Ken Cline

Ken Cline — Fri, 04 Dec 2009 03:21:04 +0000

Hi Ryan,

The MAC address will not “float” between the various ports in the port group. Once a vNIC has established an affiliation with a pNIC, ALL outbound traffic for that vNIC will traverse the same pNIC (the only exception is IP Hash, where the affiliation is tied to the IP conversation rather than the vNIC). The only time that the vNIC’s traffic will change pNICs is in the event of a path failure – in which case the vSwitch will send a gratuitous ARP packet to the pSwitch to notify it of the change in interface.

Thanks!
KLC

By: Ryan Gallier

Ryan Gallier — Thu, 03 Dec 2009 13:57:49 +0000

How does the switch then deal with the mac-address going down different physical ports during the load balancing?

By: Ken Cline

Ken Cline — Thu, 03 Dec 2009 03:10:56 +0000

No, you do not configure 802.3ad on your pSwitch if you are using the default vSwitch Port Based LB. You don’t need to make any special configuration changes on your pSwitch – that’s one of the great things about the default policy! When you configure .3ad, your pSwitch is going to expect all ports in the port group to act as one single port. This configuration used to work back in the ESX 2.x days (and it may still work now…), but it is not recommended and is not supported (by either Cisco or VMware).

HTH,
KLC

By: Ryan Gallier

Ryan Gallier — Wed, 02 Dec 2009 21:58:02 +0000

I still confused on the pSwitch configuraiton. If i am using the default vSwitch Port Based Load Balancing, Do i still configure an 802.3ad (static) port-group on my pSwitch? Why or why not? Keep in mind I’m running this on Cisco 3750 switches which can do cross switch 802.3ad.

By: Ken Cline

Ken Cline — Mon, 30 Nov 2009 23:19:56 +0000

Without knowing more about the details of your environment, I would probably stick to the default vSwitch Port ID based load balancing algorithm. I would even consider creating multiple port groups and having the SC & VMotion share one port group with pNIC1 as primary and pNIC2 as standby and have FT on the other PG with the primary/standby reversed.

FT & VMotion are both (potentially) heavy bandwidth users and I would like to keep them separated as much as possible. IP Hash will not give me the guaranteed separation I can get with the approach mentioned above.

Something to consider…

KLC

By: deercutter

deercutter — Fri, 20 Nov 2009 12:53:11 +0000

Ken – We will be using IP Hash Load Balancing on our two pNic Etherchannel which wil be carrying our “service console”, “vmotion”, and “fault tolerance logging” ports. would you agree this being the preferred way?

By: NFS and IP-HASH loadbalancing « Frank Denneman

NFS and IP-HASH loadbalancing « Frank Denneman — Fri, 13 Nov 2009 14:42:58 +0000

[...] It will not do perfect load-balancing out of the box. Remembering Ken Cline’s (@clinek) excellent article about vSwitches I knew you must dive in to the algorithm used by IP-Hash load balancing and pick [...]

By: Ami

Ami — Sun, 30 Aug 2009 20:06:16 +0000

Hi Ken,
a)Shapo on the lesson..bravo!

b)You’ve mentioned in fig. 1 that no policy applied “here”(The vmkernel is there)….but then in the end you recomend ‘ I do sometimes see a need for IP Hash is with IP based storage’..what have I missed..?!

c)Where on Earth have you got all that XOR,LSB,,,algorithm information from..? (It’s almost as if you’ve written them by yourself) A M A Z I N G!

By: Ken Cline

Ken Cline — Tue, 04 Aug 2009 19:19:32 +0000

The mapping between vNIC and vSwitch(Port Group) is static for the current vNIC invocation (i.e. from the time the vNIC is powered on until it is powered off). Once a vNIC is powered off, it leaves an “available” vSwitch port, so the next time a vNIC is powered on, it is assigned to the first available vSwitch port.

I wouldn’t worry too much about it – it’s one of those academic exercises that, in the end, doesn’t really make much difference. In most cases, your host will be up for a long time and VMs will be powered on and off many, many times which will cycle the vNICs across the vSwitch ports so many times that it almost becomes random.

Thanks for stopping by – and sorry it took so long for me to reply!
KLC