OK…if you’ve followed along this far, you’re either 1) enjoying what you’re reading, 2) a glutton for punishmnet, or 3) really, really bored. Hopefully, it’s #1 and you’re here because you’ve read the first six posts in this series and you just can’t wait for me to add #7! If you’ve not read the first six posts, I recommend that you go back and do so now. The first six posts were:
- The Great vSwitch Debate – Part 1
In this post, I discussed vSwitch functions, Port Groups, VLAN tagging/trunking, valid communications paths, and some other basic vSwitch information.
- The Great vSwitch Debate – Part 2
In Part 2, I covered the vSwitch security features (Promiscuous Mode, MAC Address Change, and Forged Transmits) as well as network traffic shaping options.
- The Great vSwitch Debate – Part 3
Here I discussed the various load balancing options that are available in a VMware vSwitch.
- The Great vSwitch Debate – Part 4
In Part 4, I covered fault detection and the Cisco Discovery Protocol.
- The Great vSwitch Debate – Part 5
In Part 5, I talked about the various networks that you have to contend with in an ESX environment as well as an approach to help in deciding which networks to combine, if you have to.
- The Great vSwitch Debate – Part 6
I introduced the first host configuration. In this part, I talked about my recommendations for when you have eight pNICs and offered up a couple alternatives, including one for using an iSCSI initiator from within a VM.
In this Part 7, I’m going to discuss configurations for systems with two, four, and six pNICS. The same ground rules I established in Part 6 are going to apply here – for those who are skipping ahead or who have short memories, here they are:
- All networks will have at least two pNICs to provide fault tolerance. While ESX will work just fine without the fault tolerance features, I don’t consider that to be an option for a real live “production” environment. If you’re building out a personal “playground” or a lab environment, feel free to cut the number of pNICs in half, but don’t come crying to me if your one and only network connection fails and your ESX/i host and/or your VMs fall off the face of the network!
- Unless otherwise specified, all configuration options are set to their defaults values. This is in line with my “Don’t change a default value unless you have a good reason” philosophy (and it makes it easier to describe!)
- There is a requirement to support two separate “application zones”. These are logically separate networks that could be separated via VLANs or physical separation.
For each configuration, we’re going to use the same set of Port Groups – regardless of how many pNICs or vSwitches we create. If you remember from the end of Part 5, I made this recommendation for manageability and scaleability, so I’m going to stick to it. So, for each of our host configurations, we’ll have the Port Group configuration shown in Figure 1 and Figure 2. Figure 1 shows the port groups for environments that are using IP Storage (iSCSI and/or NFS):
Figure 1. Port Groups with IP Storage
And Figure 2 shows the port groups for those who are using local storage or fibre channel attached SAN storage:
Figure 2. Port Groups without IP Storage
In all cases, I strongly recommend the use of VLAN tagging if your network infrastructure supports it. Tagging will provide logical separation of traffic (physical is better than logical, logical is better than nothing!). It would be up to your networking team to assign the actual VLAN numbers, so coordination is critical! Now, on with the host configurations!
Configuration with Six pNICs
In Part 6 we talked about the configuration for a host with eight pNICs, so let’s look now at a host with six pNICs. With only six pNICs, you have to start making some choices – especially if you’re using IP Storage. You no longer have enough pNICs to provide each port group with its own fully redundant set of pNICs – you have to combine some networks. This is where you might want to go back and review the interaction matrices back in Part 5. Based on your own unique set of circumstances, you would decide which networks to combine together on a single vSwitch.
Figure 3 shows a fairly standard configuration for use with six pNICs when IP Storage is used.
Figure 3. Six pNICs with IP Storage
Notice that I combined the management and VMotion port groups (PG_Mgmt & PG_VMotion) onto a single vSwitch sharing a pair of pNICs. One alternative was to combine Management and IP Storage – I opted for VMotion because there is less traffic there, so there should be less interference between the two networks. The other option was to combine IP Storage and VMotion. I don’t normally like to do that because you then have two high-volume consumers using the same set of pNICs – but it is an option, and if you really need to isolate your management network, it could be the right option.
If you’re not using IP Storage, then six pNICs is pretty straight forward – three vSwitches to support the various functions (see Figure 4):
Figure 4. Six pNICs without IP Storage
Configuration with Four pNICs
With only four pNICs, it begins to get a little more interesting. We are resource constrained to the point where we have no choice other than to mix critical networking services, so the question is – which ones? I opted to mix them up as shown in Figure 5.
Figure 5. Four pNICs with IP Storage
Based on my belief that virtual machine traffic is toxic from a security standpoint, mixing of VM traffic with “infrastructure support” traffic would be a last resort. Taking IP Storage out of the picture yields Figure 6.
Figure 6. Four pNICs without IP Storage
No surprises there! With four pNICs, I don’t see very many alternatives, although I guess you could opt for a single vSwitch with active and standby pNICs, but that adds complexity and little value. All it would do is to provide you with a failover path in the event that you lost both of your pNICs associated with a given function. Since we’ve been satisfied with single pNIC failure tolerance so far, I see no reason to start worrying about it now!
Configuration with Two pNICs
The use of only two pNICs is not recommended. You don’t have the ability to separate virtual machine traffic from infrastructure support traffic and still retain redundancy. To me, that’s not a viable configuration – except, perhaps, in a lab environment. This configuration seems to come up most frequently with some older blade servers and in “build it yourself” systems. If you have any way to add another pair of network adapters into your host, please do so!
So, let’s assume that you’re stuck and you really, really have to use a system with only two pNICs…well, there aren’t many choices! Here’s what I would do (Figure 7):
Figure 7. Two pNICs with IP Storage
Yikes! That’s an ugly diagram! Due to the limited number of pNICs, I have elected to specify active and standby pNICs for each of the port groups. Normally, I don’t like to do this because it adds significantly to the overall complexity of the environment (just look at that diagram!). By overriding the default vSwitch behavior, I’m able to separate my VM traffic from my infrastructure support traffic except under failure conditions. Not an ideal arrangement, but hey, you do what you gotta do, right?
Once again, removing IP Storage only serves to simplify things (one less port group / network type to worry about!) – see Figure 8.
Figure 8. Two pNICs without IP Storage
Configuration with One pNIC
If I don’t recommend the use of a system with only two pNICs, you can probably guess what my opinion of a single pNIC is! There is no way to separate traffic (other than by VLAN tagging – but if you can’t afford at least two pNICs, chances are good your pSwitch doesn’t support 802.1Q!) and there is no way to provide any redundancy. Redundancy was one of the requirements I set out at the beginning of this exercise, and since I can’t satisfy my requirement with a single pNIC, I’m nog even going to draw the picture.
If you absolutely, postively have to run with only one pNIC, I would still encourage you to create the port groups. It will help you to manage the environment – and if you’re just learning the environment, it will give you the ability to exercise most of the features of the management interface.
I hope you’ve enjoyed this series on vSwitches and virtual networking in general. I do have one more article that I want to write still – one about naming standards. I hope to get that one out pretty quickly, but the weekend is looking pretty busy and next week is jam packed, so it may take a while. In the meantime, enjoy the articles and please, if you have comments or questions, leave me a note. I’ll try to get back to you as quickly as I can.