The Great vSwitch Debate – Part 2

Table of Contents

Part 2

This article is a continuation of the first in a series. See The Great vSwitch Debate – Part 1 for the beginning of the series.

In this Part Two of the series on vSwitch configurations, I want to address some of the advanced configuration options, but first – I need to go back and revisit the end of Part 1. There, I was discussing the routing of traffic between VMs on the same and different port groups on a single vSwitch. I presented the figure below (Figure 1) to describe what I was talking about.

Figure 1. Port Group Communications Paths

Figure 1. Port Group Communications Paths

I need to add a scenario that I feel is quite important and I’m somewhat embarrassed that I left it out the first time. Consider the case where you have a single vSwitch with two port groups defines, VLAN_200_Dev and VLAN_200_Test – two port groups with (obviously) different labels, yet they use the same VLAN number. This is the ONLY case where traffic from a VM on two separate port groups with defined VLAN numbers (other than 0 & 4095) will communicate directly within the vSwitch without requiring an intervening Layer 3 device. This is shown as Scenario 4 in Figure 2 below.

Figure 2. Inter-Port Group Routing

Figure 2. Inter-Port Group Routing

OK, now that I’ve gotten that off my chest, let’s move on to cover some new information. As you’ll recall, I introduced the concept of vSwitch security features back in Part 1. The vSwitch offers three security features as follow:

  • Security
    • Promiscuous mode enable/disable
    • MAC Address Change enable/disable
    • Forged Transmit enable/disable

Promiscuous Mode

Let’s look at each individually. First, there’s Promiscuous mode, which Wikipedia defines as “a configuration of a network card that makes the card pass all traffic it receives to the central processing unit rather than just packets addressed to it – a feature normally used for packet sniffing“. Well, that’s fine and dandy except we’re talking about a vSwitch rather than a network card. In actuality, the vSwitch promiscuous mode is much closer to Port Mirroring, which is defined as “used on a network switch to send a copy of all network packets seen on one switch port (or an entire VLAN) to a network monitoring connection on another switch port”. That’s pretty much what vSwitch promiscuous mode does – it enables a vNIC that has been placed into promiscuous mode to receive a copy of all network packets on the same vSwitch (or Port Group). If you configure a vNIC for promiscuous mode and the vSwitch/Port Group to which it is connected does not have Promiscuous mode enabled, then the vNIC will receive only traffic destined directly for it. To illustrate this concept, let’s start with Figure 3 as an example vSwitch configuration.

 Figure 3. Promiscuous Mode vSwitch Talking Slide

Figure 3. Promiscuous Mode vSwitch Talking Slide

In Figure 3, “VLAN A” can be any VLAN number from 1 to 4094 and “VLAN B” can be any other VLAN number from 1 to 4094 – for discussion purposes, it doesn’t really matter what the numbers are, only that they are valid and different from one another. To demonstrate who can talk to whom, let’s look at Table 1:

Table 1. Promiscuous Mode Port Group Connectivity

Table 1. Promiscuous Mode Port Group Connectivity

Basically, what Table 1 is saying is that, for a vSwitch configured in promiscuous mode, a promiscuous mode vNIC connected to PG_VGT can see all traffic on the vSwitch; however, if that same promiscuous mode vNIC connected were to be connect to PG_EST it would be able to see traffic only for other vNICs connected to PG_EST (or any other port group using EST Mode tagging); however, it cannot see traffic destined for any other VLAN.

The rest of the table is pretty simple, except for the PG_VGT column. The PG_VLA, PG_VLB, and PG_VLB1 rows show both a Yes and a No for PG_VGT which means, for example, a promiscuous mode vNIC connected to PG_VLA could see only traffic destined for VLAN A, which could include traffic destined for PG_VGT.

I hope that all made sense – it all seems overly convoluted on a cursory re-read. I’ll have to revisit before I publish to see if I can figure out a better way to describe it.

MAC Address Change

A MAC Address (http://en.wikipedia.org/wiki/MAC_address) is a pseudo-unique number assigned to most networking components. It is how a particular device is identified on the network. The MAC address functions at Layer Two in the network stack, whereas IP addresses function at Layer Three. In a VMware ESX environment, each vNIC is assigned a MAC address when the VM is initially powered on (unless you have manually configured a static MAC address in the .vmx file). This is the rough equivalent of the MAC address that comes burned in on pNICs. The description of the process ESX uses to generate MAC addresses is documented at http://kb.vmware.com/kb/219. (NOTE: The MAC-generation algorithm changed somewhere along the way. The new algorithm is documented here: https://www.vmware.com/pdf/vi3_35/esx_3/r35u2/vi3_35_25_u2_3_server_config.pdf Page 67 [thanks Tyler!]) One thing that’s important is that there is a limit of unique MAC addresses per ESX host. This could be an issue in a couple of situations:

  • In a VDI environment where a large number of VMs are created, powered on, and powered off
  • In an environment where VMs are started on a “launch pad” system and DRS is used to distribute workload throughout the cluster

In any case, what this security feature deals with is allowing the guest OS within the virtual machine to change its MAC address. For information on how to change your MAC address in a variety of operating systems, see http://en.wikibooks.org/wiki/Changing_Your_MAC_Address. So, why do you care if a guest OS changes its MAC address? One reason is that MAC spoofing is a key element of ARP Poisoning (http://en.wikipedia.org/wiki/ARP_poisoning), so having the ability to lock a VM’s MAC address to the value configured in the .vmx file provides a level of enhanced security that’s not available in a physical environment.

The MAC Address Changes setting affects traffic that is received by a VM. If this configuration item is set to Accept, the vSwitch will honor requests to change the MAC address to a value other than the one stored in the .vmx file. If it is set to Reject, then any request to change the MAC address will cause the vSwitch to disable the port to which the vNIC is connected and the vNIC will not receive any traffic until its MAC address is reset back to its original value. The guest OS is not notified that the attempt to change the MAC address was not honored.

One thing of note – if you plan to configure your VMs in a unicast Network Load Balancing (NLB) cluster, you must enable the MAC Address Changes feature or your NLB cluster won’t work. A multicast NLB cluster doesn’t need this feature enabled.

Forged Transmit

Forged transmits are similar to MAC address changes – except this setting deals with traffic that is originated from rather than received by the vNIC. Basically, if this setting is set to Accept, the vSwitch will allow the guest OS to modify its MAC address and successfully transmit frames with the “forged” address. If this configuration item is set to Reject, then the vSwitch will compare the source MAC address in all outgoing frames with the MAC address stored in the .vmx file. If they do not match, the frame is dropped with no notification to the guest OS.

As with MAC Address Changes, Forged Transmit is required to be set to Accept to support unicast NLB configurations.

Traffic Shaping Configuration

A feature that I have only ever used in a lab setting is Traffic Shaping. Traffic shaping allows you to control the amount of bandwidth that a vSwitch will transmit. There are three configuration parameters that control traffic shaping, and they are:

  • Average Bandwidth. This is the “guaranteed” data rate that you want to set for data transmission. You can view this as being similar to a committed information rate (CIR – http://www.2000trainers.com/cisco-ccna-11/ccna-frame-relay-cir/), if you’re familiar with frame relay networks.
  • Peak Bandwidth. This is the absolute ceiling data rate that the vSwitch will transmit data at. Once again referring to the frame relay model, this is analogous to the Committed Burst Information Rate (CBIR).
  • Burst Size. The final configuration parameter for Traffic Shaping is Burst Size. The burst size is specified as the number of bytes that can be sent each time the vSwitch “bursts” above the Average Bandwidth.

For a detailed description of when and how you might want to use Traffic Shaping, take a look at this article (http://www.petri.co.il/traffic_shaping_with_vmware_esx_server.htm) by Jason Boche – a great example use for this feature.

Miscellaneous Security Features

In addition to the configurable security features we’ve discussed above, there are a couple things that are unique to a vSwitch that directly impact its security. These benefits are:

  • A vSwitch knows the MAC address of each vNIC that is connected to it. It relies on a notification mechanism for MAC updates and doesn’t have to rely on learning the MAC address from traffic that traverses the vSwitch.
  • Traffic received on one vmnic is never forwarded to another vmnic. This means that a vSwitch will never create a network loop and thus the vSwitch does not need to (and, indeed, does not) participate in the Spanning Tree Protocol (STP).
  • A vSwitch does not do IGMP snooping (http://en.wikipedia.org/wiki/IGMP_snooping) which is a Layer Three function; however, since the vSwitch knows which vNICs want to receive multicast traffic, multicast traffic is not flooded to all connected vNICs.
  • The vSwitch does not trust user data within the network frame as it passes through the vSwitch:
    • The vSwitch makes a private copy of any network frame data used in forwarding and filtering.
    • The vSwitch carries VLAN data outside the frame as it passes through the virtual switch. Filtering is a simple integer comparison.
  • vSwitches have no support for dynamic trunking (802.3ad)
  • vSwitches do not implement support for the “native VLAN” [1]

Summary

OK, we’ve talked about some of the security features that are implemented in the vSwitch. that’s going to wrap it up for Part 2 of my series on vSwitches within ESX. Next time, we’ll talk about NIC Teaming policies, Load Balancing policies, and Network Path Failure Detection methods. Until then, enjoy – and keep it virtual!

Back to The Great vSwitch Debate – Part 1 — Forward to The Great vSwitch Debate – Part 3

References

The Great vSwitch Debate – Part 1

MAC Address (http://en.wikipedia.org/wiki/MAC_address)

Setting a static MAC address for a virtual NIC (VMware KB Article 219) (http://kb.vmware.com/kb/219)

 (https://www.vmware.com/pdf/vi3_35/esx_3/r35u2/vi3_35_25_u2_3_server_config.pdf Page 67)

Changing your MAC address (http://en.wikibooks.org/wiki/Changing_Your_MAC_Address)

ARP Poisoning (http://en.wikipedia.org/wiki/ARP_poisoning)

Committed Information Rate (CIR – http://www.2000trainers.com/cisco-ccna-11/ccna-frame-relay-cir/)

Traffic Shaping article by Jason Boche (http://www.petri.co.il/traffic_shaping_with_vmware_esx_server.htm)

IGMP snooping (http://en.wikipedia.org/wiki/IGMP_snooping)

[1] Virtual LAN Security Best Practices: (http://www.cisco.com/warp/public/cc/pd/si/casi/ca6000/prodlit/vlnwp_wp.pdf)

About these ads

Tags: , ,

About Ken Cline

vExpert 2009

26 responses to “The Great vSwitch Debate – Part 2”

  1. Vikas Deolaliker says :

    What is the debate? It looks like you are describing a configuration.

    • Ken Cline says :

      Yep…you’re right, so far. I do plan to get into the debate beginning with Part 5; however, I felt it was important to discuss the underlying technology so that there was a solid common foundation to support my debate position. For those who have been waiting on the debate, it’s coming (or at least I hope it is!). I will be putting forth my recommendations and looking for all of you to pick them apart and make me defend (and possibly change) my position. That’s where the debate comes in…

      KLC

  2. Tyler says :

    I believe that VMware KB link you provided for MAC generation is out of date. Here is what the 3.5u2 docs have to say about it:

    http://pubs.vmware.com/vi35u2/server_config/sc_adv_netwk.6.12.html

    MACs are now semi-permanent and no longer based on things like service console IP.

  3. Mike Gough says :

    Your link above:
    CCNA 3 – Module 7 Exam Answers Version 3.1

    appears to me to be a link to someone publishing copyright violation of Cisco exam details.
    I request and recommend you disassociate from and remove you link here.
    Best regards, Mike Gough

    • Ken Cline says :

      Mike – I don’t see the specific link to which you are referring. Could you please be more specific? The only thing that I see that has anything to do with CCNA is my reference to CIR, which is to a site where copyright is claimed by 2000Trainers.com – and a quick search of that site doesn’t reveal any references to the title you’re citing.

      Can you clarify, please?

      Thanks,
      KLC

  4. Allen says :

    Ken,

    Are you missing the PG_VGT port group in Figure 3? Table 1 (and the two paragraphs that follow) references this port group, but I don’t see it anywhere in Figure 3. I only mention it because I was a bit confused when I read through this initially, though I get the idea still.

    Thanks,
    Allen

    • Ken Cline says :

      Hey Allen,

      Looking for a job as an editor? The pay sucks (you’ll make just as much as I do ;) ) — seriously, I do appreciate your feedback (at least I know you’re reading the articles!)

      KLC

  5. Allen says :

    Ha, not really. But if you keep feeding us the good content, I’ll keep the feedback coming. :-)

  6. Jason says :

    Wow, great damn series Ken. We have been looking for a definitive explanation of some of this stuff for some time. Very helpful.

    I did have a couple of questions though, and perhaps I missed it and just need clarification.

    If two port groups reside on the same vSwitch but neither has a VLAN defined, how do they communicate? Does the traffic leave the host? Thanks in advance.

    • Ken Cline says :

      If two port groups are defined on the same vSwitch and they both have the same VLAN number (even if that VLAN number is “not defined”), then traffic between the two port groups will be routed locally within the vSwitch and not exit via a vmnic (pNIC).

      Hope that clarifies things for you – and thanks for stopping by, and especially thanks for asking a question!

      KLC

  7. Pippin Wallace says :

    Ken, first off thank you very much for the terrific articles. I do have a question regarding two port groups sharing the same VLAN number.
    I am guessing the vSwitch sees all VM’s in both port groups as being on the same VLAN and thus can switch/”route” per the vSwitch’s ARP table.
    It would seem that this could produce some odd/undesirable behavior and I would like to know what warnings you would advise with this configuration?

    • Ken Cline says :

      Hi Pippin,

      You are correct that if there are two port groups with the same VLAN number on the same vSwitch, the vSwitch can deliver locally. I don’t see this as a real problem – as long as the administrators are aware of the fact that the two port groups belong to the same layer two domain and traffic will flow between them without layer three intervention.

      Using multiple port groups with the same VLAN identifier can provide a nice administrative separation between the VMs – but it is strictly administrative. I have seen users implement this tactic when the know that they WILL be implementing VLANs at some point, but just haven’t gotten the network team on-board yet.

      HTH,
      KLC

  8. Jagadish says :

    Ken, You have taken great effort to post these contents which is available at a single place and best of it is very very useful especially the security features, I have a question for you.

    Provided the pSwitch is configured with multiple VLAN’s and Tagged to a single port on the server how to assign all of these VLANs to same vSwitch?

  9. Ray Welch says :

    I have to agree with that one guys comment. This doesn’t seem to be any kind of debate, nor does it seem as though this was approached from the point-of-view of security.

    This sounds more like a glowing endorsement of the use of vSwitch technology from a person working for a manufacturer of such technology, such as VMware. And, of course, the manufacturers have a vested interest in promoting their technology and telling people that the technology carries no inherent security risks with it.

    I would have like to have read something that was more of a two person debate; one person being a pro-vSwitch person (as you have here) and the other being a pro-security person with entensive knowledge and experience in trying to protect virtualized environments, especially those using vSwitches.

    I have an organization that wants to, for all practical purposes, eliminate the use of their corporate firewalls by using a vSwitch to bridge across disparate security zones which, currently, are kept separated by the organizations firewalls. I’ve been the lone voice of dissent but they are going to do it anyway. All I know is that I don’t want to be here when they do it because I’m tired of being the organization’s “whipping boy” every time they have a breach (as there have been so many that I’ve lost count). They refuse to do anything to improve security; they prefer to run their systems wide open, like the wild, wild west. And now, in my opinion, they want to weaken security substantially further.

    I would have liked to see a whole lot more on the security ramifications of the use of vSwitching.

    • Ken Cline says :

      Hey Ray,

      You’re right – to a point. I did not approach this with security as the key motivator, and I’m not a real security person. I threw in some discussion on security because it’s always something that has to be considered. I had hoped, when I started this, that there would be much more interactivity during the development phase (thus the “debate” part of the title…). And, just so you know, when I wrote this, I was not working at VMware (no longer there, either), so it was not a “person working for a manufacturer of such technology” writing this.

      If you’d like to get more of a security focus on things, pay Edward Haletky a visit over at http://www.astroarch.com/ – he lives & breathes security and has lots of good info that you may find useful.

      Thanks for stopping by!
      KLC

  10. Ken Cline says :

    Thanks, Anton! I appreciate your taking the effort to translate this to Russian. It will enable me to reach a much broader audience.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 34 other followers

%d bloggers like this: