What is Hyper-V Switch Embedded Teaming?
With Windows Server 2016, Microsoft brings us a new NIC (network interface card) teaming solution called Switch Embedded Teaming (SET). It is only available on servers running Windows Server 2016 with the Hyper-V role installed and is not backwards compatible.
SET allows us to group up to eight physical NICs to create one or more vNICs. Microsoft’s reasoning behind this is SET is best used for NICs 10 GB or bigger. SET only supports switch independent mode to make setup easy for us to configure. You can either plug all physical NICs into one physical switch or multiple physical switches if they are in the same subnet. Connecting the SET Team to multiple physical switches can give you extra redundancy.
Due to SET’s integration with the Hyper-V Virtual Switch, you are unable to use SET Teaming inside a virtual machine (VM). SET Teaming does not present the team interface to the VM, so you can utilize LBFO (load-balancing failover) Teaming inside a VM by adding additional vNICs to the VM.
Which NICS can I use?
If your NICS have passed the Windows Hardware Qualification and Logo (WHQL) test, then you can use them to create your SET Team. SET does require all NICs that are part of a SET Team to be identical. This means that they are from the same manufacturer, are the same model and have the same firmware and driver. You can only have up to eight NICs in one SET Team.
One good thing about SET teams is that you can configure Remote Direct Memory Access (RDMA) per vNIC. This means you can create multiple vNICS for your SET Team. You can configure a vNIC for storage and enable RDMA on it. You could then create another vNIC for management traffic and not enable RDMA. This is good for hyperconverged solutions.
SET compatibility with Windows Server
SET is compatible with the following networking technologies found in Windows Server 2016.
- Datacentre bridging (DCB)
- Hyper-V Network Virtualization — NV-GRE and VxLAN are both supported in Windows Server 2016.
- Receive-side Checksum offloads (IPv4, IPv6, TCP) — These are only supported if at least one of the SET Team members support them.
- Remote Direct Memory Access (RDMA)
- Single root I/O virtualization (SR-IOV)
- Transmit-side Checksum offloads (IPv4, IPv6, TCP) — These are only supported if all the SET team members support them.
- Virtual Machine Queues (VMQ)
- Virtual Receive Side Scaling (RSS)
SET is not compatible with the following networking technologies in Windows Server 2016.
- 1X authentication
- IPsec Task Offload (IPsecTO)
- QoS in host or native operating systems
- Receive side coalescing (RSC)
- Receive side scaling (RSS)
- TCP Chimney Offload
- Virtual Machine QoS (VM-QoS)
SET modes and settings
When you first create a SET Team, you are unable to configure a team name and you are unable to set NICS to standby like you can with the NIC Teaming in Windows Server 2012 r2. In fact, with SET Teams all NICS are always active.
Another difference between SET Teams and LBFO (load balanced fail over) teams is that with NIC teaming you had a choice of three different teaming modes. SET only gives you one choice: Switch Independent. Switch Independent mode means the physical switch or switches are unaware of the SET Team and do not determine how to distribute the network traffic.
There are two team properties that need to be configured when you create a SET Team. They are:
- Member adapters
- Load balancing mode
Member adapters
As stated earlier in this post, when you are creating a SET Team, you must use up to eight identical NICS. You can create your SET Switch with just one NIC to start, but any new members must be identical.
Load Balancing mode
There are two Load Balancing distribution modes for SET Teams. They are:
- Hyper-V Port
- Dynamic
Hyper-V port
When using Hyper-V port mode for SET Teams, VMs are connected to a port on the Hyper-V Switch. This is just like a physical server would be connected to a port on a physical switch. The Hyper-V Virtual Switch port and the VM’s MAC address are used to divide the network traffic between the SET Team members.
If you use Switch Embedded Teaming with Packet Direct, you must use the Switch Independent Teaming mode and the Hyper-V port load balancing mode.
With this teaming mode, any adjacent switches will always see a VM’s MAC address on a given port. This allows the switch to distribute the ingress load (the traffic coming into the Hyper-V host from the switch) to the port where the MAC address is located. Hyper-V port mode is very useful when you utilize Virtual Machine Queues (VMQs) because a queue can be placed on the NIC where the traffic is expected to arrive.
Hyper-V port mode is not the best if you are only hosting a few VMs due to it not being granular enough to achieve a well-balanced distribution. This mode also limits each VM to the bandwidth that’s available on a single NIC.
Dynamic
When using the Dynamic port mode for SET Teams, all outbound loads are distributed using a hash of the TCP Ports and IP addresses. Dynamic Port mode also rebalances loads in real time to allow outbound flow to move between each SET Team member.
For inbound loads, all traffic is distributed in the same way as the Hyper-V port mode. Outbound loads use flowlets to dynamically balance. A flowlet is the portion of a TCP flow between two naturally occurring breaks. When the dynamic port mode detects a flowlet boundary, it will automatically rebalance the flow to another member of the SET Team, if appropriate. It is known that in some uncommon circumstances dynamic port mode may rebalance flows that do not contain any flowlets. Due to this, the affinity between a Team member and the TCP flow can change as the traffic is balanced.
Creating and managing a SET Team
Although it is recommended that you use Microsoft Virtual Machine Manager (VMM) to configure SET Teams, not every company has this. That’s where PowerShell comes in.
In the following section, we will be go through the process of creating a SET Team, adding and removing NICs to a SET team, changing the Load Balancing Mode and removing a SET team.
Create a SET Team
You can only create a SET Team at the same NIC that is used for a Hyper-V Virtual Switch. To create a SET Team with one NIC use the following PowerShell code in an elevated PowerShell window:
New-VMSwitch -Name "SET Team" -NetAdapterName "NIC 1" -EnableEmbeddedTeaming $true
You can change SET Team to any name you need, as well as adapter name NIC 1, to the name of your NIC. This can be done by using the following PowerShell command:
Get-NetAdapter
You should have a list of all your adapters. The first column is Name. This is the name of your network adapter that you need to use for the -NetAdapterName.
To create a SET Team with more than one NIC, you can use the following PowerShell command in an elevated PowerShell window:
New-VMSwitch -Name "SET Team" -NetAdapterName "NIC 1","NIC 2"
Notice when adding more than one NIC you don’t need to add the -EnableEmbeddedTeaming because it is assumed by Windows PowerShell when the argument to -NetAdapterName is an array of NICs.
Adding and removing SET Team members
The only way to add or remove SET Team members is to use the SET-VMSwitchTeam command. You basically name the SET switch and which adapters you want to be in it. Let’s say you have created your SET switch with NIC 1 and NIC 2, and now you want to remove NIC 2 and add NIC 3:
Set-VMSwitchTeam -Name SET Team -NetAdapterName "NIC 1","NIC 3"
If you now need to remove NIC 1, you can use the following command:
Set-VMSwitchTeam -Name "SET Team" -NetAdapterName "NIC 3"
To add NIC 2 back in to the SET Team, you can use the following command:
Set-VMSwitchTeam -Name "SET Team" -NetAdapterName "NIC 3","NIC 2"
Change the Load Balancing Mode
To change the Load Balancing Mode between the two mentioned above, you can use the following PowerShell command (just Change Dynamic to HyperVPort):
Set-VMSwitchTeam -Name "SET Team" -LoadBalancingAlgorithm Dynamic
To view which Load Balancing Mode you currently have:
Get-VMSwitchTeam -Name "SET Team" | FL
Remove a SET Team
The only way to remove a SET Team is by removing the Hyper-V Virtual switch completely:
Remove-VMSwitch "SET Team"
Now you have learned how to create, manage and remove a SET switch. Hopefully, you will find this post beneficial and helpful. If you have any questions, leave a comment.