Hardware VTEPs supported by NSX

I had a discussion with a client about this a little while ago and it took me some time to find the link, but here’s VMware’s current list of supported Hardware VXLAN Gateways.

I’ve also been told there are some Cisco devices supported that aren’t on the list:

With OS version 7.0(3)I4(2):

The 9300-EX should join that list soon.  I have no idea if or when supported Cisco devices will be made public.

There is an HoL (HOL-1703-SDC-1) that has a module on configuring NSX to work with an Arista (note that its a simulation, not a “real” lab) if you want some almost-hands-on experience.

If you have some physical devices you want to participate on a VXLAN network (maybe L2 connectivity is required or you just can’t change IPs), hardware VTEPs are one possible solution.


Posted in Network, NSX, Virtualization, VMware | Tagged , , | 2 Comments

VMUG Virtual Event 6.0

VMUG is once again hosting an online virtual conference complete with Hands-On-Labs, keynotes, vendors and giveways.


Such luminaries as Chris Wolf, Frank Denneman, Chris McCain and Kyle Ruddy from VMware, Rawlinson Riviers from Cohesity and many others will be on hand for presentations and Q&A.

There’s a lot of good information collected in one place including advanced NSX and vSAN troubleshooting webinars, vCenter performance guides and much more.

The price is right too.

Posted in Cloud, NSX, Virtualization, VMware | Leave a comment

Powershell report on NSX IP Pools

While troubleshooting and NSX install I began wondering about the NSX IP pools and what was being used in them.  I still need to look up the allocated IPs and report on the objects using them (name, type) but for now the (very basic) report spits back IP pool name, the configured ranges, the total IPs in the ranges, the number of IPs used and which specific IPs are used.

Pool name: VTEP Pool
Configured range(s): -
Total IPs: 101
Number of IPs used: 4
IPs used:,,,

Note when you go to configure pools, if you want to allocate disjointed ranges you need to separate them with commas – this is no longer shows in the UI with 6.3. Also, to add single IPs just make a range out of them like:,,

The script leverages a shorter version of the NSXAPI function I stole from Chris Wahl a while back.

I’ve only tested with Powershell 5.0 – the original NSXAPI function created a new type which errors in PS5:

add-type : Cannot add type. The type name ‘TrustAllCertsPolicy’ already exists.

so I chopped that part of the code out along with most comments.

I’ll update this post when I tweak the script to report fully on the IPs allocated.

function NSXAPI {
Process {
$Username = "admin"
$Password = "vmware"
$NSXManager = "nsxmanager.corp.local"

[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy

$auth = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($Username + ":" + $Password))
$head = @{"Authorization"="Basic $auth"}

$r = Invoke-WebRequest -Uri $Request -Headers $head -ContentType "application/xml" -ErrorAction:Stop
[xml]$rxml = $r.Content
} # End of process
} # End of function

foreach ($pool in $pools) {
$name = $pool.name
$universal = $pool.isUniversal
$total = $pool.totaladdresscount
$used = $pool.usedaddresscount
if (($pool.ipranges.iprangedto).count) {
$range = ""
foreach ($r in $pool.ipranges.iprangedto) {
if ($range -eq "") {
$range = $range + "$($r.startaddress) - $($r.endaddress)"
} else {
$range = $range + ", $($r.startaddress) - $($r.endaddress)"
} Else {
$range = "$($pool.ipranges.iprangedto.startaddress) - $($pool.ipranges.iprangedto.endaddress)"
$string= "https://nsxmanager.corp.local/api/2.0/services/ipam/pools/$($pool.objectid)/ipaddresses"
$allocated = (NSXAPI $string).allocatedIpAddresses.allocatedIpAddress
$ips = ""
foreach ($a in $allocated) {
if ($ips) {
$ips= $ips + ", $($a.ipaddress)"
} else {
$ips= "$($a.ipaddress)"

"Pool name: $name"
"Configured range(s): $range"
"Total IPs: $total"
"Number of IPs used: $used"
"IPs used: $ips"
" "
" "

Posted in Network, NSX, PowerShell, Scripting, Virtualization, VMware | Tagged , | Leave a comment

NSX distributed logical router appliance Part 2

In a previous post I talked about UDLR/DLR differences, why you might need to deploy an appliance with your (U)DLR and how high availability works for the optional appliance.

This post will cover more on IP addressing for the appliance and configuration changes after a brief rant on naming.

What’s in a name

Would a DLR that wasn’t referred to as an Edge smell as sweet? – W. Shakespeare

So in the 6.3 documentation, the DLR falls under Routing / Add Logical (Distributed) Router

And there it mentions An NSX Edge appliance provides dynamic routing ability if needed.

In the GUI you find the (U)DLR under NSX Edges even though it’s not on “the edge” and deploying a VM based off an Edge appliance isn’t required for a distributed logical router.

In the KB article referenced in the docs the appliance is called a DLR Control VM which the Design Guide also call it.  Most VMware presentations refer to it as a “Control” appliance.

At least it’s explicitly mentioned in the docs now, it was kind of glossed over in the install section previously (6.0/6.1).

Speaking of names, it’s still referenced repeatedly as a vShield Edge appliance.

vApp details

First time logging in – and second.

Could we please pick one name and stick with it?


As mentioned in my last post each DLR VM created will show all configured IPs for the DLR.

However, the only one that normally sees traffic is the HA heartbeat APIPA address, and in the event of a failover, the HA Interface IP will be ARPed.

If you configure OSPF or BGP, you will be asked for a Protocol Address and a Forwarding Address.

BGP config also sets up the Forwarding and Protocol Addresses.

The Forwarding address is the “next hop” for all networks managed by the DLR.  The networks and “next hop” that will be pushed out to the OSPF neighbors.

OSPF Forwarding Address is the Transit IP for the DLR.

The Protocol Address will be assigned to the DLR VM and that is how the DLR VM will exchange routing information with the OSPF neighbors.

DLR VM with OSPF Protocol Address replacing Transit Uplink

Looking at the interfaces on the DLR VM, all internal links are added to a VDR interface and used to generate the dynamic routing updates.

The DLR HA interface gets added it’s own interface (assuming its not on the same network as another uplink)

While uplink DLR interfaces (whether or not a Protocol address is created) each get their own interface.

Uplink vs Internal

Dynamic routing protocol (BGP and OSPF) neighbors are supported only on uplink interfaces.  And of course dynamic routing protocols are the only reason to depoy a DLR VM.

Firewall rules are applicable only on uplink interfaces of the DLR VM and are limited to control and management traffic that is destined to the DLR VM.


The VM appears to be configured via Tools.  NSX Manager sends an update to the host holding the VM, which pushes the update via VMware Tools.

While the Design Guide states on pg 53:

The controller leverages the control plane with the ESXi hosts to push the new DLR configuration, including LIFs and their associated IP and vMAC addresses.

I don’t believe that statement is correct.

Because without a Controller:

Adding a new OSPF area:

The new changes show up on the DLR VM:

I believe updates are pushed via the “message bus” to the DLR VIB and DLR, which would explain why changes can be sent with no Controllers and no network connectivity to the DLR VM.

Speaking of controllers, in the NSX Troubleshooting guide it says Controller Cluster allocates one of the Controller nodes to be the master for this DLR instance. and references the Controller cluster receiving route tables and updates from the DLR VM and sending that information on to the hosts.

This does appear to be the case, as deleting my controllers stopped route updates from being delivered to the hosts.


Posted in Network, NSX, Virtualization, VMware | Tagged | Leave a comment

NSX distributed logical router appliance Part 1

When you create a Universal or “regular” Distributed Logical Router in NSX you have the option of deploying an appliance along with it.

This post will cover why to deploy an appliance along with a DLR, configuring the appliance for high availability and how the DLR high availability feature works.

Deploy if you need dynamic routing on your DLR
Enable HA and add an IP address on a dedicated logical switch
Disable vSphere HA for the DLR appliance

Pick Universal if you'll eventually have cross-vcenter and will only connect to logical switches (no dPGs)

Continue reading

Posted in Network, NSX, Virtualization, VMware | Tagged | Leave a comment

Moving an NSX DLR control VM (datastore or host)

If you have deployed an appliance along with your NSX distributed logical router you might need to move it around in the vSphere environment.

However, using the normal vSphere migration method causes more than a few issues.

Oh, that doesn’t look good!

Not to worry, because NSX makes it pretty simple to relocate them.

First go to Networking & Security / NSX Edges and double click on the DLR in question.

Under configuration, select the appliance instance to move (the “Name” field will match the vSphere name) and click the pencil/edit button.

Then choose the new values.

The table will be updated with the changes …

and if the datastore was changed you can go to vSphere and see the new appliance deployed and the old one removed.

when the deployed VM is only ~1GB total it is pretty quick to replace!

Note that the same method works to move all NSX components other than NSX Manager. Note also that the same recreate/delete is used for all moves even when just changing folders!

Speaking of folders, if an NSX object is moved into a folder,  it can’t later be moved back to “no folder” using the web client.

Posted in Network, NSX, Virtualization, VMware | Tagged | Leave a comment

Can’t create Universal Transport Zone or UDLR with NSX 6.2 / 6.3

VMware NSX 6.2 introduced Cross-vCenter including Universal objects like Universal Transport Zone, Universal Logical (Distributed) Router (UDLR), Universal Logical Switch.

So what happens if you’re on 6.2+ and you can’t create those objects?

Can’t create Universal Transport Zone.

Can’t create Universal Logical rRouter.

Can’t create Universal Logical Switch.

If you find yourself trying to create these objects but the option isn’t available but you’re sure you’re on 6.2 or higher, set your primary NSX manager to “Primary”

Assign Primary Role to the main NSX Manager.

This will enable the creation of universal objects and enable more options for NSX managers including “Perform Universal Synchronization”.

And now we can create the Universal Transport Zone

and UDLR

Creating a UDLR.

Posted in Network, NSX, Virtualization, VMware | Tagged , , | Leave a comment