NSX: Upgrading to 6.4.0 from 6.3.x with a Control VM HA IP configured

tl/dr: If you upgrade from NSX 6.3.x to 6.4.0 make sure you remove the Control VM HA IP if it is set. Note that I have checked this with DLR and UDLRs.

So I wrote about this the other day and decided to see if it would blow things up during an upgrade to 6.4.0.

Official NSX Upgrade Guide

With my lab environment running 6.3.2 and the Control VM configured for HA with an IP address set:

I uploaded the bundle using the NSX Manager (note that with 6.4 you do it from the vSphere client!)

And started uploading

Enable SSH but don’t join CEiP for a temp environment

And wait …

[Note that I just did 6.3.1 to 6.3.2 to 6.4 and going to 6.4 is taking *a lot* longer for NSX manager.]

oops.  I see why it is taking long…

You should not be reading this error message

I hit enter and it came up, so I’ve got that going for me.  Which is nice.

Ok, so did the Edge blow up with the IP set like it does in 6.4 if you set the IP and then enable HA?


Ok, great so upgrading to 6.4 removed the HA IP?


Oops.  And if you disable HA…

Its still there.

Re-enable HA…

Note that repeated tests show that this might not error unless you open the window to change the IP address – even if you don’t change anything!

So, again – remove the HA IP configuration if you set it before upgrading!  and then don’t set it in HA during the install or while playing with HA.

Posted in Network, NSX, Virtualization, VMware | Tagged , , | Leave a comment

NSX 6.4 DLR HA changes

tl/dr: Enable HA during DLR deployment, don’t specify an HA IP address (if prompted),  use a unique logical switch for HA.

Edits: Some info from VMware below. Also, if you are upgrading from 6.3 I would remove the HA IP address first!

I wrote a few posts last year on the DLR components of NSX – specifically the Control VM that handles dynamic routing partnerships.

There are a few interesting changes to the Control VM for 6.4.0 I wanted to get down on paper cause they can result in a call to support if not handled right.

Enable HA and set an IP

Both of the issues concern the initial config wizard for the DLR. You are prompted on the first page to enable HA.

Make sure you enable HA here! It is very possible to not be able to enable it later w/o a call to support.

Note that whether or not you enable it, on the fourth screen you’ll need to set an HA interface connection.

Also on that fourth page note that you might be able to set an IP address (see my old posts on what happened with 6.3 when you set it).  If you don’t enable HA on the first screen you will be able to set an HA IP.  If you enable HA, you might be able to set an IP

If you see an entry for HA IP Do not set an IP address here. This isn’t that bad, as even tho you can add one, it won’t actually retain the IP address you set here.

Look ma! No IP!

The problem comes when you didn’t enable HA during install and go to add it later.  Or, disable HA.  Because when HA is disabled you can see – and add – an IP address under HA Interface Config:

Compare that to a DLR with HA enabled:

Now if you go and enable HA, you are in a world of hurt

I just deleted it instead of calling support so maybe they have a work around, but best bet is don’t do it!

EDIT: This is news to VMware apparently. Also I would really suggest removing the HA IP (if you configured it) before upgrading to 6.4!

Other Issue

The other issue is the “Connected To:” network for the HA interface.  In 6.3 you could easily set the same network for a regular (uplink/internal) interface and the HA interface.  and with 6.4 you can easily set them to be the same during the initial install.

But, after deployment, you can’t set the HA interface to one already used by an interface.

But you can set an interface to the one currently used by HA.

Is it a bug?  Are you not supposed to used the same network for HA and an interface?  If I find out I’ll let you know, but for now I’m creating a unique logical switch just for the control VM HA traffic.

EDIT:  Per VMware, set a dedicated network for HA, or use an uplink.  Exploiting the interface to set an internal network will cause problems (it will always fail the IP check).


I currently have an open support ticket out on why I lose traffic during a control VM failover – prior to 6.2.5  you would lose pings as the Active control VM would pull all routed from the hosts on its way down, but that was resolved. 

Note that I still saw it in 6.3.2 so YMMV.

Now what I see is the Edges changing the internal DLR networks to  “Weight 32768, AS Path ?” briefly when the Secondary takes over.  I have my BGP timers set to 1/3.  I’ll post what  support says when I hear from them.





Posted in Network, NSX, Virtualization, VMware | Tagged , , , | Leave a comment

VMware on AWS Quick Hits

vSphere plus SDDC Manager (VMware Cloud Foundation) plus NSX-V hosted on bare-metal at   AWS.

VMware manages hardware account and bills you (separate from any AWS account you have)

4 nodes to start – each 2 socket/36 core, 512GB RAM, 10.7TB storage (all flash)

Scales to 16 nodes currently

Just US (east and west regions) for now.

NSX-V with two IPSec VPN tunnels (one mgmt one compute) back to your data center (not required, but intended as hybrid)

Likes NSX on your DC but works with many hardware vendors for an IPSec endpoint.

One ESG, one DLR.

AWS services provided via direct connect (from compute Edge via a ENI) or internet connectivity.

Can connect to a AWS VPC Endpoint.

Includes routing table updates when connecting direct to AWS networks.

Gives a whole new meaning to “It’s not the cloud – it’s just someone else’s computer”

Posted in Cloud, Computing, Network, NSX, Security, Virtualization, VMware | Tagged , , , | Leave a comment

VMware AppDefense Quick Hits

Brought to you by the NSBU at VMware, home of NSX-V, NSX-T, VRNI.

Cloud-based with a local “proxy” (assuming local need).

Developer-focuses with tie-ins to development stream to track changes.

Supports Windows 2012, 2012R2, 2016 with *nix support coming.

Loads a “module” into the guest, which is protected in memory by the host

Watches a ton of “things” to determine proper operation and flags on improper

One “thing” : Creates hash of running executables and checks that hash periodically

Posted in Security, Virtualization, VMware | Tagged , | Leave a comment

VMware NSX-T Quick Hits

NSX-T(ransformers) is a multi-hypervisor (ESX/KVM) cousin of NSX-V.

Same SKU (if you own NSX-V 6.x, you own NSX-T 2.x).

One NSX-T Manager can have multiple vCenters as “Compute endpoints”

Standalone HTML5 client (not WebClient)

NSX-T Edges can be VMs or run on bare metal

Supports 8-way Edge ECMP but limited services on the Edge (vs V)

More/better BGP support/options/settings incl BFD

Uses Geneve instead of VXLAN for overlay due to extensible header

Protects cloud-native apps and container-level microsegmentation. The Google/Kubernetics project is protected by NSX-T

Posted in Network, NSX, Security, Virtualization, VMware | Tagged , , | Leave a comment

Migrate VMs between portgroups/virtual switches/vSS/vDS

I wrote this to help a client migrate to VXLAN from portgroups.

It pulls from a CSV file named c:\scripts\ImportPortGroups.csv which is structured:

123,dVLAN 123,

and accepts the initial number (VLAN) as a commandline parameter such as:
changeportgroups 123

The CSV file needs: identifier, portgroup name, virtual switch name

If you don’t enter a initial parameter it will remind you and exit.

Entering an initial parameter, the script grabs the line from the CSV where the first value matches your input.

It will then:
Pull the PG and VS from vcenter and verify that is what you are looking to work with.
Pull the number of VMs on the PG and VS and display and ask which way the move should go (all to VS or all to PG)
Migrate the VMs and display the # of VMs on the PG and VS.

You can get the script here.

FYI the one-liner version of the script is:

get-vm | get-networkadapter |where {$_.networkname -eq “OldPortGroup”} | set-networkadapter -networkname “NewPortGroup” -Confirm:$false

Posted in NSX, PowerShell, Scripting, Virtualization, VMware | Tagged , , , | Leave a comment

VMware Cloud Foundation Quick Hits

VMware Cloud Foundation consists of VMware vSphere, vSAN, NSX and “SDDC Manager”.

SDDC Manager will manage the lifecycle of the different components of VCF and can also manage the lifecycle of “add-ons” like VMware Horizon, VMware vRealize Suite, vRealize Operations and vRealize Log Insight.

Licensing is per-CPU but you’ll need to contact your reseller about how to buy it as while you can’t buy “SDDC manager” on its own you can reuse existing licenses for VCF – and vCenter isn’t included in VCF.

You can buy VCF pre-installed on VxRack or just buy vSAN Ready hardware and roll your own. You’ll want to check your switches against the VCF HCL also.

Who should buy VCF: anyone who wants vSphere/vSAN/NSX but doesn’t want to install the components themselves or manage patching them.

Posted in Cloud, Virtualization, VMware | Tagged , , | Leave a comment

Free NSX books from VMware

VMware NSX Micro-segmentation: Day 1 Guide

VMware NSX Micro-segmentation: Day 2 Guide

VMware Operationalizing NSX

Automating NSX for vSphere with PowerNSX

Posted in Network, Scripting, Security, Virtualization, VMware | Tagged , | 1 Comment

VMware badges – 2017 edition (vROps / vSAN)

[Edit: Added time/#Qs to vSAN after someone tried it and responded to me]

VMware certification has announced a series of “badges” that existing VCPs can add to demonstrate knowledge in either vROps or vSAN.

vSAN was announced last week

Right now the portal claims the exams are only available during VMworld US (Aug 27-29). My guess would be the price will go up after VMworld with a new date range.

Key points:
VCP required
$250 for vSAN or $125 at VMworld
$125 for vROps until 8/29 then ??

  • (note: I was told vROps has a 40% discount for the next few months, but that is not reflected at checkout, so YMMV)

at a Pearson center for vSAN
online for vROps

vSAN time: 110min plus 30min time extension for ESL.
vSAN quesitons: 60

vROps: The number of questions and time allowed are not currently posted. if anyone pays the $125 and finds out please let me know!

Note that vSAN claims a “high score” is required to pass but generally

Prep Guides (with sample questions and outline)
VMware vRealize Operations 2017 Specialist

VMware vSAN 2017 Specialist

If you are interested in taking an exam just as expensive and time consuming as a VCP that only counts as a “badge” let me know in the comments.

Posted in Certification, Virtualization, VMware | Tagged | 3 Comments

Get your VMware certification URL and PDFs

After all that hard work, it’s time to show off.  You can obtain PDFs of your VMware certifications or a URL listing all of them in the VMware Certification Manager.

The URL will look like https://www.certmetrics.com/vmware/public/transcript.aspx?transcript=V5C8E3G22MVQ1VJX and can be send to prospective employers, or added to LinkedIn as a verification of your skill set.

URL steps:
Step 1:
Login to VMware Education.

Step 2:
Click on the Certification Manger link.

Step 3:
Click Manage your Transcripts.

Step 4:
If you don’t have a transcript listed, create a new one.

Step 4a:
Leave all the defaults and enter an expiration for the transcript.  Note that the URL won’t work after this date.


Step 5:
Click the Link icon and copy the URL listed.

Step 6:


PDF Steps:

Step 1:
Login to VMware Education.

Step 2:
Click on the Certification Manger link.

Step 3:
Click Track your certification status.

Step 4:
Each certification (note that VCIX is a “badge” and doesn’t have a PDF) will have a PDF link next to it.

Step 5:
Print and hang on the fridge for all to see.

Posted in Certification, VMware | Tagged | Leave a comment