Sunday, September 29, 2019

NSX-T 2.5 Getting Started, Part 1

Since NSX-T 2.5 just came out, it's about time to do a full rebuild and getting started guide. NSX-T differs greatly from NSX-V in that the initial setup is quite a bit more complicated and doesn't have many guardrails or direct paths to initial set-up.

We'll be skipping the appliance deployment, because if you have troubles deploying an OVA this will probably be too difficult.

First off, we'll be using our applied Clos fabric for this, and we won't be multihoming these devices as of yet, as this post will be pretty lengthy as it is. Diagram is here:

With that in mind, the first step to configuring virtualized routing & switching for NSX-T is in the vCenter GUI. In this lab, I have two hosts in two separate clusters -

  • Payload: Virtual Tunnel Endpoints (VTEPs) exist primarily on the host, and are leveraged as port-groups for guest network connectivity
  • Management/Edge: No host VTEPs currently exist, as they are not required for the management VMs, nor for the Edge Appliances (Primary difference coming from NSX-V!)
Coming from the vCenter UI, it looks like this:
The NSX-T Edge Appliances need to ingest underlay networks via 802.1Q tags, instead of as individual port groups. Fortunately, vSphere has been able to do this for quite some time, so we use the lesser-known "VLAN trunking" 

From here, it's time to outline our Edge Design - BEFORE anything is built.
We'll use this as a guide throughout the configuration process., First, we set transport zones and device profiles:
We create the underlay (VLAN) transport zone to ensure that virtualized traffic can exit to the "real network":
We create the overlay network where the GENEVE VN-Segments will live next:
Then we configure the Layer 2 uplink profiles. Note: specifically configuring the Active uplink to FP-ETH0 is REQUIRED. The NSX Edges will not function without this, and NSX-T will never tell you why.
And the VTEP profiles. Note that this portion uses the name allocated in the transport node profile.
Finally, the host transport profiles. Here we set a profile that will use a single uplink for the N-VDS, add transport zones, etc. Note that the physical NIC name on the left needs to exactly match the physical NIC identifier in ESXi.
Now, we can finally start configuring transport nodes. Note that since we deployed profiles prior to this, there's not a whole lot to do as far as roll-out is concerned.

Ensure the edge appliance is ready:
Configure the edge cluster:
Now we're ready to configure routing and switching functionality. This can go several different ways, as VMWare has provided additional capabilities with regards to configuring NSX-T assets - declarative configuration methods. We'll cover that in detail, along with how to use it, in the next post!

Saturday, September 21, 2019

Spine and Leaf Networks, an Outline

With the previous post, I covered traditional data center networking, and some of the reliability compromises made to accommodate typical workloads. Here I'll be outlining my take as a series of blog posts on the next iteration of data center network design, Spine and Leaf:
Lab Diagram
Note: This page may change based on content. I'm going to try and keep things generic up until the very end.

NSX-T Datacenter 2.5 Upgrade Process and Preview

Now that NSX-T Datacenter 2.5 is downloadable, it's time to try this out in my home lab.

First things first, if you log in more than 90 days out, you'll be locked out of the appliance completely. If you make any changes the normal linux way (passwd and chage) the appliance will automatically revert it in about a minute. Since this is a home lab, VMWare has added the capability to set a higher maximum age here. In production, use Active Directory or another LDAP source to prevent yourself from losing NSX-T.

Downloading the upgrade bundle took quite a while - it seems that VMWare is having troubles with hosting capacity. I'm guessing that it's going to be a popular release!

We start these in the usual way, by uploading the upgrade bundle (.mub file):
Then we hit the upgrade button, and it prompts you step-by-step throughout the process. No progress bars, though!
Once the upgrade coordinator is set up, it's time to run pre-checks. This one warns you about the messaging port change.
It's time to start upgrading stuff!
As with other 2.4 releases, you don't have to use maintenance mode (expect an outage if you do that).
The management node will be unreachable unless you roll-out a cluster. If you see an "appliance unhealthy" or "Error Status 101" message, this simply means the appliance isn't ready yet.

Post-upgrade, we can see BGP statuses as promised:

Capacity Management is there:
Unfortunately, I see BGP status but no route table in the GUI. The documentation will probably direct things somewhat, but I do not see the full capabilities there.

NSX-T Datacenter 2.5 Released!

As of 19 September 2019, NSX-T 2.5 has been officially released and is available for download!

It's been a bit since the announcement, so let's cover some of the new capabilities of interest with NSX-T 2.5. This is a summary of what I found interesting, the complete release notes are here

NSX Intelligence

VMWare will be introducing a new paid service to analyze traffic handled by distributed firewalling, to allow infrastructure administrators to map out service applications, ports, and policies to better secure their east-west network environment. It will also provide the capability that NSX-V has natively, Application Rule Manager

Testing and Troubleshooting

VMWare added a ton of good stuff here, some of which seems a little late...

Layer 2 MTU/VLAN Checking

This one has been a big pain point for NSX administrators everywhere, especially if they don't also control the route-switch infrastructure. Prior to this, NSX-T had tunnel status (which would alarm if no VMs in a port group were on a host, causing a LOT of noise) and NSX-V had nothing.

Layer 3

We get BGP routing information from the API and GUI for the first time!

New Capabilities


We pick up SLAAC, Router advertisements allowing for automatic IP configuration. Ideally, this would not be something we really need - but I'm sure there's a use case somewhere.

Firewalling and Security

  • NSX-T now supports configuration management as well, with config drafts!
  • NSX Cloud is beginning to support native constructs in public cloud for security enforcement. This is a pretty big deal for hybrid cloud shops that won't have to use an agent to enforce consistent multi-cloud security!
  • VMWare has introduced Layer 7 (App-ID) support for gateways and is beginning to introduce FQDN filtering as a precursor to URL filtering.
  • VMWare has also added Identity-based firewalling.
  • Elliptic Curve Cryptography over IPSEC is now available
  • Preset compliance suites for VPNs are also available


  • Load Balancing GUI Improvements - We'll see the simplified GUI in a bit.
  • SNMPv3 Polling is supported on all appliances
  • The NSX-V to NSX-T migration tool has unlisted improvements
  • NSX Manager to Edge communication is changing ports - from 1234 to 5671. This could potentially break connectivity during an upgrade. Port 1235 does still need to be open.
Next, let's try it out!

Popular Posts