“VXLAN gave us a fixed 8-byte header. GENEVE gave us dynamic Type-Length-Value (TLV) metadata options. That structural difference is why VXLAN could not survive the transition to containerized microsegmentation.”

In early 2019, during my tenure as a VMware NSX Consultant at IBM Cloud, our architecture team was split over a core design decision for our next-generation software-defined datacenters.

The debate: Should we remain on VXLAN (the overlay protocol powering legacy NSX-V), or transition to GENEVE (Generic Network Virtualization Encapsulation) in NSX-T?

The traditionalists on the team argued strongly for VXLAN.

Their argument was logical: “VXLAN is standardized under RFC 7348. Every physical switch ASIC vendor—Arista, Cisco, Juniper—supports hardware VXLAN termination. Why abandon a proven protocol for GENEVE?”

They were looking at hardware compatibility. They were missing the control plane bottleneck.

---

The Fixed-Header Bottleneck

VXLAN was designed in 2011 for a simple purpose: overcome the 4,094 VLAN limit by encapsulating Ethernet frames into UDP packets with a 24-bit Virtual Network Identifier (VNI), supporting up to 16 million logical networks.

To accomplish this, VXLAN uses a fixed 8-byte header:

# VXLAN Fixed 8-Byte Header (RFC 7348)

0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|R|R|R|R|I|R|R|R| Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| VXLAN Network Identifier (VNI) | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Notice what is missing: zero space for custom metadata.

---

The Mess: Out-of-Band Controller Thrashing

As enterprise datacenters transitioned from simple multi-tenant subnets to zero-trust microsegmentation and containerized Kubernetes pods, hypervisors needed to exchange context with every packet.

When Host-A sends a packet to Host-B, Host-B’s firewall needs to answer three questions instantly:

1. What Security Tag is assigned to the source VM/Pod?
2. What TraceFlow ID is tracking this packet session?
3. Which Tenant Context owns this container pod?

Because VXLAN’s 8-byte header had no room for metadata, Host-B could only see the raw source IP.

To evaluate a Distributed Firewall (DFW) rule based on security tags, Host-B had to issue an out-of-band API query to the central SDN Controller cluster to look up the tag assigned to Host-A’s IP address!

# What happened under heavy East-West container traffic in VXLAN:
# 10,000 container pods spinning up/down -> IP addresses recycling constantly
# Host hypervisors flooded the SDN Controller cluster with out-of-band tag lookups!
# Controller CPU hit 100% -> API latency spiked -> DFW rule evaluation delayed by 50ms per packet!

Under heavy East-West microservice traffic, this out-of-band lookup architecture thrash-loaded the SDN Controllers. Control plane latency spiked, and packet evaluation slowed down across the entire datacenter fabric.

VXLAN’s fixed header was a bottleneck.

---

The Solution: Dynamic TLV Options in GENEVE

VMware, Red Hat, and Microsoft joined forces to draft GENEVE (RFC 8926), specifically designed to solve the metadata limitation.

GENEVE uses UDP port 6081 and introduces a variable-length header containing dynamic Type-Length-Value (TLV) option fields.

# Wireshark TShark Packet Capture of a Live GENEVE Packet Header
tshark -i vmk10 -Y "geneve" -V
  Geneve Header:
    Version: 0
    Protocol Type: IPv4 (0x0800)
    Virtual Network Identifier (VNI): 10050
    Option Class: VMware (0x013c)
    Option Type: Security Context (0x01)
    Length: 8 bytes
    Data: SecurityTag=Payment-Prod-App; TraceId=88421

Instead of Host-B making an out-of-band REST call to a central controller to discover Host-A’s security tags, Host-A writes the security tags directly into the GENEVE TLV header on the wire.

When Host-B receives the packet, its local hypervisor kernel reads the Option Class: 0x013c metadata in zero microseconds and evaluates the DFW firewall rule instantly at hardware speed.

No out-of-band controller lookups. Zero control plane thrashing.

# Verifying GENEVE MTU 1600 framing on host interfaces
esxcli network ip interface set -i vmk10 -m 1600

Because GENEVE’s variable TLV header adds extra bytes to the packet, we expanded the physical underlay network MTU from 1500 to 1600 bytes to prevent packet fragmentation.

---

The Impact

• Zero Control Plane Latency: Eliminated 100% of out-of-band controller lookup calls during microsegmentation rule evaluations.
• Container Context on the Wire: Allowed Kubernetes pod names, namespace tags, and security context to travel directly inside packet headers across multi-cloud networks.
• Scale Acceleration: Enabled NSX-T to scale to 10,000+ hypervisor transport nodes without overloading controller CPU resources.

Key Takeaway

Extensible Protocol Formats Outlast Fixed Standards.

Do not evaluate network protocols solely on legacy hardware compatibility. Fixed-header protocols like VXLAN cannot carry modern container and microsegmentation metadata on the wire. Choose GENEVE for software-defined overlays to leverage extensible TLV option headers for in-band security, tracing, and telemetry.

---

Architecture and decisions: mine. Debugging sessions at odd hours: mine. AI assistance: structure, syntax, first draft. — Sachin