“OSPF is fast and automatic for flat internal networks. But in a multi-tenant cloud datacenter, OSPF Link State Advertisement (LSA) flooding will crash your core switch CPUs during host migrations. External BGP is the only protocol built for multi-tenant gateway isolation.”

In November 2019, during my tenure as a VMware NSX Consultant at IBM Cloud, we led the routing architecture design for a multi-tenant cloud platform.

The client’s internal networking team was in a heated debate over which dynamic routing protocol to use to peer our NSX Tier-0 Gateway Edges with their physical Arista 7050 top-of-rack switches.

The client’s network lead advocated strongly for OSPF:

“OSPF is simple and automatic. It uses multicast (224.0.0.5) to discover neighbors instantly with zero manual IP peering setup. Why complicate our lives with BGP AS numbers, route maps, and neighbor statements?”

He was evaluating the protocol based on initial setup ease. He was ignoring the failure domain of link-state algorithms under cloud scale.

---

Link-State vs. Path-Vector Dynamics

To understand why OSPF fails at the cloud border, you must understand how routing engines process state updates:

• OSPF (Link-State Algorithm): Every router in an OSPF Area maintains a 100% identical copy of the entire network topology database. If a single interface flaps, OSPF floods Link State Advertisements (LSAs) to every node in the Area, forcing all switches to run Dijkstra’s Shortest Path First (SPF) calculation.
• eBGP (Path-Vector Algorithm): Peering is established via explicit, point-to-point TCP unicast connections (port 179). BGP routers only exchange path vectors (prefixes and AS paths)—never raw topology maps. A subnet flap in Tenant A is handled locally without affecting Tenant B.

---

The Mess: The SPF Recalculation Storm

To demonstrate why OSPF could not be used at the border gateway layer, we built a staging proof-of-concept peering our Tier-0 Edges to physical core switches using OSPF Area 0.

We simulated a routine cloud maintenance event: evacuated an ESXi hypervisor host running 50 tenant subnets, causing 50 virtual interface endpoints to flap briefly.

The result was an operational disaster.

# Arista Physical Core Switch Console Log during OSPF Subnet Flap
%OSPF-5-ADJCHANGE: Neighbor 192.168.200.2 (Tier-0-Edge) Link-State Update received
%OSPF-4-FLOODING_STORM: LSA Type 3 flood rate exceeded 500 LSAs/sec in Area 0
%SYSTEM-2-CPU_OVERLOAD: Process 'ospf' consumed 100% CPU | Running Dijkstra SPF calculation...

The 50 virtual interface flaps generated an LSA Type-3 flooding storm across OSPF Area 0.

Every physical core switch in the datacenter froze as its main CPU hit 100% utilization, re-calculating Dijkstra’s SPF tree over and over.

For 45 seconds while SPF recalculated, global routing tables were locked. Traffic across unrelated tenants was dropped datacenter-wide.

Worse, OSPF has zero native support for multi-tenant traffic manipulation. It lacks BGP Community attributes (NO_EXPORT, LOCAL_PREF), making it impossible to steer traffic per tenant or prevent cross-tenant route leaks.

---

The Solution: Standardizing on eBGP for Gateway Peering

We banned OSPF from the cloud border layer and standardized on eBGP (External BGP) for all Tier-0 Gateway peering.

# Cloud Datacenter Routing Layer Separation

| Routing Layer                      | Protocol Choice    | Operational Rationale                                                |
| ---------------------------------- | ------------------ | -------------------------------------------------------------------- |
| **Physical Underlay (Leaf-Spine)** | OSPF / IS-IS / BGP | Fast internal hardware topology convergence                          |
| **SDDC Overlay Border Gateway**    | **eBGP (TCP 179)** | Surgical per-tenant policy control, community tagging, LSA isolation |

! # Arista ToR Switch eBGP Neighbor Policy for NSX Tier-0 Gateway
router bgp 65000
  router-id 192.168.200.1
  neighbor 192.168.200.2 remote-as 65001
  neighbor 192.168.200.2 send-community standard
  neighbor 192.168.200.2 route-map TENANT-STEERING-IN inbound
!
route-map TENANT-STEERING-IN permit 10
  match community 65001:100
  set local-preference 300 # Primary path for Tenant A

Why eBGP Wins at the Cloud Border

1. LSA Isolation: When 50 tenant subnets flap, eBGP sends simple BGP WITHDRAW messages over TCP port 179. No LSA flooding occurs, and switch CPUs remain at a baseline 5% utilization.
2. Surgical Path Control: By utilizing BGP Communities (set local-preference 300), we steer Tenant A’s primary traffic out Gateway 01 and Tenant B’s traffic out Gateway 02 with zero static routing hacks.
3. Deterministic Failure Boundaries: eBGP enforces explicit administrative boundaries between the physical network team (AS 65000) and the cloud platform team (AS 65001).

---

The Impact

• Zero SPF CPU Spikes: Eliminated 100% of core switch CPU spikes during host evacuations and subnet flaps.
• Per-Tenant Traffic Steering: Implemented active-active load distribution across Tier-0 Edges using BGP Community local preferences.
• Deterministic Isolation: Established clear BGP Autonomous System boundaries between physical underlay and virtual overlay teams.

Key Takeaway

Standardize on eBGP for Multi-Tenant Overlay Gateway Peering.

Never use OSPF to peer software-defined cloud gateways with physical core switches. OSPF LSA flooding will trigger datacenter-wide Dijkstra SPF recalculation storms during routine virtual interface flaps. Reserve OSPF strictly for internal physical underlays, and deploy eBGP at the border to leverage BGP Community tagging, path vector isolation, and deterministic traffic steering.

---

Architecture and decisions: mine. Debugging sessions at odd hours: mine. AI assistance: structure, syntax, first draft. — Sachin