https://www.linkedin.com/company/network-switch/
Executive Summary (TL;DR)
- Answer first: VXLAN extends Layer 2 segments over an IP underlay, while EVPN can distribute overlay reachability; a production fabric still requires a validated underlay, overlay, multihoming, gateway, MTU, policy, failure, operations, and interoperability design. Use RFC 7348 and RFC 8365. Continue through ASR 9000 deployment guide, VXLAN/BGP-EVPN guide, Cisco AI and SD-WAN remediation update, Cisco Cloud Control and SD-WAN alert, AI infrastructure and WAN report digest, the enterprise BGP guide, and the Router 101 hub. Evidence boundary: vendor announcements, specifications, projections, and preserved examples are source-attributed inputs, not an independent benchmark or guaranteed outcome. Support boundary: capability, compatibility, software, licenses, lifecycle, remediation, service, credentials, delivery, and commercial terms require exact PIDs, releases, dates, entitlement, region, and written evidence.
- The Solution: VXLAN (Virtual eXtensible LAN) encapsulates Layer 2 Ethernet frames inside Layer 3 UDP packets, allowing you to stretch a Layer 2 network over a robust, routed Layer 3 infrastructure.
- Massive Scalability: VXLAN replaces the 12-bit VLAN ID with a 24-bit VNI (VXLAN Network Identifier), expanding the limit from 4,096 to 16 million isolated networks.
- Architecture boundary: BGP EVPN with VXLAN is a common data-center design, not a universal 2026 requirement. Select it only when topology, scale, operations, features, and interoperability justify it.
- Hardware boundary: select spine, leaf, route-reflector, gateway, optics, and cabling by exact feature and scale requirements. Vendor mix is acceptable only after control- and data-plane interoperability testing.
Breaking the Layer 2 Boundary
Imagine a scenario common in modern enterprise data centers: You need to migrate a virtual machine (VM) from Rack A to Rack Z without changing its IP address or dropping its active connections. To do this, Rack A and Rack Z must be on the exact same Layer 2 subnet.
Historically, engineers achieved this by extending VLANs across the entire network using trunk links. However, stretching Layer 2 across a large campus relies on Spanning Tree Protocol (STP), which blocks redundant paths, wastes bandwidth, and risks catastrophic broadcast storms.
Evidence boundary: AI and virtualized workloads do not make every VLAN design obsolete. Use measured scale, failure domains, mobility, operations, latency, loss, congestion, and lifecycle requirements to choose the fabric.
What is VXLAN? (The MAC-in-UDP Magic)
At its core, VXLAN is a tunneling protocol. It takes an original Layer 2 frame generated by a server and wraps it inside a Layer 3 UDP packet. This process is known as MAC-in-UDP encapsulation.
Because the packet is now a standard IP packet, it can be routed across your core network using Equal-Cost Multi-Path (ECMP)-taking full advantage of all available physical links without Spanning Tree getting in the way.
To make this work, VXLAN introduces two critical hardware concepts:
- VTEP (VXLAN Tunnel End Point): The hardware (usually a Top-of-Rack Leaf switch) responsible for encapsulation and decapsulation.
- VNI (VXLAN Network Identifier): The VXLAN equivalent of a VLAN ID, utilizing a 24-bit space to allow for 16 million unique networks.
Feature Comparison: Traditional VLAN vs. Modern VXLAN
| Feature | Traditional VLAN | Modern VXLAN (BGP-EVPN) |
| Network Scale | Max 4,096 segments | Max 16 million VNIs |
| Layer 2 Protocol | Spanning Tree (STP) - Blocks links | ECMP - 100% Link Utilization |
| Transport Method | Native Ethernet (Layer 2) | UDP Encapsulation (Layer 3) |
| Workload Migration | Limited to physical broadcast domains | Seamless cross-rack, cross-region migration |
Underlay vs. Overlay Networks Explained
To understand VXLAN, you must separate the physical network from the logical network.
The Architectural Metaphor:
The Underlay is the physical highway, while the Overlay is the private express lane for your VMs.
The Underlay Network (The Physical Highway):
This is your actual hardware-the fiber optic cables, the Huawei Spines, the NSComm Leafs. The Underlay is strictly a Layer 3 routed network. It uses protocols like OSPF or BGP to establish IP connectivity. The Underlay doesn't know anything about MAC addresses or VMs; its only job is to move IP packets from Switch A to Switch B as fast as possible.
The Overlay Network (The Private Express Lane):
This is the virtual Layer 2 network built on top of the Underlay. Through VXLAN tunnels, a server plugged into a Leaf switch in New York can believe it is on the exact same local Ethernet switch as a server in London.
BGP-EVPN: The Brains of AI-Ready Data Centers
Early versions of VXLAN relied on "Flood and Learn" mechanisms using Multicast to find MAC addresses. Today, the industry standard is VXLAN with BGP-EVPN (Ethernet Virtual Private Network).
Instead of flooding the network, EVPN uses the BGP routing protocol as a highly intelligent "control plane." When a new server connects to an NSComm Leaf switch, the switch learns its MAC and IP address and instantly advertises it to all other switches in the fabric via BGP.
AI/RoCEv2 boundary: VXLAN and QoS do not guarantee lossless behavior. Validate PFC or ECN design, queues, buffers, MTU, hashing, congestion, telemetry, failure behavior, and application performance on the exact fabric.
The 2026 Strategic Multi-Vendor VXLAN Blueprint
Building a VXLAN fabric entirely with Tier-1 OEM hardware will exhaust your CapEx budget. Because VXLAN is an open IEEE/IETF standard, the smartest IT architects utilize a Strategic Multi-Vendor Architecture.
How VXLAN Works in 3 Steps:
- Encapsulation: The NSComm Leaf switch (VTEP) receives a standard Ethernet frame from a server and wraps it in a VXLAN header.
- Transport: The packet travels across the Huawei/Ruijie Underlay as a standard Layer 3 IP packet via ECMP routing.
- Decapsulation: The destination Leaf switch strips the header and delivers the original Layer 2 frame to the target server.
Here is our lab-verified deployment blueprint:
- The Spine Layer (Route Reflectors): Deploy Huawei CloudEngine CE8800 or Ruijie 800G Data Center Switches. These switches act as the high-speed Underlay core and BGP Route Reflectors. They process massive routing tables without breaking a sweat.
- The Leaf Layer (VTEPs): Deploy NSComm 25G/100G Data Center Switches at the Top-of-Rack. These act as the VTEPs. They take the Layer 2 traffic from your servers, perform the VXLAN encapsulation in hardware (ASIC), and route it up to the Spine.
- The Physical Underlay (Optics): VXLAN generates massive amounts of East-West traffic. Connect your NSComm Leafs to your Huawei Spines using lab-verified NSComm QSFP28 (100G) or OSFP/QSFP-DD (400G/800G) optical transceivers.
Expert Field Notes: The #1 VXLAN Troubleshooting Mistake
When clients call our support desk complaining that their newly deployed VXLAN fabric is suffering from intermittent packet drops, random application timeouts, or failing OSPF neighbor adjacencies, 90% of the time, the culprit is MTU (Maximum Transmission Unit).
A standard Ethernet frame is 1500 bytes. When a VTEP encapsulates that frame in VXLAN, it adds an outer MAC, outer IP, UDP header, and VXLAN header. This adds 50 to 54 bytes of overhead. If your physical Underlay switches are still set to the default 1500-byte MTU, they will instantly drop the encapsulated packet.
Lab boundary: no reproducible topology, exact PIDs, software, configuration, traffic, packet size, MTU sweep, raw counters, timestamps, or reviewer record was attached; the fixed 9216-byte recommendation is withdrawn.
Frequently asked questions (FAQs)
Can I mix different switch brands as VTEPs in the same fabric?
Multivendor VTEPs can interoperate only when the exact platforms and releases align on VXLAN and EVPN route types, encapsulation, multihoming, gateways, MTU, BFD, policy, scale, telemetry, and failure behavior. Reproduce the design before production.
Does VXLAN replace VLANs completely?
No. VLANs can remain at the access edge and map to VNIs, while VXLAN carries the overlay across an IP underlay. The exact mapping, gateway, flood handling, policy, and scale are design-specific.
Why is UDP used for VXLAN instead of TCP?
VXLAN uses UDP encapsulation and its source-port entropy can help underlay ECMP hashing. Validate hashing fields, polarization, MTU, fragmentation, paths, flow distribution, congestion, and platform implementation.
Future-Proof Your Architecture
Designing a BGP-EVPN VXLAN fabric requires precise hardware selection, compatible ASICs, and flawless optical connectivity. As your Global Enterprise Network Infrastructure Partner, Network-Switch.com offers:
- Verified Interoperability: We guarantee our NSComm Leaf switches and optical modules work seamlessly within a Huawei or Ruijie VXLAN environment.
- Expert Topologies: Consult with our CCIE/HCIE engineers to map out your Underlay OSPF/BGP routing and VNI architecture.
- Global Agile Delivery: Procure your entire multi-vendor BOM and have it delivered in as little as 5 days.
Contact us today to discuss your data center migration and receive a customized hybrid hardware quote.
Did this article help you or not? Tell us on Facebook and LinkedIn . We’d love to hear from you!
