Kingshuk Mandal
Senior Director & Distinguished Technologist

BFD over VXLAN: Resiliency in Data Center Networks

November 15, 2016 by Kingshuk Mandal

Virtual extensible LAN (VXLAN) is one of the most popular and effective overlay networking technologies for building data center networks. Using VXLAN a virtual overlay network can be built on top of existing Layer 2 or Layer 3 network infrastructure. These VXLAN overlays have been very effective in supporting elastic network architectures. The VXLAN-based network design logically isolates tenants and applications in a cloud environment.1

In a multi-tenant architecture, each tenant requires its own logical network to communicate with others. A virtual machine (VM) can communicate with another VM only if they are on the same logical network. Each logical network is identified by an identifier (a 24-bit number). In VXLAN’s terminology, this is called VXLAN network identifier (VNI) or VXLAN Segment ID. In short, VXLAN provides tunnels (identified by VNI) for communication that allows VMs pertaining to the same tunnel to communicate with each other. Hence it creates a virtual network for tenants.

The existence of the VXLAN tunnel is transparent to VMs. VXLAN tunnels are terminated at VXLAN tunnel end point (VTEP). The hypervisor hosting the VMs acts as the VTEP. TOR switches or gateway devices can act as VTEPs as well. VTEPs are responsible for doing the job of tunneling (encapsulation and decapsulation of the VXLAN header).

To provide successful communication between the tenants, it is very important to ensure that the VTEPs are alive, active, and heathy. This is where BFD comes into the picture. BFD over VXLAN helps to perform continuity check (CC) between two tunnel end points. BFD is an operation and maintenance (OAM) mechanism that can signal the breakage of continuity, and the higher layers can act to rectify the breakage or switch active communications to a backup path.

Over time, technology vendors have brought in many innovations to handle broadcast, unknown unicast, and multicast (BUM) traffic over a VXLAN network. VXLAN specification (RFC-7348) suggests mapping broadcast communication to multicast. This forces the underlay network to be multicast-capable and makes it necessary to run the multicast routing protocol as part of the underlay. But this approach complicates the underlay network and makes it difficult to manage for the network operator.

To keep the underlay simple, technology vendors have introduced the concept of “service node” or “replicator.” Whenever a VM has to send egress BUM traffic, it sends it to one service node only. The service node takes the responsibility to transport it to the desired VTEP or set of VTEPs. To achieve reliability, there could be multiple service nodes in the network. For a VTEP, ingress BUM traffic can arrive from any service node, but egress traffic is sent to one service node only.

Reliability and availability of service nodes are mission-critical for BUM traffic handling.  With help of BFD, the health of all service nodes can be monitored. If BFD signals one BUM service node as DOWN, then another BUM service node with BFD status as UP can be selected proactively without letting the communication among tenants fail.

2Thus validating reliability and performance of BFD over VXLAN is mission critical in a cloud architecture. Ixia’s industry-leading network infrastructure validation tool, IxNetwork, supports BFD emulation over VXLAN. It can emulate endpoints representative of “service node” or BFD-over-VXLAN-capable tunnel end points (VTEPS).  Highly configurable, Ixia’s BFD stack serves various network engineer’s needs. It can perform BFD-over-VXLAN-based continuity checks at different granularity levels. Rich statistics can show the stress conditions that cause a device under test to fail to respond to continuity checks or the stress-handling capacity of a device. IxNetwork makes it extremely easy for engineers to run the BFD-over-VXLAN emulation and benchmark device robustness, responsiveness to continuity checks, and ability to trigger corrective measures proactively.