Using VXLAN to network virtual machines, jails, and other fun things on FreeBSD

VXLAN allows an administrator to create many (up to 16 million) distinct virtual Ethernets over any UDP/IP transport which supports multicast. It combines features of both traditional VLANs and traditional point-to-point network tunnels (such as gif/EtherIP) while providing additional features found in neither. The underlying transport network can be Ethernet, InfiniBand, 802.11 wireless, or just about anything else. Hosts participating in one or more VXLAN networks do not need to be configured with any information about the other host(s) carrying the same network(s). Rather, they become virtual tunnel endpoints (VTEPs) and dynamically learn which (inner) network nodes are reached via which (outer) VTEPs, much like a switch associates MAC addresses with specific ports. Packets whose (inner) destination is broadcast, unknown or multicast are delivered via multicast on the outer network to all participating VTEPs.

It is perhaps most commonly used in large and/or multi-tenant virtualization projects, but VXLAN has many potential applications. Want to bridge your Ethernet across a wireless link? VXLAN has you covered. Can't get enough VLANs on your switch? VXLAN to the rescue. And of course it's a great way to have lots of networks for jails and virtual machines and keep them all distinct from each other and from the host machine's network(s).

FreeBSD, OpenBSD and Linux have all had VXLAN support for some time. Some switch vendors are starting to add it to their products as well (though one of the benefits of VXLAN is that it does not require switch support). This presentation will introduce VXLAN in greater detail, compare it to VLANs and traditional L2-over-L3 tunnels, and describe and demonstrate several use cases for VXLAN on FreeBSD including vnet jails, bhyve VMs, and wireless bridging.