DragonFlow SDN based Distributed Virtual Router for OpenStack Neutron

In my previous posts I presented a PoC implementation of an embedded L3 controller in Neutron that solves the network node bottleneck the SDN way, and overcomes some of the DVR limitations.

In this post I am going to cover the first release of DragonFlow - an SDN based Distributed Virtual Router L3 Service Plugin (aligned with OpenStack Kilo), now an official sub-project of Neutron.

The main purpose of DragonFlow is to simplify the management of the virtual router while improving performance, scale and eliminating single point of failure together with the network node bottleneck.

The DrangonFlow solution is based on separation of the routing control plane from the data plane. This is accomplished by implementing the routing logic in distributed forwarding rules on the virtual switches (called "flows" in OpenFlow terminology). To put this simply, the virtual router is implemented using OpenFlow flows only.

DragonFlow eliminates the use of software stack to act as a virtual router (the Linux network namespaces in the DVR and the legacy L3 architecture), it use purely OVS flow to act as a virtual router. A diagram showing DragonFlow components and overall architecture can be seen here:

DragonFlow High-level Architecture

DragonFlow Features for the Kilo Release:

• East-West traffic is fully distributed using direct flows, reactively installed upon VM-to-VM first connection.

• Support for all ML2 type drivers GRE/VXLAN/VLAN

• Support for centralized shared public network (SNAT) based on the legacy L3 implementation

• Support for centralized floating IP (DNAT) based on the legacy L3 implementation

• Support for HA, in case the connection to the Controller is lost, fall back to the legacy L3 implementation until recovery. Reused all the legacy L3 HA. (Controller HA will be supported in the next release).

Key Advantages:

• Performance improvements for inter-subnet network communication by removing the number of kernel layers (namespaces and their TCP stack overhead)
• Scalability improvements for inter-subnet network communication by offloading L3 East-West routing from the Network Node to all Compute Nodes
• Reliability improvements for inter-subnet network communication
• Simplified virtual routing management, mange only active flows and not all possible 
• Non Intrusive solution, does not rely on ML2 modification

How it works

1. On bootstrap the L3 service plugin sends an RPC message to the L2 service plugin, setting the L3  Controller Agent as the controller of the integration bridge.
2. The Controller queries the OVS  for its port configuration via Openflow, matches the actual ports configured on the OVS to the Neutron tenant networks data model. 
3. Then, it installs the bootstrap flow pipeline that offloads all L2 traffic and local subnet L3 traffic into the NORMAL pipeline, while sending every unmatched VM-to-VM inter-sunbet traffic to the controller.

DragonFlow L3 Service Bootstrap

The following diagram shows the multi-table OpenFlow pipeline installed onto the OVS integration bridge (br-int) in order to represent the virtual router using flows:

bootstrap flows pipeline

The base table pipe-line is installed proactively on bootstrap while the East-West rules on the L3 Forwarding table are installed reactively on each first VM-to-VM communication.

If you would like to try it yourself, install guide is available here.

To join the development effort: 

• Homepage: http://launchpad.net/dragonflow
• Source: http://git.openstack.org/cgit/stackforge/dragonflow
• Bugs: http://bugs.launchpad.net/dragonflow
• Blueprints - https://blueprints.launchpad.net/dragonflow

My next post will cover the L3 reactive OpenFlow application, and how we install the East-West reactive flows. 

Hybrid OpenFlow Switch

In my last post I summarized the DVR solution and tried to explain the motivation for yet another L3 implementation in Neutron that I am going to present in the coming posts.

This 2-post series is intended to cover basic SDN and OpenFlow mechanisms that we used in the L3 controller:

• Hybrid OpenFlow Switch 
• SDN models for managing the forwarding elements (switches)

Hybrid OpenFlow Switch

Hybrid OpenFlow switch was introduced in OpenFlow/1.1. Hybrid switches support both OpenFlow operation pipeline and normal (legacy) Ethernet switching functionality.

The hybrid switch allows forwarding of packets from the OpenFlow

pipeline to the normal pipeline through the NORMAL and FLOOD reserved ports.

The main reason for introducing the hybrid switch was to optimize the handling of operations like MAC learning, where a reactive approach was just not efficient - Doing MAC learning in the OpenFlow controller poses a significant cost in terms of network bandwidth and latency and does not scale for large networks.

The NORMAL action comes to the rescue and lets us offload legacy non-OpenFlow pipeline (like MAC learning mechanism, VLAN, ACL, QoS and other base features) to the forwarding element kernel module, which is optimized to handle such operations in near-line-rate.

But what happens when the NORMAL action is used in an OpenFlow flow?

Basically, what happens is that the traffic is redirected to a completely separated processing pipeline. This is illustrated in the diagram below.

OVS Hybrid OpenFlow Switch Pipelines

The OpenFlow pipeline and the Normal Pipeline, each act as a completely isolated switch.

There are, however, some issues with this hybrid approach.

• NORMAL pipeline is not standardized, so it behaves differently on switches from different vendors - There is a variance in the supported features, and no standard way to configure them
• NORMAL pipeline does not play well with some OpenFlow actions, for example if the port is tagged for the NORMAL pipeline (using the ovs-vsctrl), you can not tag it using OpenFlow actions and then forward it to the NORMAL path, because it will end-up dropped due to double tagging error.

The Open virtual Switch extensions to OpenFlow were developed to support these extra features using Flows, for example the LEARN action (Open vSwitch extension to OpenFlow) for MAC learning .

In my next post I will cover the SDN models for managing the forwarding elements in reactive, proactive and mix modes.