Latency and Time to first Packet/Byte

This question has come up a few times from customers and researchers alike.  How much latency do you add using an OpenFlow based networking stack.

The question has a few answers and I will try to cover them here.

The main answer is, the added latency normally only affects the first packet or two.  In my setup I generally see between 20-30 ms RTT for the first packet, then 0.1-0.2 ms after that.  This is an important factor to consider.  If your OpenFlow based networking stack (i.e. a router, controller and any shims you have inserted such as RFProxy) is on a fast machine, you should see the TTFP/TTFB in about the same range.  I’ve been sending 64 byte packets at line rate across 20+ interfaces and experiencing around 30 ms.  Once the flow is programmed, all of the latency comes from the switch/router only. i.e. the OpenFlow controlled device.

So to properly gauge the impact you need to first determine what the latency/jitter/delay is of your device when doing OpenFlow based forwarding.  Most switches are very fast, under 0.1 ms, in fact some switches are in even faster.  This is a benefit of OpenFlow based forwarding.  While in a normal network stack, you would send ALL packets through multiple devices, with OpenFlow you can program the first device to send the packet straight into the network.

Here is a text based diagram.

First Packet:

Packet -> OF Switch -> Controller -> Router -> Proxy -> Controller -> OF Switch -> End Device

Second to nth packet:

Packet -> OF Switch -> End Device

You can see just from the diagram how much is cut out of the path.  In a normal network you would always be in the middle ground such as:

Packet -> Router -> Switch -> End Device

With OpenFlow you can cut out the Router at a minimum.  Yes you take a hit on the TTFB, but after that, things go much faster.  As technology improves, you will see pre-population of OpenFlow paths, better optimization, etc.  In the end the OpenFlow (or just the Network API) world can be much better.

Being December 25th, I hope everyone has a wonderful day be it a normal day or a family centric day.