Sonata Telemetry System Paper Review

ReviewNetwork


Problem

How to build a telemetry system that is both scalable and expressive with the help of programmable forwarding planes?

Introduction

Existing network traffic has grown exponentially, easily reaching terabits per second at an exchange point. Conventional telemetry system either sacrifices expressiveness by recording a subset of information of the traffic or scalability by heuristically representing the packets in the storage.

Previous Work

Chimera, Gigascope, OpenSOC, and NetQRE are existing telemetry systems that only support low line rates. Everflow, UnivMon, OpenSketch, and Marple, on the other hand completely execute on the forwarding plane and loses the expressiveness of queries. Works in other fields like database systems provide the concept on query planning, allowing query to be partitioned into different stages.

Implementation

The paper advances the state of the art by creating a telemetry system that provides an abstraction based on map-reduce for the network operators to express the queries. The framework requires limited storage space and leverages the existing stream processing framework to handle the query. Additionally, multiple optimizations on reducing the count of processed packets while not losing the expressiveness are proposed.

There are mainly four components in the system: Runtime system, Streaming system, programmable forwarding planes, and the Interface. The runtime system interacts with the streaming system to create an optimized query plan for execution on the forwarding plane and streaming driver. Network operators could express or tweak the query without knowing the underlying execution details.

Insights

Packet abstraction is made to allow network operators to define queries based on the packet semantics. To support scalable query execution, query partitioning is used to assign part of the query to be handled by the forwarding plane, which is faster but less flexible, and rest to the stream processor. Additionally, dynamic refinement is used to further optimize the query execution by planning the query to be worked on more interesting traffics.

Future Work

Sonata does not support reasoning with the packet at the byte level and distributing it to multiple switches. However, the existing industrial applications of Sonata does not involve handling multiple switches and Sonata could fulfill the requirement with minor tweaks. Additionally, more optimizations for query execution and storage could be found by future researches.