How Netflix uses eBPF flow logs at scale for network insight — screenshot of netflixtechblog.com

How Netflix uses eBPF flow logs at scale for network insight

Netflix built a performant eBPF-based sidecar, Flow Exporter, to capture TCP flows at scale for network observability. This provides critical insight into their vast cloud network, addressing challenges like app dependencies and segmentation.

Visit netflixtechblog.com →

Questions & Answers

What is Netflix's Flow Exporter?
Netflix's Flow Exporter is a network observability sidecar that leverages eBPF tracepoints to capture TCP flows in near real-time on instances powering their microservices architecture. It operates with minimal CPU and memory consumption.
Who benefits from Netflix's eBPF flow log solution?
Service owners, centralized teams, security teams, and other partner teams at Netflix benefit from this solution. It provides visibility into application dependencies, data flows, network bottlenecks, and supports incident analysis and security monitoring.
How does Netflix's eBPF flow logging compare to traditional VPC Flow Logs?
While Netflix also uses VPC Flow Logs, their eBPF Flow Exporter offers near real-time TCP flow capture directly on instances, providing more granular and performant data. It is an in-kernel solution that extends functionality safely, offering deep insights into network activity.
What are the primary use cases for Netflix's eBPF flow data?
The eBPF flow data is used for network monitoring, network usage forecasting, machine learning-based network segmentation, and by security and partner teams for insight and incident analysis across Netflix's cloud ecosystem.
How does Netflix scale and enrich its eBPF flow logs?
Netflix uses a regional service called Flow Collector to ingest and enrich eBPF flows. It consumes data from Flow Exporters and IP address change events from Sonar, attributing application metadata to flow data before routing it to Hive and Druid datastores.