What is NetFlow? An Overview of the NetFlow Protocol
What is NetFlow?
NetFlow is a protocol used to collect metadata on IP traffic flows traversing a network device.
Developed by Cisco Systems, NetFlow is used to record metadata about IP traffic flows traversing a network device such as a router, switch, or host. A NetFlow-enabled device generates metadata at the interface level and sends flow data information to a flow collector, where the flow records are stored to enable network traffic analytics and management. A network operator can use NetFlow data to determine network throughput, packet loss, and traffic congestion at a specific interface level. NetFlow data also supports other network-level monitoring use cases such as DDoS detection and BGP peering.
The History of NetFlow and Versions Over Time
Netflow was initially developed by Cisco in 1995 to monitor and record all network traffic coming through their network devices. Over time, they realized that the network flow data was incredibly useful and it led to an entire branch of network monitoring that several other platforms have copied and rebranded. Now, a version of NetFlow protocols have become industry standard for optimizing network performance. This ongoing evolution of NetFlow has led to several versions over the years with different features.
NetFlow Variants
While the term “NetFlow” is commonly used to refer to all types of flow records and datagrams, there are actually three important variants in regular use within live production networks:
- NetFlow is the technology and term used exclusively by Cisco Systems.
- IPFIX is an IETF standard flow record format that is very similar in approach and structure to NetFlow v9 (see more on NetFlow version numbering below). It is sometimes called “NetFlow v10” since IPFIX plays a key role in coalescing all NetFlow variants and equivalents as the standards process evolves the IPFIX specifications over time.
- sFlow is a similar but importantly different type of flow protocol and data record standard introduced and promoted by InMon Corp. sFlow does not sample all packets like NetFlow does, nor does it timestamp traffic flows. It relies on accurate and reliable statistical sampling methods for documenting flows, thereby reducing the amount of flow information that ultimately needs processing and analysis.
- Other vendor-specific flow record formats that are similar in nature to one of three most common variants listed above (in most cases these are either substitutions or very close adaptations) include:
– J-Flow from Juniper Networks, which essentially conforms to NetFlow v5;
– NetStream from 3Com/Huawei. - xFlow, while not a variant, is the generic term often used to refer collectively to all flow record variants (NetFlow, sFlow, IPFIX, J-Flow, etc.).
NetFlow Versions
The fields that make up a NetFlow flow record depend on the version of NetFlow supported by the NetFlow exporter. Since the protocol was first introduced by Cisco in 1996 it has been updated numerous times in a series of backward-compatible versions, the most commonly used of which are versions 5 and 9:
- v1: First implementation, now obsolete, and restricted to IPv4 (without IP mask and AS Numbers).
- v2: Cisco internal version, never released.
- v3: Cisco internal version, never released.
- v4: Cisco internal version, never released.
- v5: First commonly deployed version, available (as of 2009) on many routers from different brands, but restricted to IPv4 flows.
- v6: No longer supported by Cisco.
- v7: Like version 5 with a source router field.
- v8: Several aggregation forms, but only for information that is already present in version 5 records
- v9: Template based, available (as of 2009) on some recent routers. Mostly used to report flows like IPv6, MPLS, or even plain IPv4 with BGP nexthop. Includes added fields for security and traffic analysis use cases
How Does NetFlow Work?
NetFlow Components
NetFlow monitoring solutions are typically comprised of three main components:
- Flow exporter: A NetFlow-enabled device that generates flow records and periodically exports them to a flow collector.
- Flow collector: A program running on a server that is responsible for receiving, storing, and pre-processing flow records received from NetFlow-enabled devices.
- Flow analyzer: An analysis application that processes flow records collected by the flow collector into reports, alerts, and other interpreted results.
Flow Records and Data
A NetFlow exporter (NetFlow-enabled device) identifies a flow as a unidirectional stream of packets having in common (at least) the following:
- Input interface port
- IP source address
- IP destination address
- Source port number
- Destination port number
- Layer 3 protocol field
- Type of service
These same attributes that define a given set of IP packets as a flow make up the core metadata (information about the flow rather than the information that’s actually in the packets) that is included in the NetFlow “flow record” for that flow. Each time a new unidirectional IP traffic flow starts traversing a device a new NetFlow flow record is created and tracked in the device’s on-board NetFlow cache. And of course, the function of NetFlow is to export this network flow data to a NetFlow collector for storage and analysis.
A flow record is ready for export when one of the following is true about the corresponding flow:
- The flow is inactive (no new packets received) for a duration defined in a timer. Timers are configurable but defaults are typically used.
- The flow is long-lived (active) but lasts for longer than the active timer (e.g., a long FTP download).
- A TCP flag (i.e., FIN, RST) indicates that the flow is terminated.
At export, the flow record is encapsulated in a UDP datagram and sent to a NetFlow collector that is typically external to a NetFlow-enabled device. The collector collects and stores the flow metadata in a record format that is defined by the protocol. The data points found in a NetFlow record typically include:
- Source and destination IP address
- Source and destination TCP/User Datagram Protocol (UDP) ports
- Type of service (ToS)
- Packet and byte counts
- Start and end timestamps
- Input and output interface numbers
- TCP flags and encapsulated protocol (TCP/UDP)
- BGP routing information (next-hop address, source autonomous system (AS) number, destination AS number, source prefix mask, destination prefix mask)
After the flow collector receives this flow record, it’s sent to the flow analyzer to extract the data for insights. Network admins use these insights to identify security threats and traffic patterns such as application performance and bandwidth usage that serve to improve the overall performance of a network.
Why use NetFlow: Key Benefits and Capabilities
By now it should be obvious that organizations can utilize flow-based analysis techniques with NetFlow to visualize traffic patterns for complete network visibility. This makes the responsibilities of the network admins considerably easier in many ways.
Some of the key benefits include:
- Optimized bandwidth usage and capacity planning
- Deeper network visibility
- Enabling of root cause analytics for application performance slowdowns
- Identification and investigation of security threats on the network
In addition, it has multiple use cases for network monitoring:
- Network, user, and application monitoring
- Capacity planning
- Cost reporting and analysis of network usage
- Network security
- Network troubleshooting
Related Resources about NetFlow
For an excellent overview of the origins, evolution and extensibility of NetFlow, see Kentik CEO Avi Freedman’s blog posts on NetFlow, sFlow, and Flow Extensibility: Part 1 and NetFlow, sFlow, and Flow Extensibility: Part 2.
More NetFlow resources:
- The Network Traffic Analytics that Enterprises Need: Learn why traditional network monitoring tools don’t scale to meet today’s network visibility needs, why more enterprises are moving from faults & packets to flow, and how Kentik can help.
Related Kentipedia entries:
How Kentik Helps with Netflow Monitoring
The Kentik Network Intelligence Platform provides a comprehensive view of network traffic flows, including hybrid and multicloud environments. By visualizing and analyzing network flow data, Kentik enables network professionals to understand traffic patterns, identify performance issues, and enhance security. Key features include:
- Real-Time Network Flow Monitoring: Visualize and analyze flow data in real-time across all network environments.
- Advanced Network Traffic Analytics: Identify anomalies and optimize network performance with detailed flow insights.
- Unified Dashboard: Monitor network paths, performance, and security from a single interface.
Kentik offers a suite of advanced network monitoring solutions designed for today’s complex, multicloud network environments. The Kentik Network Intelligence Platform empowers network pros to monitor, run and troubleshoot all of their networks, from on-premises to the cloud. Kentik’s network intelligence solution addresses all three pillars of modern network monitoring, delivering visibility into network flow, powerful synthetic testing capabilities, and Kentik NMS, the next-generation network monitoring system.
To see how Kentik can bring the benefits of network intelligence and observability to your organization, request a demo or sign up for a free trial today.
