NetFlow is a protocol used to collect metadata on IP traffic flows traversing a network device.
Developed by Cisco Systems, NetFlow is used to record metadata about IP traffic flows traversing a network device such as a router, switch, or host. A NetFlow-enabled device generates metadata at the interface level and sends flow data information to a flow collector, where the flow records are stored to enable network traffic analytics and management. A network operator can use NetFlow data to determine network throughput, packet loss, and traffic congestion at a specific interface level. NetFlow data also supports other network-level monitoring use cases such as DDoS detection and BGP peering.
The History of NetFlow and Versions Over Time
Netflow was initially developed by Cisco in 1995 to monitor and record all network traffic coming through their network devices. Over time, they realized that the network flow data was incredibly useful and it led to an entire branch of network monitoring that several other platforms have copied and rebranded. Now, a version of NetFlow protocols have become industry standard for optimizing network performance. This ongoing evolution of NetFlow has led to several versions over the years with different features.
While the term “NetFlow” is commonly used to refer to all types of flow records and datagrams, there are actually three important variants in regular use within live production networks:
NetFlow is the technology and term used exclusively by Cisco Systems.
IPFIX is an IETF standard flow record format that is very similar in approach and structure to NetFlow v9 (see more on NetFlow version numbering below). It is sometimes called “NetFlow v10” since IPFIX plays a key role in coalescing all NetFlow variants and equivalents as the standards process evolves the IPFIX specifications over time.
sFlow is a similar but importantly different type of flow protocol and data record standard introduced and promoted by InMon Corp. sFlow does not sample all packets like NetFlow does, nor does it timestamp traffic flows. It relies on accurate and reliable statistical sampling methods for documenting flows, thereby reducing the amount of flow information that ultimately needs processing and analysis.
Other vendor-specific flow record formats that are similar in nature to one of three most common variants listed above (in most cases these are either substitutions or very close adaptations) include:
– J-Flow from Juniper Networks, which essentially conforms to NetFlow v5;
– NetStream from 3Com/Huawei.
xFlow, while not a variant, is the generic term often used to refer collectively to all flow record variants (NetFlow, sFlow, IPFIX, J-Flow, etc.).
The fields that make up a NetFlow flow record depend on the version of NetFlow supported by the NetFlow exporter. Since the protocol was first introduced by Cisco in 1996 it has been updated numerous times in a series of backward-compatible versions, the most commonly used of which are versions 5 and 9:
v1: First implementation, now obsolete, and restricted to IPv4 (without IP mask and AS Numbers).
v2: Cisco internal version, never released.
v3: Cisco internal version, never released.
v4: Cisco internal version, never released.
v5: First commonly deployed version, available (as of 2009) on many routers from different brands, but restricted to IPv4 flows.
v6: No longer supported by Cisco.
v7: Like version 5 with a source router field.
v8: Several aggregation forms, but only for information that is already present in version 5 records
v9: Template based, available (as of 2009) on some recent routers. Mostly used to report flows like IPv6, MPLS, or even plain IPv4 with BGP nexthop. Includes added fields for security and traffic analysis use cases
Flow exporter: A NetFlow-enabled device that generates flow records and periodically exports them to a flow collector.
Flow collector: A program running on a server that is responsible for receiving, storing, and pre-processing flow records received from NetFlow-enabled devices.
Flow analyzer: An analysis application that processes flow records collected by the flow collector into reports, alerts, and other interpreted results.
Flow Records and Data
A NetFlow exporter (NetFlow-enabled device) identifies a flow as a unidirectional stream of packets having in common (at least) the following:
Input interface port
IP source address
IP destination address
Source port number
Destination port number
Layer 3 protocol field
Type of service
These same attributes that define a given set of IP packets as a flow make up the core metadata (information about the flow rather than the information that’s actually in the packets) that is included in the NetFlow “flow record” for that flow. Each time a new unidirectional IP traffic flow starts traversing a device a new NetFlow flow record is created and tracked in the device’s on-board NetFlow cache. And of course, the function of NetFlow is to export this network flow data to a NetFlow collector for storage and analysis.
A flow record is ready for export when one of the following is true about the corresponding flow:
The flow is inactive (no new packets received) for a duration defined in a timer. Timers are configurable but defaults are typically used.
The flow is long-lived (active) but lasts for longer than the active timer (e.g., a long FTP download).
A TCP flag (i.e., FIN, RST) indicates that the flow is terminated.
At export, the flow record is encapsulated in a UDP datagram and sent to a NetFlow collector that is typically external to a NetFlow-enabled device. The collector collects and stores the flow metadata in a record format that is defined by the protocol. The data points found in a NetFlow record typically include:
Source and destination IP address
Source and destination TCP/User Datagram Protocol (UDP) ports
Type of service (ToS)
Packet and byte counts
Start and end timestamps
Input and output interface numbers
TCP flags and encapsulated protocol (TCP/UDP)
BGP routing information (next-hop address, source autonomous system (AS) number, destination AS number, source prefix mask, destination prefix mask)
After the flow collector receives this flow record, it’s sent to the flow analyzer to extract the data for insights. Network admins use these insights to identify security threats and traffic patterns such as application performance and bandwidth usage that serve to improve the overall performance of a network.
Why use NetFlow: Key Benefits and Capabilities
By now it should be obvious that organizations can utilize flow-based analysis techniques with NetFlow to visualize traffic patterns for complete network visibility. This makes the responsibilities of the network admins considerably easier in many ways.
Some of the key benefits include:
Optimized bandwidth usage and capacity planning
Deeper network visibility
Enabling of root cause analytics for application performance slowdowns
Identification and investigation of security threats on the network
In addition, it has multiple use cases for network monitoring: