Flow-based network monitoring relies on collecting information about packet flows (i.e., a sequence of related packets) as they traverse routers, switches, load balancers, ADCs, network visibility switches, and other devices. Network elements that support traffic monitoring protocols such as NetFlow and sFlow extract critical details from packet flows, like source, destination, byte and packet counts, and other attributes. This “metadata” is then streamed to “flow collectors” so that it can be stored and analyzed to produce network-wide views of bandwidth usage, identify abnormal traffic patterns that could represent possible security threats, and zero in on congestion.
Flow-based monitoring provides significant advantages over other network monitoring methods. It can capture substantially more detail about network traffic composition than SNMP metrics, which show only total traffic volume. It’s also significantly less expensive and easier to deploy than raw packet capture. And since flow generation capability is now a built-in feature of almost all modern network equipment, it provides a pervasive monitoring footprint across the entire network.
NetFlow is the trade name known for a session sampling flow protocol invented by Cisco Systems that is widely used in the networking industry. In networking terms, a “flow” is a unidirectional set of packets sharing common attributes such as source and destination IP, source and destination ports, IP protocol, and type of service. NetFlow statefully tracks flows (or sessions), aggregating packets associated with each flow into flow records, which are then exported. NetFlow records can be generated based on every packet (unsampled or 1:1 mode) or based on packet sampling. Sampling is typically employed to reduce the volume of flow records exported from each network device.
The most commonly deployed NetFlow versions are v5 and v9, with the main difference being that v9 supports “templates” which allow flexibility over the fields contained in each flow record, while v5 flow records contain a fixed list of fields. IPFIX is an IETF standards-based protocol that is largely modeled on NetFlow v9. Other NetFlow variants supported by various network equipment vendors include J-Flow (NetFlow v5 variant) from Juniper, cflowd from Alcatel-Lucent and Juniper, NetStream from 3Com/Huawei and RFlow from Ericsson.
sFlow, short for “sampled flow,” is a packet sampling protocol created by InMon Corporation that has seen broad network industry adoption. This includes network equipment vendors that already support NetFlow including Cisco, along with router vendors like Brocade (now Extreme) and in many switching products including those from Juniper and Arista. It is used to record statistical, infrastructure, routing, and other metadata about traffic traversing an sFlow-enabled network device. sFlow doesn’t statefully track flows, as is the case with NetFlow, but instead exports a statistical sampling of individual packet headers, along with the first 64 or 128 bytes of the packet payload. sFlow can also export metrics derived from time-based sampling of network interfaces or other system statistics.
sFlow does not support unsampled mode like NetFlow does, nor does it timestamp traffic flows. It relies on accurate and reliable statistical sampling methods for documenting flows, thereby reducing the amount of flow information that ultimately needs processing and analysis.
Here are some key differences between NetFlow and sFlow:
|Flow Protocol Capabilities||NetFlow||sFlow|
|Packet Capturing||Not Supported||Partially Supported|
|Interface Counters||Not Supported||Fully Supported|
|IP/ICMP/UDP/TCP||Fully Supported||Fully Supported|
|Ethernet/802.3||Not Supported||Fully Supported|
|Packet Headers||Specific Fields Only||Fully Supported|
|IPX||Not Supported||Fully Supported|
|Appletalk||Not Supported||Fully Supported|
|Input/Output Interfaces||Fully Supported||Fully Supported|
|Input/Output VLAN||Some Vendors||Fully Supported|
|Source & Destination Subnet//Prefix||Fully Supported||Fully Supported|
Network monitoring and analysis are obviously of great value to network operators. Network flow data—because it carries additional information over technologies such as raw packet capture or SNMP—enables deeper analysis. Applications of NetFlow and sFlow enable a wide variety of network monitoring, application monitoring, network planning, network troubleshooting and network security applications, such as:
Both NetFlow and sFlow use sampling techniques. With sFlow it’s required, and with NetFlow it’s optional. There is a long-running discussion in the industry about the accuracy of data and insight derived from sampling-based flow protocols. With sampling enabled, network devices generate flow records from a 1-in-N subset of traffic packets. As the variable N increases, flow records derived from the samples may become less representative of the actual traffic, especially for low-volume flows over short time windows. In the real world, how high can N be while still enabling us to see a given traffic subset that is a relatively small part of the overall volume? What is the impact on the accuracy of flow record analysis? Testing performed by Kentik indicates that even at sampling rates as low as 1:10000, lower bandwidth traffic patterns are discernible even in high throughput networks.
So which is better? In many ways, sFlow provides a more comprehensive picture of network traffic, because it includes the full packet header, from which any field can be extracted, where NetFlow typically contains only a subset of those fields. sFlow also typically places less load on network devices. In many cases, the choice is not up to the user though, because most networking gear supports only one or the other. Many networks contain gear from multiple vendors, and the key question for the network operator then becomes — does my network monitoring platform support all of the flow protocols that my network generates? This is an important consideration to ensure there are no visibility gaps across the infrastructure.
Contemporary big data network monitoring platforms, such as Kentik, are well suited to cope with network monitoring challenges. Kentik’s adoption of a big data architecture is at the core of their network flow-based monitoring platform, which supports NetFlow, IPFIX, and sFlow protocols. This allows Kentik to correlate high volumes of flow data records for customers, eliminating network monitoring accuracy concerns. Big data is not only about handling large volumes of data, but also letting network operations staff navigate through and explore that data very quickly.
For an excellent overview of the origins, evolution and extensibility of NetFlow and sFlow, see Kentik CEO Avi Freedman’s blog posts on NetFlow, sFlow, and Flow Extensibility: Part 1 and NetFlow, sFlow, and Flow Extensibility: Part 2 and, more recently, his overview of network telemetry types.