Big Data Stats Reveal Industry Trends

Roughly 100 billion flow records each and every day. That's how much flow data is ingested by Kentik Data Engine (KDE), the distributed big data backend that powers Kentik Detect®. It's also just one of the many interesting statistics that we run across as we operate our SaaS platform for network traffic analytics. As the year draws to a close, we thought we'd share some of those fascinating tidbits with you, our loyal readers. Needless to say, everything presented below comes from customers that have granted permission to use their customer data. The data has been aggregated and anonymized to protect data privacy, which we've built into our platform and is an issue that we take very seriously (see our FAQ web page for more detail).

Network Devices: Which Brands are Trending?

Cisco Systems is by far the market leader in the overall IT Networking sector. So it's interesting to see that Juniper Networks, at 49.6%, is the leading maker of routers and switches sending flow to Kentik Detect. On the other hand, given Arista's recent market traction, it's not surprising to see them in the number-three spot. And how about the fact that Linux OSes show up in our Top 10 list as well? Maybe the “white box” trend really is taking off?

International Traffic: Where To, and From?

The graphs below are both based on flow record from Kentik customers for traffic whose source or destination is outside the United States. At left we see the top 10 destinations, by country, for traffic that is leaving the United States. Since most of the network operators in the US have a network POP in Canada (represented on the graph as CA), it makes sense that our northern neighbor would be in the number-one spot, with 18% of the traffic. Canada is closely followed by China (CN), Great Britain (GB), Germany (DE), and Mexico (MX).

What about the other direction: traffic that originates in foreign countries and is destined for the United States? In this case, as we can see from the graph at right above, the rankings are a bit different. Canada is down in the number 4 spot, and China is down to number 6. The top 3 spots are occupied by the Netherlands (NL), Great Britain (GB), and Germany (DE).

Packet Sizes

Here's one that's probably not too surprising: there appears to be a tendency towards larger packets on the Internet, with most of the traffic being made up of packets greater than 1000 bytes. 1400 bytes appears to be the most common, with 55% of the traffic. There are jumbo frames, those greater than 1500 bytes, among the data set, but they make up less than 1% of the traffic so they were left off the graph to make it easier to read.

IP Versions and Internet Protocols

Google maintains Internet-wide stats on the overall IPv6 adoption rate. As of this writing, those numbers show that about 17% of traffic on the Internet is IPv6. But for Kentik customers, the number is much lower. While Kentik Detect supports IPv6, only 0.2% of traffic for which flow records are sent to KDE use IPv6 (below left). Apparently most of our customers don't yet have a high volume of IPv6 traffic.

As for IP protocols, when we look at which ones our customers are most commonly seeing on their networks (above right), it's not surprising that TCP (protocol 6, for those not familiar) comes out on top, with almost 71% of the traffic. It is interesting, though, how the rest of the list comes out. UDP (protocol 17) is down in the number 5 spot with only 3.2% of the traffic, and ICMP (protocol 1) comes in right behind it with 2.7%. To be honest, I had to go look up what some of these other protocols are as I have not seen those protocol numbers in years.

Port Numbers and Services

Now let's take a look at TCP/UDP port numbers, so we can get an idea of what kind of services are in use. It's not surprising to see 443 (HTTPS) and 80 (HTTP) as the top two hitters given most Internet applications are now HTTP based. I was a little surprised to see, though, that port 443 beat out port 80. I guess that speaks to the increasing need for SSL-encrypted traffic.


The above just gives a quick taste of the interesting data points that you can see from the traffic data that Kentik customers can explore with our big data analytical tools. Are you ready to see what kinds of interesting data points are hiding in your network traffic? Get started today by scheduling a demo or starting a free trial.