Kentik Blog: Network Engineering

AWS VPC flow Logs and Azure NSG flow Logs offer network traffic visibility with different scopes and formats, but both are essential for multi-cloud network management and security. Unified network observability solutions analyze both in one place to provide comprehensive insights across clouds.

The 2024 EMA Network Megatrends surveyed hundreds of IT professionals about their approach to managing, monitoring, and troubleshooting their networks. In this post, we examine the report’s findings to learn the business and technology trends shaping network operations strategy.

Modern networking relies on the public internet, which heavily uses flow-based load balancing to optimize network traffic. However, the most common network tracing tool known to engineers, traceroute, can’t accurately map load-balanced topologies. Paris traceroute was developed to solve the problem of inferring a load-balanced topology, especially over the public internet, and help engineers troubleshoot network activity over complex networks we don’t own or manage.

By automating capacity planning for IP networks, we can achieve cost reduction, enhanced accuracy, and better scalability. This process requires us to collect data, build predictive models, define optimization objectives, design decision algorithms, and carry out consistent monitoring and adjustment. However, the initial investment is large and the result will still require human oversight.

In this post, we discuss the crucial differences between resilience versus redundancy in networking. Learn how to optimize your network for seamless performance.

True observability requires visibility into both the application and network layers. For companies reliant on multi-zonal cloud networks, the days of NetOps existing as a team siloed away from application developers are over.

eBPF is a powerful technical framework to see every interaction between an application and the Linux kernel it relies on. eBPF allows us to get granular visibility into network activity, resource utilization, file access, and much more. It has become a primary method for observability of our applications on premises and in the cloud. In this post, we’ll explore in-depth how eBPF works, its use cases, and how we can use it today specifically for container monitoring.

Resiliency is a network’s ability to recover and maintain its performance despite failures or disruptions, and redundancy is the duplication of critical components or functions to ensure continuous operation in case of failure. But how do the two concepts interact? Is doubling up on capacity and devices always needed to keep the service levels up?

What does it mean to build a successful networking team? Is it hiring a team of CCIEs? Is it making sure candidates know public cloud inside and out? Or maybe it’s making sure candidates have only the most sophisticated project experience on their resume. In this post, we’ll discuss what a successful networking team looks like and what characteristics we should look for in candidates.

The advent of various network abstractions has meant many day-to-day networking tasks normally done by network engineers are now done by other teams. What’s left for many networking experts is the remaining high-level design and troubleshooting. In this post, Phil Gervasi unpacks why this change is happening and what it means for network engineers.

A packet capture is a great option for troubleshooting network issues and performing digital forensics, but is it a good option for always-on visibility considering flow data gives us the vast majority of the information we need for normal network operations?

BGP is a critical network protocol, and yet, BGP is often not monitored. BGP issues that go unchecked can turn into major problems. This post explains how Kentik can help you to easily monitor BGP and catch critical issues quickly.

Last week I had the honor to participate in the PTC 2021 conference. Held in Hawaii every January, PTC’s annual conference is the Pacific Rim’s premier telecommunications event. Although this year’s conference was all virtual (no boondoggles to Honolulu!), it was no less important as the theme this year was New Realities. In the following blog post, I summarize what I presented in my PTC panel entitled Strategies to Meet Network Needs.

Kentik’s Hybrid Map provides the industry’s first solution to visualize and manage interactions across public/private clouds, on-prem networks, SaaS apps, and other critical workloads as a means of delivering compelling, actionable intelligence. Product expert Dan Rohan has an overview.

Today we published a report on “The New Normals of Network Operations in 2020.” Based on a survey of 220 networking professionals, our report aims to better understand the challenges this community faces personally and professionally as more companies and individuals taking their worlds almost entirely online.

If you missed Akamai, Uber and Verizon Media talking about networking during COVID-19, read our recap of Kentik’s recent virtual panel. In this post, we share details from the conversation, including observed spikes in traffic growth, network capacity challenges, BC/DR plans, the internet infrastructure supply chain, and more.

The Gartner Market Guide for Network Performance Monitoring and Diagnostics recently published and revealed new network challenges for buyers. In this post, we take a deeper look at the challenges, as well as opportunities for NPMD investment.

Google is investing and innovating in SD-WAN. In this post, Kentik CTO Jonah Kowall highlights what Google has been up to and how Google’s SD-WAN work can apply to the typical organization.

Kentik recently hosted a virtual panel with network leaders from Dropbox, Equinix, Netflix and Zoom and discussed how they are scaling to accommodate the unprecedented growth in network traffic during COVID-19. In this post, we highlight takeaways from the event.

When more of the workforce shifts to working remotely, it puts new and different strains on the infrastructure across different parts of the network. In this post, we discuss strategies for managing surges in network traffic coming from remote employees and share information on how Kentik can help.

The most complex point of today’s networks is the edge, where there are more protocols, diverse traffic, and security exposure. The network edge is also a place where Kentik provides high value. In this post, we discuss how to implement Kentik in your data center.

Gartner recently published its 2019 version of the Market Guide for AIOps Platforms. In this post, we examine our understanding of the report and discuss how Kentik’s domain-centric AIOps platform is built from the ground up for network professionals.

“My advice to anyone who is considering Kentik is to get into it. Get a trial, import your data, and start playing with it,” says Rick Carter, head of networks at Superloop and Kentik user. In this vlog, Rick shares how Kentik helps Superloop bring connectivity to APAC by uncovering important network and business insights.

Packet is focused on automating single-tenant bare metal compute infrastructure. On a mission to enable the world’s companies with a competitive advantage of infrastructure, Packet turned to Kentik for network visibility and insights.

Kentik CTO Jonah Kowall highlights challenges and opportunites in network automation and describes how Kentik is leading the way in providing next-generation solutions for automation, notification, advanced API integrations with telemetry, and more. “Every organization has an automation goal, and it’s no doubt that network automation is not only essential to avoid costly outages, but also helps organizations scale without putting people in the work path… The problem is that every organization has a storied history of automation tools, meaning we already have at least a dozen of them in our organizations across various silos and stacks, some of which are commercial and some are open source.”

Learn how enterprise video communications leader, Zoom, uses Kentik for network visibility, performance, peering analytics and improved customer support. Zoom’s Alex Guerrero, senior manager of SaaS operations, and Mike Leis, senior network engineer, share how they use Kentik to help Zoom deliver “frictionless meetings.”

At one point, data was called “the new oil.” While that’s certainly an apt description for the insights we can extract from data, most organizations today are finding that new data repositories and “data lakes” often don’t provide the expected benefits due to the analytics challenge. CTO Jonah Kowall explains how advanced data enrichment techniques, leveraging AIOps technologies, can make the promise of data analysis a reality.

CTO Jonah Kowall introduces Kentik’s AIOps solution for network management, explaining the need for evolution and innovation in the network performance monitoring and diagnostics market and providing an overview of the components of Kentik’s revolutionary platform.

Kentik’s Jim Meehan and Crystal Li report on what they learned from speaking with enterprise network professionals at Cisco Live 2019, and what those attendees expect from a truly modern network analytics solution.

The most commonly used network monitoring tools in enterprises were created specifically to handle only the most basic faults with traditional network devices. CTO Jonah Kowall explains why these tools don’t scale to meet today’s network visibility needs, why more enterprises are moving from faults & packets to flow, and how Kentik can help.

Learn how to maintain multi-cloud visibility within hybrid and multi-cloud architectures, as well as the requirements for modern cloud monitoring tools.

As enterprises integrate their networks with their public cloud strategies, several best practices for network teams are emerging. In this guest blog post, Enterprise Management Associates analyst Shamus McGillicuddy dives into his recent cloud networking research to discuss several strategies that can improve the chances of a successful cloud networking initiative.

In this post, we provide an overview of a new research report from EMA analyst Shamus McGillicuddy. The report is based on the responses of 250 enterprise IT networking professionals who note challenges and key technology requirements for hybrid and multi-cloud networking.

In this blog post, we provide a starting point for understanding the networking model behind Kubernetes and how to make Kubernetes networking simpler and more efficient.

Broadband providers must manage high infrastructure costs on a price-per-bit basis. Visibility into their network performance and security is critical. In this post, we dig into how that led Viasat, a global provider of high-speed satellite broadband services and secure networking systems, to tap Kentik for help.

Building a cloud application is like building a house. If you don’t at least acknowledge the industry’s best practices, it may all come tumbling down. Here we look at AWS’ Well-Architected Framework as a good starting point for building effective cloud applications and outline why having the right tools in place can make all the difference.

Today we released a new report: “AWS Cloud Adoption, Visibility & Management.” The report compiles an analysis based on a survey of 310 executive and technical-level attendees at the recent AWS user conference. Simply put, we found: It’s a multi-cloud, cost-containment world.

In a new report, 2019 Trends in Cloud Transformation, 451 Research analysts dig into seven trends happening amidst the move to the cloud, as well as recommendations, and clear winners and losers for each of the trends. In this post, we provide an overview and a licensed copy of the report for you.

There are five network-related cloud deployment mistakes that you might not be aware of, but that can negate the cloud benefits you’re hoping to achieve. In this post, we provide an overview of each mistake and a guide for avoiding them all.

Music streaming service Pandora recently announced its migration to Google Cloud Platform (GCP). For the NetOps and SecOps teams behind the migration, we know cloud visibility is now more important than ever. That’s why we caught up with Pandora’s James Kelty to tell us how the company plans to maintain visibility across its infrastructure, including GCP, with help from Kentik.

At recent AWS re:Invent, we heard many attendees talking about the push for cloud-native to foster innovation and speed up development. In this post, we take a deeper dive on what it means to be cloud-native, as well as the challenges and how to overcome them.

Cloud providers take away the huge overhead of building, maintaining, and upgrading physical infrastructure. However, many system operators, including NetOps, SREs, and SecOps teams, are facing a huge visibility challenge. Here we talk about how VPC flow logs can help.

To some, moving to the cloud is like a trick. But to others, it’s a real treat. So in the spirit of Halloween, here’s a blog post to break down two of the spookiest (or at least the most common) cloud myths we’ve heard of late.

In this post we look at the difference between NetFlow and sFlow and how network operators can support all of the flow protocols that their networks generate.

With increased business reliance on internet connectivity, the network world has and will continue to get increasingly complex. In this post, we dig into the key findings from our new “State of Network Management in 2018” report. We also discuss why we’re just in the early stages of how our industry will need to transform.

Managing quality of service and service-level agreement (SLAs) is becoming more complex for service providers. In this post, we look at how and why enterprise cloud services and application usage is driving service providers to rethink service assurance metrics. We also discuss why network-based analytics is critical to satisfying service assurance needs.

NetFlow offers a great way to preserve highly useful traffic analysis and troubleshooting details without needing to perform full packet capture. In this post, we look at how NetFlow monitoring solutions quickly evolved as commercialized product offerings and discuss how cloud and big data improve NetFlow analysis.

Digital transformation is not for the faint of heart. In this post, ACG Analyst Stephen Collins discusses why it’s critical for ITOps, NetOPs, SecOPs and DevOps teams to make sure they have the right stuff and are properly equipped for the network visibility challenges they face.

Silos in enterprise IT organizations can inhibit cross-functional synergies, leading to inefficiencies, higher costs and unacceptable delays for detecting and repairing problems. In this post, ACG analyst Stephen Collins examines how IT managers can start planning for now as the first step in moving from silos to synergy.

In this post, we look at the best practices for an effective capacity planning solution that ensures optimal network performance and visibility.

Network performance and network security are increasingly becoming two sides of the same coin. Consequently, enterprise network operations teams are stepping up collaboration with their counterparts in the security group. In this post, EMA analyst Shamus McGillicuddy outlines these efforts based on his latest research.

5G is marching towards commercialization. In this post, we look at the benefits and discuss why network monitoring for performance and security considerations are even more crucial to the operation of hybrid enterprise networks that incorporate 5G network segments.

At that ONUG Spring 2018 event, ACG analyst Stephen Collins moderated a panel discussion on re-tooling IT operations with machine learning and AI. The panelists provided a view “from the trenches.” In this post, Collins shares insights into how panelists’ organizations are applying ML and AI today, each in different operational domains, but with a common theme of overcoming the challenge of managing operations at scale.

Modern enterprise networks are becoming more dynamic and complex, which poses significant challenges for today’s IT leaders. In this post, data center and IT service provider phoenixNAP discusses how Kentik Detect helps overcome network visibility challenges.

Predictive analytics has improved over the past few years, benefiting from advances in AI and related fields. In this post, we look at how predictive analytics can be used to help network operations. We also dig into the limitations and how the accuracy of the predictions depends heavily on the quality the data collected.

Forward-thinking IT managers are already embracing big data-powered SaaS solutions for application and performance monitoring. If hybrid multi-cloud and hyperscale application infrastructure are in your future, ACG Analyst Stephen Collins’ advice for performance monitoring is “go big or go home.”

While we’re still in the opening phase of the hybrid multi-cloud chess game, ACG analyst Stephen Collins takes a look at what’s ahead. In this post, he digs into what enterprises embracing the cloud can expect in the complex endgame posing many new technical challenges for IT managers.

While data networks are pervasive in every modern digital organization, there are few other industries that rely on them more than the Financial Services Industry. In this blog post, we dig into the challenges and highlight the opportunities for network visibility in FinServ.

Fixing a persistent Internet underlay problem might be as simple as using a higher bandwidth connection or as complex as choosing the right peering and transit networks for specific applications and destination cloud services. In this blog, ACG analyst Stephen Collins advices that to make the best-informed decision about how to proceed, IT managers need to be equipped with tools that enable them to fully diagnose the nature of Internet underlay connectivity problems.

Traditional monitoring tools for managing application performance in private networks are not well-suited to ensuring the performance, reliability and security of SaaS applications. In this post, ACG analyst Stephen Collins makes the case for why enterprise IT managers need to employ a new generation of network visibility and big data analytics tools designed for the vast scale of the Internet.

College, university and K-12 networking and IT teams who manage and monitor campus networks are faced with big challenges today. In this post, we take a deeper look at the challenges and provide requirements for a cost-effective network monitoring solution.

Sankey diagrams were first invented by Captain Matthew Sankey in 1898. Since then they have been adopted in a number of industries such as energy and manufacturing. In this post, we will take a look at how they can be used to represent the relationships within network data.

Kentik Detect now incorporates IP reputation data from Spamhaus, enabling users to identify infected or compromised hosts. In this post we look at the types of threat feed information we use from Spamhaus, and then dive into how to use that information to reveal problem hosts on an ad hoc basis, to generate scheduled reports about infections, and to set up alerting when your network is found to have carried compromised traffic.

While defining digital transformation strategies is a valuable exercise for C-level executives, CIOs and IT managers also need to adopt a pragmatic and more tactical approach to “going digital.” That starts with acquiring tools and building the new skills needed to ensure business success and profitability in the face of digital disruption. In this post, ACG Principal Analyst Stephen Collins looks at how to manage it all.

A new EMA report on network analytics is full of interesting takeaways, from reasons for deployment to use cases after analytics are up and running. In this post, we look specifically at the findings around adoption to see why and how organizations are leveraging network analytics.

CDNs have been around for years, but they’ve gained new importance with the rise of video streaming services like Netflix and Hulu. As traffic from those sites soars, CDNs introduce new challenges for network operations teams at both service providers and enterprises. Kentik Detect’s new CDN Attribution makes identifying and tracking CDN traffic a whole lot easier. In this blog, we provide examples of how companies can implement this functionality.

As IoT adoption continues, enterprises are finding a massive increase in the number of devices and the data volumes generated by these devices on their networks. Here’s how enterprises can use network monitoring tools for enhanced visibility into complex networks.

From telcos, to financial services, to tech companies, we asked 30 of our peers one question: What are your 2018 networking predictions? Yes, it’s a broad question. But respondents (hailing from network, data center, and security operations teams) surfaced five main predictions for the year ahead.

What brands of network devices are Kentik customers using? Where does their international traffic come from and go to? What’s the current norm for packet sizes and internet traffic protocols? Drawing on Kentik Detect’s ability to see and analyze network traffic, this post shares some intriguing factoids, and it sheds light on some of the insights about your own network traffic that await you as a Kentik customer.

Advances in open source software packages for big data have made Do-It-Yourself (DIY) approaches to Network Flow Analyzers attractive. However, careful analysis of all the pros and cons needs to be completed before jumping in. In this post, we look at the hidden pitfalls and costs of the DIY approach.

Media reports tell us that Cyber Monday marked a single-day record for revenue from online shopping. We can assume that those sales correlated with a general spike in network utilization, but from a management and planning perspective we might want to go deeper, exploring the when and where of traffic patterns to specific sites. In this post we use Kentik Detect to see what can be learned from a deeper dive into holiday traffic.

As last week’s misconfigured BGP routes from backbone provider Level 3 caused Internet outages across the nation, the monitoring and troubleshooting capabilities of Kentik Detect enabled us to identify the most-affected providers and assess the performance impact on our own customers. In this post we show how we did it and how our new ability to alert on performance metrics will make it even easier for Kentik customers to respond rapidly to similar incidents in the future.

Managing network capacity can be a tough job for Network Operations teams at Service Providers and Enterprise IT. Most legacy tools can’t show traffic traversing the network and trend that data over time. In this post, we’ll look at how Kentik Detect changes all that with new Dashboards, Analytics, and Alerts that enable fast, easy planning for capacity changes and upgrades.

At Kentik, we built Kentik Detect, our production SaaS platform, on a microservices architecture. We also use Kentik for monitoring our own infrastructure. Drawing on a variety of real-life incidents as examples, this post looks at how the alerts we get — and the details that we’re able to see when we drill down deep into the data — enable us to rapidly troubleshoot and resolve network-related issues.

As security threats grow more ominous, security procedures grow more onerous, which can be a drag on productivity. In this post we look at how Kentik’s single sign-on (SSO) implementation enables users to maintain security without constantly entering authentication credentials. Check out this walk-through of the SSO setup and login process to enable your users to access Kentik Detect with the same SSO services they use for other applications.

In today’s world of heterogeneous environments and distributed systems, APIs drive synergistic innovation, creating a whole that’s more powerful than the parts. Even in networking, where the CLI rules, APIs are now indispensable. At Kentik, APIs have been integral to our platform from the outset. In this post we look at how partners and customers are expanding the capabilities of their systems by combining Kentik with external tools.

In our latest post on Interface Classification, we look beyond what it is and how it works to why it’s useful, illustrated with a few use cases that demonstrate its practical value. By segmenting traffic based on interface characteristics (Connectivity Type and Network Boundary), you’ll be able to easily see and export valuable intelligence related to the cost and ROI of carrying a given customer’s traffic.

With much of the country looking skyward during the solar eclipse, you might wonder how much of an effect there was on network traffic. Was there a drastic drop as millions of watchers were briefly uncoupled from their screens? Or was that offset by a massive jump in live streaming and photo uploads? In this post we report on what we found using forensic analytics in Kentik Detect to slice traffic based on how and where usage patterns changed during the event.

Domain Name Server (DNS) is often overlooked, but it’s one of the most critical pieces of Internet infrastructure. As driven home by last October’s crippling DDoS attack against Dyn, the web can’t function unless DNS resolves hostnames to their underlying IP addresses. In this post we look at how combining Kentik’s software host agent with Dashboards in Kentik gives you the tools you need to ensure DNS availability and performance.

Kentik addresses the day-to-day challenges of network operations, but our unique big network data platform also generates valuable business insights. A great example of this duality is our new Interface Classification feature, which streamlines an otherwise-tedious technical task while also giving sales teams a real competitive advantage. In this post we look at what it can do, how we’ve implemented it, and how to get started.

Major cyber-security incidents keep on coming, the latest being the theft from HBO of 1.5 terabytes of private data. We often frame Kentik Detect’s advanced anomaly detection and alerting system in terms of defense against DDoS attacks, but large-scale transfer of data from private servers to unfamiliar destinations also creates anomalous traffic. In this post we look at several ways to configure our alerting system to see breaches like the attack on HBO.

With BGP and NetFlow correlated into a unified datastore, Kentik Detect’s advanced analytics provide valuable insights for both engineering and sales. In this post we look into a fairly recent addition to Kentik Detect, Route Traffic Analytics. Especially useful for capacity planners and peering coordinators, RTA makes it easy to see how many unique routes are represented in a given percent of your traffic, which indicates the route capacity needed in your edge routers.

Can BGP routing tables provide actionable insights for both engineering and sales? Kentik Detect correlates BGP with flow records like NetFlow to deliver advanced analytics that unlock valuable knowledge hiding in your routes. In this post, we look at our Peering Analytics feature, which lets you see whether your traffic is taking the most cost-effective and performant routes to get where it’s going, including who you should be peering with to reduce transit costs.

Among Kentik Detect’s unique features is the fact that it’s a high-performance network visibility solution that’s available as a SaaS. Naturally, data security in the cloud can be an initial concern for many customers, but most end up opting for SaaS deployment. In this post we look at some of the top factors to consider in making that decision, and why most customers conclude that there’s no risk to taking advantage of Kentik Detect as a SaaS.

What do summer blockbusters have to do with network operations? As utilization explodes and legacy tools stagnate, keeping a network secure and performant can feel like a struggle against evil forces. In this post we look at network operations as a hero’s journey, complete with the traditional three acts that shape most gripping tales. Can networks be rescued from the dangers and drudgery of archaic tools? Bring popcorn…

Operating a network means staying on top of constant changes in traffic patterns. With legacy network monitoring tools, you often can’t see these changes as they happen. Instead you need a comprehensive visibility solution that includes real-time anomaly detection. Kentik Detect fits the bill with a policy-based alerting system that continuously evaluates incoming flow data. This post provides an overview of system features and configuration.

As one of 2017’s hottest networking technologies, SD-WAN is generating a lot of buzz, including at last week’s Cisco Live. But as enterprises rely on SD-WAN to enable Internet-connected services — thereby bypassing Carrier MPLS charges — they face unfamiliar challenges related to the security and availability of remote sites. In this post we take a look at these new threats and how Kentik Detect helps protect against and respond to attacks.

Obsolete architectures for NetFlow analytics may seem merely quaint and old-fashioned, but the harm they can do to your network is no fairy tale. Without real-time, at-scale access to unsummarized traffic data, you can’t fully protect your network from hazards like attacks, performance issues, and excess transit costs. In this post we compare three database approaches to assess the impact of system architecture on network visibility.

NetFlow data has a lot to tell you about the traffic across your network, but it may require significant resources to collect. That’s why many network managers choose to collect flow data on a sampled subset of total traffic. In this post we look at some testing we did here at Kentik to determine if sampling prevents us from seeing low-volume traffic flows in the context of high overall traffic volume.

Telecom and mobile operators are clear on both the need and the opportunity to apply big data for advanced operational analytics. But when it comes to being data driven, many telecoms are still a work in progress. In this post we look at the state of this transformation, and how cloud-aware big data solutions enable telecoms to escape the constraints of legacy appliance-based network analytics.

In higher education, embracing the cloud enhances your ability to achieve successful outcomes for students, researchers, and the organization as a whole. But just as in business, this digital transformation can succeed only if it’s anchored by modern network visibility. In this post we look at the network as more than mere plumbing, identifying how big data network intelligence helps realize high-priority educational goals.

Stuck with piles of siloed tools, today’s network teams struggle to piece together both the big picture and the actionable insights buried in inconsistent UIs and fragmented datasets. The result is subpar performance for both networks and network teams. In this post we look at the true cost of legacy tools, and how Kentik Detect frees you from this obsolete paradigm with a unified, scalable, real-time solution built on the power of big data.

Large or small, all ISPs share the imperative to stay competitive and profitable. To do that in today’s environment, they need traffic visibility they can’t get from legacy network tools. Taking their lead from the world’s most-successful web-scale enterprises, ISPs have much to gain from big data network and business intelligence, so in this post we look at ISP use cases and how Kentik Detect’s SaaS model puts key capabilities within easy reach.

SDN holds lots of promise, but it’s practical applications have so far been limited to discrete use cases like customer provisioning or service scaling. The long-term goal is true dynamic control, but that requires comprehensive traffic intelligence in real time at full scale. As our customers are discovering, Kentik Detect’s traffic visibility, anomaly detection, and extensive APIs make it an ideal source for actionable traffic data that can drive network automation.

Without package tracking, FedEx wouldn’t know how directly a package got to its destination or how to improve service and efficiency. 25 years into the commercial Internet, most service providers find themselves in just that situation, with no easy way to tell where an individual customer’s traffic exited the network. With Kentik Detect’s new Ultimate Exit feature, those days are over. Learn how Kentik’s per-customer traffic breakdown gives providers a competitive edge.