Fasten your seat-belts, this one is a big deal. It’s the first release within a bigger plan for end-to-end visibility of your traffic, which is a holy grail objective of flow data reconciliation. What do we mean by “end-to-end visibility”? It means an easy way to figure out what volumes of traffic are flowing in and out of your network, from any source to any destination network.
A great example of this is assessing potential peer or transit prospects. How many times have you had to toggle between multiple spreadsheets that contain only approximations of traffic to or from various ASNs, getting bogged down in hacked, convoluted excel formulas, all in order to guess the ROI of what should be a simple decision?
What about trying to figure out how much traffic from a peer is being routed locally versus over more costly long-haul links? You need to able to figure out precisely at the site and device level — and at the interface level in the future — the traffic flowing between network entry and exit points.
It turns out that the sophistication of flow consolidation and reconciliation needed to achieve this task is beyond home-grown tools, data infrastructure, and software engineering capacities of many network engineering teams. And for good reason. It’s a hard problem.
Introducing two newly added destination dimensions (fanfare, please):
How do you use these? Let’s say you are a transit provider. You move packets from content providers to eyeball ISPs, and carry them over a costly global backbone. You want to look at the traffic you’re exchanging with one of the major content providers like Google, and see where it comes in, and where it comes out of your network.
Let’s further assume that you run a well organized network, so you indicate within your Interface description nomenclatures any interconnections with Google. This means you can easily include these interconnects with a simple filter. For example:
BTW, if you know that you’re going to be looking at these often, you can also make yourself a nice Saved Filter (see below) and just apply it any time you need it.
Then you can use that saved filter in any Data Explorer query you’re working on.
So here’s what you want to look at, in sequence:
Using Kentik Detect’s handy new dimensions you can now answer this question with the following query:
For a useful visualization, select the Sankey display type:
Looking at the generated Sankey diagram (above), you can now instantly see what traffic is flowing between the entry Site and the Ultimate Exit site, and which eyeball networks are reached. What you would typically do at this point is look at where transport is the most expensive or least performant between your Entry Site and Ultimate Exit site and optimize for either of them.
In the above Sankey chart, you can see that you’re shipping a lot of traffic from Frankfurt to Marseilles. So a few questions come to mind that can be explored further in Kentik Detect:
You can’t even start this ROI exploration when you’re stuck in spreadsheet hell. Stay tuned, because there’s a lot more coming over the next few months in this arena.
Our Custom Dimension infrastructure has been upgraded, allowing us to upgrade our default provisioning rules:
Every now and then we will preview an upcoming feature. We also believe that occasionally there is value in releasing an early/crude version of a feature-set in order to get early feedback from our users, which we can then use to quickly iterate until we arrive at the feature that users really want. In the case of User-Based Filtering (see Knowledge Base article), we are previewing here a feature that we have decided to introduce as an early release.
Kentik Detect currently supports two different user levels: Member and Admin. User-Based Filtering allows an Admin user to apply a user filter that restricts the data available to a Member user. The underlying idea is for Admins to be able to grant (very) granular rights on what specific Members are allowed to see and/or query.
Admin users can set up a user filter on the Users page (Admin » Users).
A user-based filter is composed like a filter in the Filters pane of the Data Explorer sidebar. Once a user filter is associated with a given user, these filters are systematically appended (ANDed) with any query run by that user, including:
One use case example is allowing only certain users to query flows from backbone routers, as shown in the following screenshot:
Another example, shown below, allows certain users to query only flows for CUSTOMER interfaces on ‘Ashburn DC3’ and ‘Ashburn DC4’:
As explained above, we have released the minimum amount of functionality for this feature, and hope to leverage the feedback of interested users to iterate it.
Some open questions we have for this feature include:
Please let us know your feedback on email@example.com. Is this a useful feature that you would like to rely on? What should the next iteration look like?
This is one for the nerdier users out there. As you may know, our ingest platform includes smart ways of re-sampling flows exported by your devices to match your contracted FPS. We’ve been improving this functionality quite a lot recently. Our goal is to resample accurately and keep the resampling-bound distortion as close to zero as possible.
In order to keep our engineering work accurate, we actually had to add Sampling Rate to our available dimensions, metrics, and filters, as shown in the images below:
This could come in handy on your end when debugging potential flow sampling misconfigurations.
As we see our customer’s usage of the Data Explorer evolve we often throw in additional convenience features that we think will streamline the overall user experience. This time around, we’ve added a couple of convenience tweaks, both of which are geared towards making queries return faster by allowing users to optionally skip certain processes.
With reverse DNS enabled:
With reverse DNS disabled:
We’ve just added the capability for you to ship alert notifications to good ole Syslog infrastructure. This has been a recurring ask since we’ve released v3 of our Anomaly Detection and Alerting platform. Your voice has been heard! Syslog alerting works in the same way than the JSON Webhook feature does, which is by offering a new type of notification channel, aptly named “Syslog.”
When configuring a threshold in an Alert Policy (Alerting → Alert Policies → edit a policy), you will notice that in addition to the existing Email and JSON webhook options a new entry has been added to the Create Notification Channel button. You can tune all of the config knobs when you create the channel, including Port, UDP/TCP transport, Syslog Severity, and Syslog Facility.
We’ve just added new support in our Alert Policies for: