Kentik - Network Flow Analytics

Product Updates

October / November 2018

It’s been a busy and fruitful season for our product team, with final launch of many new features that required sophisticated design and heavy lifting. This issue’s theme? Visibility for the Cloud! We officially announced our cloud visibility solution at NFD19 and AWS re:Invent. We’ve added new integrations with AWS and introduced a new portal Admin UI that makes it easy to onboard flow logs from your cloud providers. Beyond that, there’s also new functionality and UI enhancements in the areas of data enrichment, mitigation, dashboarding, customized visualizations, and more. Now let’s dive in…

Cloud and Hybrid Visibility, Phase 1

Since Kentik’s founding, our core focus has been to provide a 360° view of the performance, composition, and paths of actual network traffic. We started with visibility for owned/on-prem infrastructure. As more and more organizations have incorporated cloud infrastructure, we’ve extended our traffic analytics to follow suit, first by integrating with GCP VPC Flow Logs this past summer, and now with support for AWS Flow Logs. By enabling a unified view across all of your infrastructure — whether it’s on-prem, hybrid-cloud, or multi-cloud — Kentik is now a complete infrastructure visibility solution.

Out-of-the-Box GCP and AWS Visibility

Need cloud visibility, but you’re not sure where to begin? No problem! To get you started we’ve now expanded our portal Library with a broad set of predefined dashboards that provide detailed views of cloud network traffic flows. You’ll find these preset dashboards under Cloud and Data Center in the Library sidebar. As shown below, there are eight dashboards for AWS and nine for GCP. Several are highlighted in the Library’s Featured Content (indicated by an orange check mark).

Library SidebarOur cloud dashboards are a joint effort of our field engineering and solution engineering teams. They’re based on real-world customer scenarios, such as investigating rejected traffic that utilizes the security action field (accepted/rejected) in AWS Flow Logs. For GCP, we’ve created dashboards that use the RTT_MSEC field in GCP flow logs, which measures application and network latency, to help you analyze the performance of your services and applications in Google Cloud.

We’ve also incorporated some of these same cloud metrics into alert policy presets that we’ve added to our Alerting Library. These new policies can identify unexpected changes in the source, destination, or composition of traffic in key parts of your cloud infrastructure.

AWS VPC Flow Logs Integration

As mentioned above, Kentik now integrates with AWS. If you have workloads running in VPCs on AWS, Kentik can now ingest the AWS VPC Flow Logs as a primary data source for analytics.

Configure Data Sources

The steps to connect AWS infrastructure and add AWS VPC Flow Logs as a data source are fairly straightforward:

  1. Create an AWS Role - In the AWS IAM console, create a new AWS role with permissions that allow access by Kentik’s AWS services.
  2. Create an S3 Bucket - In the AWS S3 console, create a bucket to which logs can be published and from which Kentik can pull them.
  3. Configure Log Publishing - In the AWS VPC Dashboard, configure each VPC (or subnet or interface) to publish logs to a destination folder in the bucket.
  4. Confirm Log Publishing - Back in the S3 console, confirm that logs for each VPC are being published to the destination folders in the bucket.
  5. Register Cloud in portal - In the Kentik Detect portal, create a new Cloud pointing to the S3 bucket (as shown below), which results in a Kentik “cloud device” being automatically created for each destination log folder.
Add AWS Cloud

Once the new AWS Cloud is added it will appear in the Admin » Clouds page with details including name, provider, and status.

AWS Marketplace Integration

Last but not least, Kentik Detect is now available in the Amazon AWS Marketplace. Customers can simply pick a plan and subscribe to the Kentik analytics service on AWS.

AWS Marketplace

For complete Cloud Visibility information, please see the Kentik for AWS and Kentik for GCP topics in the Kentik Knowledge Base or contact the Kentik Customer Success team at support@kentik.com.

Improved Onboarding and Discoverability

In concert with our cloud efforts we’ve also taken an overall look at how best to support two important useability goals: improving ease of onboarding and making it easier for you to discover valuable features that you might not be aware of.

Redesigned Onboarding Workflow

To simplify Kentik Detect setup in the portal we’ve added a landing page to the Admin section, with setup tasks grouped into panes that each correspond to a stage of the setup workflow:

  1. Add data sources – Access to dialogs and Admin pages for routers and switches, servers and hosts, AWS, and GCP.
  2. Tell us about your network – Interface and network classification, sites, and device labels.
  3. Enrich your data:
    - Custom dimensions: Add business context to your network data.
    - Flow tags: Categorize your network data.
    - Custom geo: Align geolocation with your business territories or regions.
    - Custom DNS: Specify alternate servers for DNS resolution.
  4. Customize your Kentik experience:
    - Set up tenants for the My Kentik Portal.
    - Configure subscriptions for reporting.
    - Define saved filters.
  5. Access and security:
    - Register and manage users.
    - Set up single sign-on and access control.
  6. Plans – Check your utilization of Kentik Detect compared to the limits defined in your plans.

Collapsible Admin Sidebar

In addition to adding a landing page in Admin we’ve also made the sidebar collapsible, so it’s out of the way when you’re not navigating from page to page. To access the Admin menu, just move your mouse toward the left edge of any Admin page, which will expand the sidebar and show the familiar list of Admin pages.

Improved Feature Discoverability

Admin isn’t the only area that’s been rethought. The landing page of the portal Library has also been revamped to make it easier to discover features that will maximize the benefits you can gain from Kentik Detect. We’ll continue to make valuable features more apparent. At the moment we’re using the landing page to highlight the following options:

  • Configure data sources now prompts you to add new devices and clouds to your existing fleet, which is typically the first action a new Kentik user will want to perform.
  • Featured Content exposes noteworthy Kentik presets for Dashboards and Saved Views. This section will evolve to promote content that’s tailored to individual users based on role, type of network, and use cases.
  • Resources lists useful links that can help new Kentik users get up to speed and bootstrap their knowledge of Kentik Detect and the portal. Expect to see more personalization here over time as well.
  • Product Updates includes links to the latest entries in this Product Update blog, so you no longer have to check for them on the Kentik website.

AS Groups

Autonomous systems (ASes) represent the networks that collectively make up the Internet and enable the use of BGP to compute routes for the delivery of traffic. With our new AS Groups feature we allow you to assign one or more AS to a group that you name so that you can view traffic from a set of ASes as if they were a single entity. There are several scenarios in which this ability to control how ASes are represented in query results may be useful to the operator of a network:

  • Logical grouping of ASes – ISPs that are part of a broader consortium may have multiple ASes spanning multiple countries. AS grouping gives you the ability to assess the traffic from a multi-AS organization as a single entity, which is particularly important for interconnection/capacity planners in the context of peering.
  • Mapping of existing AS labels – Many organizations refer to ASes with labels that are distinct from the official name of the AS. An AS group may be used to associate an existing label with one or more ASes, so that the AS naming by which ASes are represented in Kentik Detect can be made consistent with the AS naming that you already use in other systems.
  • Naming by internal topology – It’s now common in modern, large-scale data centers to deploy Clos architecture, which relies on BGP to the rack, with each rack corresponding to its own private ASN. AS groups enable you to reference and visualize groups of private ASes in a way that reflects this Clos-based datacenter topology.

As shown here, Kentik Detect users with Admin access can find the AS Groups page either from the Admin landing page, in the Enrich Your Data pane, or via the popout sidebar on all Admin pages.

Managing AS Groups

The AS Groups page allows users to configure/add groups of ASes, either private or public. If a group includes just one AS then the group’s name will simply override Kentik’s default name for that AS.

The use of AS groups has the following effect on Kentik Detect queries:

  • Group-by - If at least one AS group has been configured in your organization then the Use AS Groups switch will be present in the Advanced Options section of the Query pane in the Data Explorer sidebar. When the switch is on (default) and the group-by dimensions include Destination AS or Source AS, the results from all ASes in a given group will be summed for top-X evaluation, graph plotting, and display in the results table (as shown below, a group icon will appear at the left of the group name). In this context, no additional dimension is needed to use AS groups.
  • Filtering - Inclusion or exclusion of traffic from a given AS group may be achieved by setting that group as the value of a filter on the dimension AS Group, as shown below.

For more on the AS Groups feature, please see the AS Groups article in the Kentik Knowledge Base or contact the Kentik Customer Success team at support@kentik.com.

Functionality, Performance, and Scalability

Beyond new features, we’ve continued our ongoing work on refining the utility and performance of Kentik Detect. The following enhancements cover areas that you can see as well as areas that are under the hood.

Enhanced Mitigation States and Controls

Anomaly detection, alerting, and mitigation, which are among the core features of Kentik Detect, sometimes involve complex situations like multiple mitigation actions and overlapping alarms. To better handle these scenarios we’ve simplified our state machine model. Updates include:

  • Take manual control – Users can now assert manual control over mitigations that were originally triggered automatically. To support this change, we’ve created a separate set of manual mitigation states that parallel the states used in automated mitigation.
  • Easier mitigation deletion – When deleting a mitigation, users now don’t have to additionally clear the mitigation on the mitigation appliance or wait for state transition to occur.
  • Mitigation escalation – When an alarm escalates (starts as Minor and becomes Major), mitigations will now escalate in parallel. That means users can now associate a particular mitigation method with the minor threshold and a different method with the major threshold.

In addition to these backend changes, the UI for mitigation actions in the Active Alarms table (Alerting » Active) has been changed to provide more flexible and granular control. Play and Stop icons have now been replaced with context dependent icons and tool tips that reflect the current mitigation state.

Selective Interface Classification

A number of customers requested that we allow Interface Classification rules to be applied to some devices and not others. As shown below, the IF settings in the Add Rule dialog now include two new controls that enable you to tailor sets of Included Devices and Excluded Devices that govern application of the rule.

These whitelists and blacklists will also be displayed, as shown below, in the Rules List on the main classification page (Admin » Interface Classification).

For more information, please see the Rule IF Settings topic in the Kentik Knowledge Base or contact the Kentik Customer Success team at support@kentik.com.

Query Engine Improvements

Kentik Data Engine is the backend where your network traffic data is collected and enriched, and from which it is pulled at query run-time. Recent enhancements enable Kentik Detect to support ad-hoc queries over longer time ranges with much higher cardinality. For example, we can see source/destination IP pairs as a time series over a time range of 90 or more days. As requested by some customers, Data Explorer’s Table view can also now display much deeper results — up to 50,000 rows — for queries on certain group-by dimensions when the metric type is Total. Additional changes include improved performance for queries that filter on long lists of IP addresses.

Usability and UI Improvements

As we work to make Kentik Detect more powerful we’re also aware that features are most valuable to users when they’re easy and straightforward to use, as well as to customize for individual needs. The improvements in the next couple of sections are designed with that in mind.

Redesigned Dimension Selector

Working with dimensions for both group-by and filtering is an integral part of defining the queries used throughout Kentik Detect. We’ve taken a fresh look at how customers access and select dimensions, and made the following improvements in the dimension selectors for both group-by and filtering:

  • Dimension categories – Dimensions are now organized into more intuitive categories, such as Network & Traffic Topology, IP & BGP Routing, and Cloud.
  • Directional columns – Additionally, dimensions are now organized into Source, Destination, and Non-Directional columns to make them easier to locate. This also means that the Source and Destination versions of related dimensions are always located next to each other in the same row.
  • New dimensions for clouds – The newly-added categories include provider-specific categories for AWS and GCP dimensions.
Amazon Web Services
Google Cloud Platform
  • Bi-directional filtering – In the dimension selector for Ad-Hoc Filter Groups, we’ve also added a Source or Destination column to simplify filtering on the same value in two related dimensions.

Custom Color Palettes

Kentik customers have always recognized UI design as one of our major differentiators. The ability to customize aspects of that design, tailoring it to individual needs and preferences, makes it even more powerful. Our latest step in this direction is the new Visualizations tab, which you’ll find on the My Profile page (access from the drop-down menu at far right in the main portal navbar).

As shown below, you can tune five different color settings that determine how visualizations are rendered:

  • Theme – Toggle between the Standard Theme with a light background or the Dark Theme.
  • Labels – Customize the color of labels by choosing from a popup color palette or entering a hex number.
  • Overlays – Change the color of the Total and Historical overlays that are used on time-series visualizations.
  • Quantitative – Choose a color theme for all chart types that display quantitative data such as Stacked Area Chart, 100% Stacked Area Chart, Stacked Column Chart, and Bar Chart.
  • Qualitative – Choose a color theme for all chart types that show qualitative data such as Line Chart, Pie Chart and Sunburst.

When changing the Quantitative and Qualitative color palettes, you can preview the effect on different view types by choosing a view type from the drop-down menu at upper right of the preview visualization. Once you save the changes, the new palette will be applied to all of your existing (and future) visualizations of that type (Quantitative or Qualitative) throughout the portal.

Finally, for Qualitative and Qualitative settings, you can create custom color palettes by turning on the Use custom values switch.

For more information, please contact the Kentik Customer Success team at support@kentik.com.

We use cookies to deliver our services.
By using our website, you agree to the use of cookies as described in our Privacy Policy.