What is Synthetic Monitoring?

Synthetic monitoring is the active process of simulating visitor requests to a network-accessible resource and testing for availability, response time, and other performance metrics.

Increasingly, modern Network Performance Monitoring solutions are incorporating synthetic monitoring features, which are traditionally associated with a process/market called “Digital Experience Monitoring”. In contrast to flow or packet capture (which we might characterize as passive forms of monitoring), synthetic monitoring is a means of proactively tracking the performance and health of networks, applications and services.

In the networking context, synthetic monitoring means imitating different network conditions and/or simulating differing user conditions and behaviors. Synthetic monitoring achieves this by generating different types of traffic (e.g., network, DNS, HTTP, web, etc.), sending it to a specific target (e.g., IP address, server, host, web page, etc.), measuring metrics associated with that “test” and then building KPIs using those metrics.

These synthetic tests are both automatic and periodic. They are conducted at regular intervals (even down to sub-one-minute intervals in the case of synthetic monitoring solutions like Kentik Synthetics) to ensure that an application or service is still performing as expected.

When researching synthetic monitoring, the related topic of passive “Real User Monitoring” (RUM) is likely to turn up. Real User Monitoring involves capturing and analyzing each user transaction/interaction with an application. The idea is to track and monitor the actual usage of an application, and record what happened (after the fact). In contrast to synthetic monitoring, RUM is a form of passive monitoring.

It’s not a matter of which is better. It depends on what you’re trying to accomplish. Here are a couple of short definitions for both of these terms as they pertain to monitoring:

• Active: A proactive test simulates a user by accessing a resource, waiting for a response, and interpreting any results from the communication. Synthetic monitoring is a form of active monitoring. A network management station that polls devices on a network is also an active monitor.

• Passive: Passive monitoring reactively monitors actual/authentic user interactions with a resource then measures and interprets the communications. Real user monitoring is a form of passive monitoring. Placing a packet analyzer on the network and observing traffic is also a form of passive monitoring.

Both active and passive monitoring are useful and, while they are alternatives, combining both enables improvements and optimizations in the end-user digital experience.

On the surface, the goal of active synthetic monitoring is to ensure that an application or service (e.g., web page, DNS) is up and responding to actual end users in a timely manner. It can answer questions such as:

• Is the DNS responding to end users in under 50 milliseconds?
• Is the website up and loading in under 1500 milliseconds?
• Are shopping cart transactions working?
• Are API endpoints available and responding in the expected manner?

To accomplish this, the synthetic transaction monitor generally measures two things:

• The availability of the resource over time (e.g., 99.999% up-time)
• The responsiveness in milliseconds of the resource over time

This reliable routine operation can provide more details than the above implies. Trending and alerting also play significant roles in this topic. These details are below. (See also: “What is Synthetic Transaction Monitoring?” for more detail about the monitoring of specific web transactions that result from user actions in a site or web application.)

Synthetic monitoring solutions (such as Kentik Synthetics) and the synthetic tests they enable have a wide variety of use cases—including network performance monitoring, web application monitoring, and monitoring/improving user experience.

Synthetic monitoring enables web application developers and operations professionals get a better understanding of web application availability, API performance, and user experience. Using HTTP-based synthetic tests, the availability and proper operation of GET, HEAD, PATCH, POST, and PUT methods for various API endpoints can be ensured. Both the availability of an endpoint, and also its HTTP response status codes (e.g., ensuring that authentication is working as expected), can be monitored. Other metrics such as average time to last byte, size of the response, etc. can also be observed.

Additionally, synthetic tests can be used to analyze and alert on application/network health and performance metrics such as ping latency, jitter, packet loss, HTTP latency and certificate expiry. Solutions like Kentik also allow automatically running traceroute tests to understand metrics such as the maximum number of hops between various network destinations.

In web applications, page load speed is an important component of user experience. Synthetic tests can be used to automatically and periodically test load times for various pages in a web application. These tests can provide a granular breakdown of how every component on the page is loading. Using that information, web app developers can easily track down exactly what’s impacting a site’s performance—whether it’s a site hosted on-premises, in the cloud, or by a SaaS provider.

In Kentik, these tests analyze a full browser page load using Headless Chromium run by any number of Kentik app agents, which then return detailed statistics about how the page loaded—showing the status codes and load times for various page components as well as times for initial response, navigation, domain lookup, etc.

For a short video overview of how page load testing works, see “Solving Slow Web Applications with the Kentik Synthetics Page Load Test“.

In addition to testing the basic response/loading behavior of a website, network service, or application, synthetic monitoring can be used to simulate a user’s actual interactions with a web application. A “transaction” in this context is a series of actions that a user might take (e.g., logging in, searching for some product, adding that product to a shopping cart, and checking out).

After creating a clickpath script or recording a clickpath that represents the transaction, remote agents can replay that script (simulating the user requests), to test the performance and availability of all of the various steps and collect performance data about the transaction.

For example in Kentik, such tests are a series of actions that are driven by a Google Puppeteer script (created using the Recorder tab in Chrome Developer Tools) that is executed by an agent running Headless Chromium. The results of such tests include the health status and total transaction time for each agent being tested from, and can show any screenshots that the script creator specified as actions in the script.

Learn more about this topic in our article, “What is Synthetic Transaction Monitoring?”

Though it’s really just a special case of web application monitoring, it’s worth mentioning that synthetic monitoring can be used to automoatically test the availability and proper operation of any REST API (a collection of API endpoints) or an individual API endpoint. As in web performance monitoring, the availability and proper operation of GET, HEAD, PATCH, POST, PUT methods for various API endpoints can be tested. In Kentik Synthetics, ping, traceroute and BGP (border gateway protocol) data can also be collected.

Synthetic monitoring can also be used to monitor network performance to IP addresses, hostnames, mission-critical SaaS services, or to set up tests between agents themselves.

Additionally, solutions like Kentik Synthetics can monitor BGP prefixes and reachability and automatically notify network operators about potential route leaks and BGP route hijacks.

A unique feature in Kentik’s synthetic monitoring solution allows for “autonomous testing”, wherby the platform intelligently leverages actual network traffic data (provided by data sources such as netFlow) to automatically generate tests towards the most frequent network sources and destinations.

By analyzing flow records along with correlated network traffic data (such as SNMP, BGP, GeoIP, etc.) the platform is able to automatically select IP addresses to test from or toward. Types of tests that can be automatically configured include ASN tests, CDN tests and tests to and from various geographic locations.

Synthetic monitoring can also be used to measure the availability and responsiveness of DNS Servers or to verify the DNSSEC keychain. In this application tests report on the performance of one or more DNS servers associated with a hostname, showing the address resolution time and the resulting IP address provided by those servers.

Connections to online resources might originate from anywhere in the world. This means we can’t monitor a cloud-hosted resource for availability and responsiveness from just a single location. Synthetic tests typically measure from multiple, geographically diverse locations, using testing agents located in different datacenters around the world.

For example, Kentik has built out a global network of synthetic testing agents that are used by customers to verify performance levels of all major public or private cloud-based applications and SaaS applications. These agents have been located inside Amazon Web Services, Google Cloud Platform, Microsoft Azure, Alibaba Cloud, IBM Clouds. Additionally, customers can easily install custom agents in their own geographically-diverse data centers.

With numerous network and infrastructure elements that need to be tested and a user population that is geographically dispersed, the number of testing combinations and permutations can grow to be very large. As a result, the configuration of synthetic tests can become complex and time-consuming when, for example, you want to test between multiple cloud regions (such as AWS regions). See also: “Using Synthetics for Your Cloud Monitoring Needs” and “A Guide to Cloud Monitoring Through Synthetic Testing”.

To manage this complex environment, vendors have developed tools that make implementing and monitoring synthetic tests possible on a large scale. In the following video, Kentik Synthetics expert, Anil Murty, demonstrates the easy configuration of a synthetic testing “mesh” between a variety of cloud services:

This video is a brief excerpt from “How to Continuously Monitor Inter- and Intra-Cloud Performance”—you can watch the entire presentation here.

Reputation and service level agreement validation are two big reasons why staying ahead of performance issues is a must. It’s really simple—poor connections can lead to unhappy customers and declining revenue. The sooner NetOps can detect an availability issue, the faster they can react.

Many users today see excessive application latency as a form of outage. Understanding and monitoring when a critical service drops below a minimum level of performance is paramount. A global network of agents that allow for omnidirectional synthetic transaction monitoring is one of the best ways to ensure that all users, regardless of physical location, are experiencing an acceptable level of service.

Some of the most prominent benefits of synthetic monitoring include:

Using synthetic monitoring, organizations can continuously validate the performance of network, web, and application components, track changes over time, and be alerted to anomolies or problems before they impact the experience of actual end-users.

Continually testing various components of a network or application can establish a baseline for expected behavior under normal conditions. Having established baseline metrics, operations personnel can be alerted when performance of any particular components drops below an acceptable threshold.

Synthetic monitoring can be used to perform tests from the perspective of specific users by their role (e.g., a logged in user, a logged out user, admin user, etc.) or by geographic location, helping to ensure system performance in a global sense (both literally and metaphorically).

The Kentik Network Observability Cloud offers a modern, SaaS-based approach to digital experience monitoring. Kentik delivers network performance monitoring and diagnostics that combine flow-based monitoring, cloud network observability and synthetic monitoring features to enable for proactive monitoring of all types of networks.

Start a free trial to try it yourself. Kentik’s 30-day free trial includes a generous allotment of synthetic testing credits that can be used to perform real-world testing on the applications and services important to your own organization.