Overview of Big Data DDoS Protection

Big data DDoS protection is the application of big data technology to the process of detecting and mitigating Distributed Denial of Service (DDoS) attacks. By utilizing cloud-scale compute and storage resources, big data DDoS makes it possible to collect and examine network flow data in highly granular fashion, at great scale and with superior baselining intelligence.  The combination of scale, granularity and baselining smarts ensures greater accuracy in stopping DDoS attacks than is possible with legacy DDoS protection appliances. 

Big data DDoS protection has evolved because the limiting, scale-up architecture of appliance-based DDoS detection is no longer sufficient to track the massive amount of flow records and individual IP addresses that operationally characterize today’s digital business networks.

The primary goal of a DDoS attack is to either limit access to an application or network service, thereby denying legitimate users access to the services. There are many types of DDoS attack schemes that are used today and they are steadily becoming more sophisticated.  However, their common goal is to overwhelm targeted network resources with traffic or requests for service from many different sources – potentially hundreds of thousands or more. This effectively makes it impossible to stop the attack simply by identifying and blocking a single IP address.  The sheer distribution of attacking sources also makes it very difficult to distinguish legitimate user traffic from attack traffic when spread across so many points of origin.

DDoS protection deploys manual or automated detection, plus one or more remediation techniques including traffic flow filtering, rate-limiting, and blocking/dropping, so attacks can’t consume resources needed to service legitimate traffic.  In order to detect an attack, one has to gather a sufficient amount of data by monitoring network-wide traffic flows, then rapidly and accurately analyze that data.

The Omnipresent DDoS Threat

There is no doubt, as evidenced in the alarming rise of DDoS attacks, that identifying and stopping DDoS attacks is an absolute necessity for businesses that rely on Internet traffic to in order for them to avoid disruption of applications and services, revenue loss, and brand damage.

The majority of DDoS attacks are volumetric in nature, with the intent to congest all of the target network’s bandwidth related to a service or network segment. Roughly 90% of all DDoS attacks are volumetric, with application-layer attacks making up the remaining 10%. According to Akamai’s Q3 2016 State of the Internet Report, the majority of volumetric attacks are IP flood attacks involving a high volume of spoofed packets such as TCP SYN, DNS, UDP, or UDP fragments.

DDoS attacks are rapidly increasing in frequency and size. While mega attacks that last for many hours and reach 200 Gbps or more make the news, the vast majority of attacks last under an hour and are less than 1 Gbps in volume. Smaller attacks often happen without being noticed, though they may be harbingers of larger attacks to come. Mid-sized attacks are more readily felt, but distinguishing between a friendly surge in normal traffic and an attack is key to timely response. Large attacks are fairly obvious, and in these cases diagnosing the traffic is important to understand network entry points and sources. In all cases, a clear detection assessment is important to understand the best most expedient way to mitigate the attack.

In-line versus Out-of-band DDoS Detection

There are two primary means of implementing DDoS detection: an in-line detection appliance or out-of-band detection via traffic flow record analysis that integrates with any mitigation technique, whether on-premise or in the cloud. When performing in-line DDoS detection, all traffic is sent through one or more DDoS detection appliances that support deep packet inspection. The appliances examine the incoming traffic, and if particular traffic flows or packets are determined to be attacks, they are discarded, while legitimate traffic is allowed to pass through.

The basic in-line DDoS detection capabilities of network devices such as load balancers, firewalls or intrusion prevention systems may have once provided acceptable detection and mitigation when DDoS attacks were less sophisticated but high-volume attacks can overwhelm these devices, and sophisticated multi-vector DDoS attacks can often evade them because they are network location specific.

Out-of-band DDoS detection utilizes flow data from NetFlow, J-Flow, sFlow, and IPFIX-enabled routers and switches to understand unusual traffic spikes that occur across the network and detect attacks.  Attack mitigation via in-line devices or routing methods that drop selected traffic is then triggered manually or automatically.

How Cloud and Big Data Help Detection Accuracy

The first generation of out-of-band DDoS detection solutions were based on single server software design.  While far better than nothing, single servers simply don’t have the compute, memory and storage resources to track high volumes of traffic data on a network-wide basis.  This is particularly true when attempting to perform dynamic baselining, which requires scanning tens of billions of flow records to establish baselines over time, in order to assess whether current conditions constitute an anomaly. Regardless of whether it is deployed on-premises or in the cloud; single server DDoS protection is insufficient to accurately detect and mitigate today’s attacks in a consistently reliable fashion.

The key to solving the DDoS protection accuracy issue is big data. By using a scale-out system with far more compute and memory resources, a big data approach to DDoS protection can continuously scan network-wide data on a multi-dimensional basis without constraints.

Cloud-scale big data systems make it possible to implement a far more intelligent approach to the problem, since they are able to:

  • Track and baseline millions of IP addresses across network-wide traffic, rather than being restricted to device level traffic baselining.
  • Monitor for anomalous traffic using multiple data dimensions such as the source geography of the traffic, destination IPs, and common attack ports. This allows for greater flexibility and precision in setting detection policies.
  • Apply learning algorithms to automate the upkeep of detection policies to include all relevant destination IPs.

Of course, constructing an open-source big data system for DDoS detection isn’t trivial.  However, cloud-based/SaaS network visibility and DDoS detection systems are making it far easy to get the advantages of big data for DDoS mitigation without making major capital and R&D investments.

These advances are making a new hybrid model possible, where DDoS detection is performed by a best-of-breed, cloud service that automates the triggering of RTBH, on-premises and cloud-based mitigation appliances.  Big data detection systems also provide the added benefit of deep, forensic analytics, plus the ability to incorporate network performance, planning and other capabilities.

On Kentik & More Reading

Kentik Detect offers the industry’s only big data network visibility and DDoS protection solution built from the ground up on big data and delivered as a cost-effective SaaS.  Kentik Detect offers the industry’s most accurate DDoS detection, and can automatically trigger mitigation via RTBH, Radware DefensePro or A10 Thunder TPS mitigation.

For more information on how big data delivers 30% greater DDoS detection speed and accuracy, check out the blog post on Big Data for DDoS Protection, read the PenTeleData case study, or download The Case for Big Data-Powered DDoS Protection white paper. Know you want to get big data-powered DDoS protection today?  Start a free trial.