Understanding DDoS Detection: A Tutorial
Overview of DDoS Detection
DDoS detection is the process of distinguishing Distributed Denial of Service (DDoS) attacks from normal network traffic, in order to perform effective attack mitigation. The primary goal of a DDoS attack is to either limit access to an application or network service, thereby denying legitimate users access to the services. There are many types of DDoS attack schemes that are used today and they are steadily becoming more sophisticated. However, their common goal is to overwhelm targeted network resources with traffic or requests for service from many different sources – potentially hundreds of thousands or more. This effectively makes it impossible to stop the attack simply by identifying and blocking a single IP address. The sheer distribution of attacking sources also makes it very difficult to distinguish legitimate user traffic from attack traffic when spread across so many points of origin.
The first step in avoiding or stopping a DDoS attack is knowing that an attack is taking place. To detect an attack, one has to gather a sufficient network traffic information, then perform analysis to figure out if the traffic is friend of foe. This process can be performed manually or in an automated fashion. DDoS detection is the key to quickly stopping or mitigating attacks and in order for this to happen, two success criteria need to be met: 1) speed of detection and 2) accuracy of detection. So detection methods are a key consideration in formulating a strong DDoS defense.
The DDoS Threat
There is no doubt, as evidenced in the alarming rise of DDoS attacks, that DDoS detection is an absolute necessity for businesses that rely on Internet traffic in order for them to avoid disruption of applications and services, revenue loss, and brand damage. Neustar, Inc. and others regularly publish reports on DDoS attack and protection trends. Neustar’s October 2016 report highlights that DDoS attack volume has remained consistently high and that these attacks cause real damage to organizations. Some highlights from that report:
- DDoS attacks are unrelenting and show no sign of abating: The overwhelming majority of surveyed organizations (73 percent) suffered a DDoS attack. Eighty-five percent of attacked organizations were attacked more than once and 44 percent were attacked more than five times.
- DDoS attacks are only the tip of the spear in complex assaults: The majority of organizations that suffered a DDoS attack (53 percent) also experienced some form of additional compromise. Forty-six percent of breached organizations discovered a virus, malware was activated at 37 percent of breached organizations, and ransomware was encountered at 15 percent of breached organizations.
- DDoS attacks are time-consuming and expensive: It can take hours to detect and mitigate a DDoS attack at significant cost to the organization. Seventy-one percent of organizations took an hour or more to detect a DDoS attack and 72 percent took an additional hour or more to respond to the attack. Forty-nine percent of surveyed organizations lose $100,000 or more per hour of downtime during these attacks.
The most common type of DDoS attack is the volumetric attack, with the intent to congest the target network’s bandwidth related to a service or network segment. Roughly 90% of all DDoS attacks are volumetric, with application-layer attacks making up the remaining 10%. According to Akamai’s Q3 2016 State of the Internet Report, the majority of volumetric attacks are IP flood attacks involving a high volume of spoofed packets such as TCP SYN, DNS, UDP, or UDP fragments. A growing percentage of attacks are reflection and amplification attacks using small, spoofed SNMP, DNS, or NTP requests to many distributed servers to bombard a target with the much more bandwidth-heavy responses to those requests.
DDoS attacks are rapidly increasing in frequency and size. While mega attacks that last for many hours and reach 200 Gbps or more make the news, the vast majority of attacks last under an hour and are less than 1 Gbps in volume. Smaller attacks often happen without being noticed, though they may be harbingers of larger attacks to come. Mid-sized attacks are more readily felt, but distinguishing between a friendly surge in normal traffic and an attack is key to timely response. Large attacks are fairly obvious, and in these cases diagnosing the traffic is important to understand network entry points and sources. In all cases, a clear assessment is important to understand the best way to mitigate the attack.
In-line versus Out-of-band DDoS Detection
There are two primary means of detecting DDoS attacks: in-line examination of all packets and out-of-band detection via traffic flow record analysis. Either approach can be deployed on-premises or via cloud services. The basic in-line DDoS detection capabilities of network devices such as load balancers, firewalls or intrusion prevention systems may have once provided acceptable detection when DDoS attacks were smaller but high-volume attacks can overwhelm these devices, since they utilize memory-intensive stateful examination methods.
Dedicated DDoS mitigation appliances are the primary way to accomplish in-line detection (and remediation) today. However, they can become costly and have a short life cycle in the face of higher volume threats. These appliances are still necessary and relevant for mitigation because ASIC and Network Processor power is needed for deep packet inspection when scrubbing traffic but for cost-effectiveness and scaling reasons, moving detection out of mitigation devices has become the norm.
Out-of-band DDoS detection is accomplished by a process that receives flow data from NetFlow, J-Flow, sFlow, and IPFIX-enabled routers and switches, then analyzes that flow data to detect attacks. Mitigation of the attacks is then triggered manually or automatically, via routing or appliance-based methods.
How Cloud and Big Data Help Detection Accuracy
The first generation of out-of-band DDoS detection solutions were based on single server software design, mostly running on standalone rackmounted server appliances. While far better than nothing, single servers simply don’t have the compute, memory and storage resources to track high volumes of traffic data on a network-wide basis. This is particularly true when attempting to perform dynamic baselining, which requires scanning massive amount of flow data to understand what is normal, then looking back days or weeks in order to assess whether current conditions constitute an anomaly. Regardless of whether it is deployed on-premises or in the cloud, single server DDoS detection is insufficient to accurately detect today’s attacks in a consistently reliable fashion.
The key to solving the DDoS protection accuracy issue is big data. By using a scale-out system with far more compute and memory resources, a big data approach to DDoS protection can continuously scan network-wide data on a multi-dimensional basis without constraints.
Cloud-scale big data systems make it possible to implement a far more intelligent approach to the problem, since they are able to:
- Track and baseline millions of IP addresses across network-wide traffic, rather than being restricted to device level traffic baselining
- Monitor for anomalous traffic using multiple data dimensions such as the source geography of the traffic, destination IPs, and common attack ports. This allows for greater flexibility and precision in setting detection policies.
- Apply learning algorithms to automate the upkeep of detection policies to include all relevant destination IPs.
Of course, constructing an open-source big data system for DDoS detection isn’t trivial. However, cloud-based/SaaS network visibility and DDoS detection systems are making it far easy to get the advantages of big data for DDoS mitigation without making major capital and R&D investments.
These advances are making a new hybrid model possible, where DDoS detection is performed by a best-of-breed, cloud service that automates the triggering of RTBH, on-premises and cloud-based mitigation appliances. Big data detection systems also provide the added benefit of deep, forensic analytics, plus the ability to incorporate network performance, planning and other capabilities.
On Kentik & More Reading
Kentik Detect offers the industry’s only big data network visibility and DDoS protection solution built from the ground up on big data and delivered as a cost-effective SaaS. Kentik Detect offers the industry’s most accurate DDoS detection, and can automatically trigger mitigation via RTBH, Radware DefensePro or A10 Thunder TPS mitigation.
For more information on how big data delivers 30% greater DDoS detection speed and accuracy, check out the blog post on Big Data for DDoS Protection, read the PenTeleData case study, or download The Case for Big Data-Powered DDoS Protection white paper. Know you want to get big data-powered DDoS protection today? Start a free trial.