My sales and solutions engineers are often asked to discuss and further explain how we specifically monitor and triage a Distributed Denial of Service (DDoS) attack. The aim of this article is to highlight and provide some context to why organizations should be addressing cost and risk by employing Dyn’s protection stack to combat against DDoS.
With major workloads being interrupted and compromised more frequently, Distributed Denial of Service is and will continue to be a hot topic in the market. It’s important to partner with a provider that has an expert level understanding of what in_protocol and out_of_protocol attacks look like, what the attacker’s goals are, and how attacks can affect you when you are the target, when you are the amplifier, or when you are collateral damage.
Mean Time to Identification
Identification generally consists of a traffic analysis to determine the attack type and vector. We also derive an action plan from services impacted as well as an analysis of relative link saturation.
Dyn operates a global IP anycast network. Unlike the micro-PoP design, globalized IP Anycast helps ensure workload continuity as well as decreased mean time to identification (MTTI) which improves the timeline for mean time to mitigation (MTTM). More simply put, anycast clarifies which of Dyn’s assets are network adjacent to the source of the attack traffic which helps us formulate the best mitigation strategy.
For example, this design approach keeps an attack originating in Southeast Asia to that locality instead of East Coast America or Western Europe while offering the choice to steer compromised workload to other regions of the world . To this end, we keep a continuous eye on DDoS chatter and hotspots and ensure appropriate coverage for those regions. This routing strategy also changes the nature of the attack itself. Because the DDoS is contained by the locality of the attack it can’t be directed at a single point Tsunami style, and instead appears as a smaller overall increase across the network.
The next layer of defense is reducing the risk to the impacted assets in a multitenant environment. This pertains more to our namespace architecture and how the impacted delegated authority listens for requests rather than a hardware segregation, which by decreasing overall resources could increase risk. Though our nameserver pools are all part of the same announcement, Dyn can also re-route individual nameserver IPs so an attack on one IP can be isolated for remediation. Some of Dyn’s clients, find immense benefit in having their own private announcements, and therefore private or vanity nameserver pools.
Also, a secondary but major benefit of private authoritative DNS pooling is the added level of protection of DDoS attacks. Risks associated with shared infrastructure are often cited as a concern when discussing migrating enterprises to a DNS for Internet Performance environment. With a private pool, you would be the only tenant on the delegation. This will reduce the spillover should there be an attack on a shared delegation, and any impact while Dyn mitigates on that IP set.
Mean Time to Mitigation:
Mitigation is where we execute the required actions to minimize impact to Dyn’s products and services, i.e Firewall filters, Authoritative DNS traffic steering, and BGP re-routing.
Our globally distributed NOC utilizes specific tools that show an updated display of DNS query resources, that can ascertain the usage of the adjacent DNS servers. We in turn manipulate anycast announcements and traffic in response to the size and scope of an attack. We determine if the attacks are occurring in or out of protocols like UDP or TCP Fragments. Alternatively for attacks occurring in-protocol, query logs can aid in determining the source and destination of an attack and the best course of action in mitigating and remediating any impact to the platform.
Upstream from the network itself, we and Dyn’s transit providers can perform filtering with both hardware and software utilities on criteria matching the attack vector to scrub the traffic inbound to Dyn’s network. Utilizing these devices enables us to perform deep packet inspection to analyze incoming packets, and only allow certain requests through to Dyn’s network. In today’s norm of amplification attacks aimed at clogging a network’s bandwidth, this will most often eliminate the attack threat.
Mean Time to Validation
We exhaustively examine Dyn’s Internet Performance Management Platform to ensure that everything is as it should be:
- Did Dyn’s filters have the desired effect?
- Have we properly identified the geographic location of the event source?
- Is traffic flowing as expected?
- Is Dyn’s 3rd party monitoring healthy?
When an attack subsides or is stopped because of mitigation, it is very important for the impacted business to utilize any of the aforementioned tools to ensure that services are no longer being impacted. In the event that attack traffic subsides it is prudent to continue mitigation and identification processes until a consensus that the risk has been fully eliminated has been established.
Any time any event mitigation takes place, it is very important that the impacted business continuity and network incident response plans are employed and then held to a detailed retrospective. Many times, new attacks will allow businesses to uncover new information like specific protocols and tactics employed by the attackers-at-large and should be added to the impacted team’s playbooks and forensics strategies. During these critical events, an incident report is created at the first identification that a bandwidth or BGP alarm is in-fact, an attack. This incident is initially at a low risk level, but allows documentation to be consolidated throughout the incident to provide the best post mortem coverage. This also allows transparency in business impact and cross-team visibility — all parties have full context of what is occurring and what has happened thus far. Additionally, packet captures can breed new filtering rules and playbooks can be reviewed and altered for the most deliberate handling and best client experience — especially in the most critical moments.
With these introspections, we make Dyn better for Dyn’s clients with every iteration.