Wow, it’s been a crazy and hectic past 24 hours for a lot of people out there on the Internet.
Unless you live under a rock, you’ve heard about some high profile Distributed Denial of Service (DDoS) attacks against major web properties like Twitter, Facebook, LiveJournal and even our own WebHop service. We’ve been carefully monitoring the situation, checking our system performance and reviewing and practicing our DDoS mitigation strategies. Luckily, we’ve not been involved directly with the social media attacks and only experienced a small flood to Webhop.
So what is a DDoS attack? I’m going to discuss some of the technicalities, what it is and some best practices for network design and mitigation. If you’re not at least a level 3 (out of 10) geek, you might want to turn back now.
(Oh and for the media: it’s a Distributed Denial of Service (DDoS) attack, not a Domain Name System (DNS) attack! See the difference? Please be more careful about the content of the stories you report.)
So what’s a DDoS?
A DDoS is any attempt by a human directly or by human-created software to deny users access to a service of some sort through the use of a distributed, coordinated attack. The attacker’s goal is to overwhelm the target’s computer system in such a way that it is no longer able to provide service to valid users of that system.
Here are some examples:
- Using multiple machines to generate a lot of ICMP “ping” packets to a target as quickly as possible to overload the Internet connection, the firewall or the target system itself. Taking any of these components out of this system will “choke” the network, preventing legitimate users from accessing it.
- Distributing malware to as many machines as possible which runs on unsuspecting users’ computers and have that malware connect back to a central host that provides control of these machines.
- Using the control system to tell the “zombied” machines to send out lots of SPAM. Yes, sending SPAM is a type of DDoS attack. These complex networks of control hosts and zombied machines are something we in the industry call a “botnet” and have many capabilities than just sending out SPAM.
- Get a bunch of your friends together (>20) and head down to the Bridge Cafe in Manchester, NH at lunch time. Walk in and all try to order something at the same time. (I highly suggest the Luna tuna melt.) You’ll effectively launch a DDoS (distributed == multiple people, coordinated == all y’all planned it) on the Bridge staff. Don’t tell Charlie I told you to do this!
In the earlier days of the Internet, we saw a more basic form of this attack in which one machine on the Internet attacked another machine on the Internet one on one. In this case, the effectiveness of the DoS was limited to the size of each party’s Internet connection or computing resources.
In summary: a DDoS is an attempt to “choke” out some weak portion of an Internet system by overloading it remotely.
In the case of many of these recent attacks, the target was the website servers handling Twitter, Facebook and LiveJournal. The attackers likely tried to overload the Internet connection to these sites, the firewalls, the load balancers or the web servers. This is a direct attack against the web site itself.
But there are other types of attacks too. Remember that to even get to the website you want to go to, your computer has to do a DNS request to determine the IP address of the site you want to head to. Performing a DDoS attack against the DNS servers for a particular site can also take a site offline, but this wasn’t the case in yesterday’s attacks. Generally the DNS flavor of the attack is much more complex to launch successfully, requires more “zombies” to help and is harder to mitigate (a great reason to outsource your DNS to experts).
Beyond that, think of the dependencies around social media and the web today. Many websites rely on social networking site APIs to gather information to present their home pages. With all the impact on the social media sites, some of these other sites may have failed as well.
Defending Your Website
Unfortunately, this is a task of diminishing returns. If you’re not generally a DDoS target, you’re going to spend a lot of money to fully protect yourself. But if you are a target, you’re going to spend a lot of money to protect yourself and you’re likely at an arms race with the attacker as to who can build the bigger, more powerful network to either generate the DDoS or filter and mitigate it. Only a few providers in the world have networks and services capable of this.
My key points on how to prevent and mitigate DDoS are simple:
- Don’t make yourself a target. Keep your network clean of spammers and other miscreants that make trouble. You’re less likely to get wrapped up in their shenanigans.
- Awareness. Know your network’s normal behavior, so you can know when you come under a DDoS. There are many tools that can help you do this including NetFlow, sFlow, Splunk, Nagios, Cacti, Smokeping, Munin, DSC and others.
- Capacity. If you can, build the biggest network you can with effective elements for installing wirespeed access control lists at the edge of your network (on the routers facing the Internet) for Layer 3 and 4 mitigation and a deep packet inspection / caching / scrubbing layer in the core of the network, for advance mitigation in Layers 4 through 7 (generally the secret sauce). Finally, make sure you provision enough server capacity and tune them for best performance under high load.
- Practice your defense plans. Knowing how to use your defensive strategy is just as important as buying and installing it. If you don’t know how to use it effectively, why even have it? As though you are in the military, practice the drills over and over to get this committed to your staff’s minds