01.18.2012 By Tom Daly
Protesting SOPA and PIPA With Web Blackouts
Since our last post on the topic of SOPA back in December, there has been a few significant events that have caused our concerns regarding SOPA to move over to PIPA.
First, House Judiciary Committee Chairman Lamar Smith announced that he plans to make a manager’s amendment to SOPA to remove the DNS blocking provisions from the bill. It seems as though Congress has recognized the importance of the way the DNS is constructed and how the former provisions would have caused a fracturing of the DNS and put up false barriers to the ongoing deployment and support of DNSSEC.
As we blogged in December, the technical means for implementing SOPA now lie with the domain registrar and the authoritative DNS provider – the same way that Internet abuse handling techniques have handled these issues for years.
Second, Dyn has begun to monitor a piece of legislation known as the Protect IP Act (PIPA) that was introduced to the Senate by Senator Patrick Leahy. The Senate Judiciary Committee has passed the bill, but it has been placed on hold by Senator Ron Wyden. This bill also has provisions for DNS based redirection and blocking of sites, which we continue to believe that in implementation will result in a degradation of DNS services offered across the Internet.
Third, today marks a day of active web protest against SOPA and PIPA, indicating that an implementation of SOPA or PIPA would effectively subject the Internet to U.S. national censorship; a concept which becomes a technical feasibility under the implementation of SOPA or PIPA. Sites including Wikipedia and Google are participating in the protest by blocking out portions of their web sites from access today.
Frankly, these protests prove a point on what the result of such legislation could levy against Internet operations as a whole. You can learn more about this web protest here. If you look at the upper left of Dyn’s website, you can see an update of our logo to voice our opposition.
At Dyn, we continue to strongly oppose any legislation that puts the stability and availability of the global DNS system as risk. We believe that there are existing processes at the domain name registrar and authoritative DNS levels to deal with the issues raised by SOPA and PIPA and that DNS-level blocking or redirection would effectively break the DNS.
Read More01.11.2012 By Tom Daly
Anycast Vs. Unicast: The Skinny on Nameserver Routing
You’re the guy/gal charged with making sure your business’ web site and ecommerce storefront are running nice and fast, so you run a quick waterfall chart on your site and learn that DNS is limiting your site’s performance. You jump on the Internet, do some Google searches and learn about this thing called Anycast DNS.
You then follow some more links and learn about another thing called Unicast DNS. You read people talking about having both and others talking about having one or the other. You can’t really decipher between the two because there’s hardly any accessible documentation about it. I’m going to break this mystery down for you in this post, kicking it off with a simple guidance statement:
It’s all about the routing, redundancy and geography.
Read More01.04.2012 By Tom Daly
Introducing Labs: Let’s Go Do Some Science!
DNS, Email, Labs – you’ve seen the references, you’ve read about them and you know what those first two pieces of our business are, but you might not know too much about Labs.
Labs is our technology playground: our virtual sandbox of projects, our Skunkworks. It’s the place where we do things to help make the Internet a better place for everyone. And up until now, it’s been one of the more “virtual” departments of Dyn, but that all changes today with a new addition and a familiar face that will help us form and re-ignite the Labs department.
Read More12.12.2011 By Tom Daly
SOPA: Why Do We Have To Break The DNS?
Last month, we posted our position piece on the Stop Online Piracy Act, also known as SOPA or the E-Parasite Act. In this post, I’m going to examine the technical details of the act and how it relates to the operation of the global Domain Name System (DNS).
SOPA proposes the idea of using DNS-based filtering by Internet Service Providers (ISPs) as a means to remove U.S. support of a foreign infringing website.
While the bill doesn’t specifically define how the ISP should technically go about this, it does seem to indicate that an ISP should capture, redirect and modify DNS query / response pairs to ensure that a downstream user does not access the site. There’s a number of ways to “remove support” from a foreign infringing website at the DNS level, so we’ll take a look at the techniques that could be used at all the layers of the DNS and why some are more destructive than others.
Read More12.06.2011 By Tom Daly
Worldwide DNS Infrastructure Upgrades Continue: Australia, Hong Kong, Dallas
Back in January, Dyn’s Operations Team was given a monumental task: perform a series of infrastructure upgrades to Dyn’s global anycast DNS network without causing any downtime or degradation of service for our customers. This meant upgrades to each of our 17 anycast data centers – new routers, switches, servers and supporting gear.
I’ve already blogged about our significant upgrades to our US infrastructure, so now it’s time to talk about our efforts overseas.
Read More12.02.2011 By Tom Daly
Observed DNS Anomaly: Bumps in DNS ANY Query Activity
For the past 5 days, we’ve been seeing a 20-25% bump in DNS QPS for random 5-10 minutes intervals that started back up this morning. It’s certainly not a pattern that we’re accustomed to seeing, so we classified this as a traffic anomaly and sent our Operations Team searching for answers on what the source of this might be.
Here’s the particulars of what we’re seeing:
Read More11.16.2011 By Tom Daly
AllOps: Evolution From The DevOps Culture Movement
For those in the software engineering, system administrator and generalist IT fields, you may have heard about a culture shift known as DevOps.
Designed to break down the interdepartmental barriers that exist between business technology divisions at strategic and tactical levels, the tactics behind DevOps bring together traditionally partitioned IT resources, such as software engineering and development, with IT operations, including system administration and infrastructure, by enhancing communication, collaboration and integration. The long-term strategy yields a more productive IT organization that is able to deliver more business value throughout the company.
For too long, IT organizations have been plagued with the blame game, which is what the concept of AllOps aims to eliminate. Tell me if you’ve heard of or been part of one of these scenarios before…
Read More11.09.2011 By Tom Daly
Evaluating The Growth Of Internet Traffic
At the opening of NANOG 53, Kevin McElearney of Comcast commented that within Comcast Regional Area Networks (CRANs), the company is regularly pushing 40+ Gbps of traffic out to the global Internet. This is a massive amount of traffic and in many cases, it’s more traffic than entire countries around the world push out to the Internet.
It got me thinking about just how much traffic there is on the Internet and the rate at which that traffic will grow over the coming years.
Read More11.07.2011 By Tom Daly
Post Mortem: Attack To Dyn Standard DNS Nameservers
15:03 UTC: the Dyn Operations team was notified of an issue with Dyn Standard DNS nameservers. The team then immediately began investigating the issue and identified it as a Distributed Denial of Service (DDoS) attack against all five Dyn Standard DNS nameservers. Compounding this issue was a series of wide scale Internet stability issues caused by a software bug in a major networking vendor’s routing code, which affected BGP routing for the a good majority of the Internet. This added complexity in identifying the DDoS vector, ultimately delaying our efforts to begin mitigation of the attack.
15:20 UTC: the nature of the attack was identified and our DynStatus site was updated. Operations began deploying our well-practiced DDoS countermeasures and mitigations. At 15:40 UTC, a majority of Dyn Standard DNS nameservers were offline due to complete exhaustion of server resources attempting to migitate the attack. At 16:10 UTC, all Dyn Standard DNS nameservers went offline as server resources were completely exhausted.
16:32 UTC: our ns2.mydyndns.org nameserver returned to service, protected by a variety of anti-DDoS mitigation systems including router ACLs, firewalls and DDoS scrubbing devices. At 16:50 UTC, the ns3.mydyndns.org nameserver returned to service. Due to complexities of fully reloading edge nameservers, it took until 17:50 UTC to return ns1.mydyndns.org and ns4.mydyndns.org to service. Finally, ns5.mydyndns.org was back in service at 18:15 UTC.
An additional complicating factor was that our DynStatus site became overwhelmed with traffic at 16:30 UTC. At this time, we opted to use both Twitter feeds to communicate with our users (primarily @DynDNS and @DynInc) while we altered the configuration of the DynStatus site to handle additional load. At 17:23 UTC, the DynStatus site was online again.
So today, for the first time since 2001, we experienced a full 22 minute outage of our Dyn Standard DNS nameservers, which means that we reset our Dyn Standard DNS uptime counters back to zero. For that, we’re disappointed and we apologize to our customers that were affected by both the outage and the hiccup with our DynStatus site that prevented us from communicating to the extent that we wanted to do. We believe that transparency is critical in keeping our customers informed and will be taking efforts to harden our DynStatus site to ensure it is always available, even if or DNS servers are not. We appreciate your decision to use our services and we thank you for the patience during this issue.
As Dyn is constantly dealing with DDoS attacks, we have a tradition of naming them similar to the way hurricanes are named in the US. Today’s event was named Fiona. Attached to the name is a post mortem analysis of the event to identify the area of weakness in our network and systems, so that immediate improvements can be made. That process has already started.
For customers utilizing Dyn’s DynECT Managed DNS platform, served from 17 global datacenters, no issues or outages were observed during the course of the event.
Read More11.07.2011 By Tom Daly
Snowtober: Managed DNS, Email Delivery Aren’t Much Different than Power & Water Utilities
Just over a week ago, the northern New England corridor was hit with a historic October Nor’easter, duly named “Snowtober”. On Saturday, October 29th, more than a foot of snow was dropped across southern New Hampshire and northern Massachusetts, causing numerous power outages, road closures and general discomfort for residents throughout the region.
When the power goes out, the ramifications are endless. Pumps cannot be run to supply water, furnaces cannot run to supply heat and refrigerators cannot run to keep food cold and/or frozen. When the water supply is impacted, it prevents people from cooking, cleaning and bathing, basically meaning that general human hygiene can be affected.
The interesting part to me is that these basic functions are rarely thought of, except during a utility outage like the ones caused by the Snowtober Nor’easter. For the most part, most people simply expect power to be on and water to be flowing as these are utility services that are provided to the home and one really doesn’t think about these services until an outage occurs.
The same parallel can be drawn to DNS and Email Delivery systems –- people don’t always spend time thinking about them until an outage or issue arises.
Read More