Today, we are incredibly excited to announce the production deployment of Traffic Director’s new geolocation map. This is the first ever geolocation map that focuses on carefully geolocating DNS server IPs via a combination of approaches, rather than simply applying a commercial geolocation product. By more accurately geolocating DNS servers, we’re helping improve the performance of our customer’s Internet presences and the Internet at large.
Let’s get into the problem of geolocation, why this improved map is a significant step towards solving these problems, and how we achieved a significant improvement in our geolocation accuracy.
The Problem of Geolocation Accuracy
We launched Traffic Director just over a year ago and it has already become the fastest growing advanced service in Dyn’s history. Yet we knew we could make Traffic Director better, particular when it came to its geolocation accuracy.
That’s because geolocation of an IP is a hard problem even when it comes to normal end-users. The Internet, by definition, is in a constant state of change and this is especially true of IP addresses. There is no authoritative, agreed to, source of truth for geolocation data of IP addresses or for how the changes are tracked. This is a problem for several reasons:
- IP blocks change hands all the time, especially with IPv4 addresses becoming more scarce. This could be caused by normal business transactions, companies going out of business, and other events.
- IP space is moved around internally as companies and their infrastructures evolve.
- Internet provider outages cause traffic patterns and IP announcements to change, effectively changing the location of a particular IP block.
- In many cases while a company has IP blocks being used in different locations, their IP space is registered to a single location (like their corporate headquarters). Great examples of this include companies like Comcast (Philadelphia) and Google (Mountain View) whose geolocation data in most databases are often represented as being their respective corporate headquarters, not where it’s actually located.
When it comes to DNS and other Internet Infrastructure, it becomes an even harder problem because none of the commercial geolocation providers specialize in analyzing Infrastructure IPs. Many commercial providers rely on data of end users as one of their data sources for improving their geolocation of IP space that ISPs use for residential and commercial customers. Since infrastructure IP space itself has no end users (after all, it’s a bunch of routers, switches, and servers), that IP space is underrepresented when it comes to geolocation data.
All of this conspires against Internet Performance. When the geolocation of an IP block is incorrect, for instance Europe instead of the U.S., companies and their customers feel the impact with increased latency, sometimes significant latency. That is something we must avoid.
Building an Improved Geolocation Map
When we acquired Renesys earlier this year, we gained access to the many techniques Renesys pioneered for improving the geolocation of their own products that have been in existence for over a decade. Since Dyn has an amazingly diverse and large customer base, we have one of the best sources of Authoritative DNS traffic data in the industry. Our new colleagues were extremely excited to get their hands on our data and see what they could find out.
We initially constructed a list of millions of traffic sources that send DNS queries to our network on a daily basis. Early on, we discovered that less than 10% of the DNS servers that query our network generate over 90% of the DNS traffic we see. This was quite the eye opening statistic! By narrowing down our initial list of sources to those that generate the most traffic on our network using data taken from several different time periods, we greatly reduced the amount of data that needed to be analyzed from millions to hundreds of thousands of sources. Even this type of data crunching was not trivial and took an enormous amount of time: we had to make sure we were targeting the right sources of traffic.
Utilizing Dyn Internet Intelligence (DII) and its worldwide network of 150+ data collectors embedded within ISPs, we performed an intensive analysis of DNS servers that was part of our focus group. This utilized techniques we have developed over the years to help pin down where Internet infrastructure is located. Everything from latency measurements, where anycast routed the query to DNS, where peering takes place, and many other techniques we developed over the years were used to build evidence and eventually determine where each DNS server is likely to be located. It was an intensive effort that took months to complete by our data analysis and research teams.
For those DNS servers that originate 90% of our DNS traffic, we were able to improve our geolocation accuracy by over 20% compared to other commercial geolocation providers we analyzed! To further quantify these improvements, we’ve measured a 25ms median latency improvement for requests involving these corrected IP addresses. No one in the DNS industry has a geolocation map specifically built for DNS traffic in mind. We’re the first.
Customers do not need to make any changes as this improved geolocation map is now fully integrated into all existing and new Traffic Director instances. Additionally, we’re not done: this is just the beginning of improvements and new functionality we will be adding to Traffic Director over time thanks to our data analysis capabilities powered by DII and other technologies.
If you’re a customer utilizing our older advanced services such as Traffic Manager and Active Failover and would like to learn more about Traffic Director, you can reach out to your Account Manager or our Sales team for more details. Our support and sales engineering teams are also on standby for any questions or implementation details you may have.