Internet Performance Delivered right to your inbox

Tracking CDN Usage Through Historical DNS Data

The increased traffic levels that e-commerce sites see around holidays can potentially impact performance if the sites have not planned appropriately. Some sites have extra infrastructure headroom and can absorb increased traffic without issue, but others turn to CDN providers to ensure that their sites remain fast and available, especially during holiday shopping periods.

To that end, I thought that it would be interesting to use historical Internet Intelligence data (going back to 2010) collected from Oracle Dyn’s Internet Guide recursive DNS service, to examine CDN usage. As a sample set, I chose the top 50 “shopping” sites listed on Alexa, and looked at which sites are being delivered through CDNs, which CDN providers are most popular, and whether sites change or add providers over time. Although not all of the listed sites would commonly be considered “shopping” sites, as a free and publicly available list from a well-known source, it was acceptable for the purposes of this post.

The historical research was done on the www hostname of the listed sites, with the exception of store.steampowered.com. The site was considered to be using a given CDN provider if the hostname was CNAMEd to the provider’s namespace, or if associated A records belonged to the provider. For time periods before listed sites began using a CDN provider for whole site delivery as shown in the charts below, it is possible that they were delivering embedded content through a CDN, but that is not captured here. (Google.com is also on the Alexa list, but is not included below because it is not a shopping site, and because it is delivered directly from Google’s own infrastructure.)

In the interest of making the analyzed data easier to review here, the list of sites is broken out into several categories:

  • Brick & Mortar: the online presence of well-known retailers with physical stores
  • Online Native: retailers that primarily exist online
  • Automotive: car shopping sites
  • Video: media providers
  • Publishing: focusing on academic & research content
  • Nutrition: vitamin & supplement providers
  • Amazon: the .com and .co.uk properties, along with Zappos

Brick & Mortar

Looking at the chart below, it is clear that Akamai has an extremely strong presence within this set of sites, as a long-term CDN provider across all of them. Although Bed, Bath & Beyond and Forever21 previously relied on AT&T for CDN usage, they eventually transitioned to Akamai, likely as a result of the strategic alliance between the two providers. Among this group, Walgreens is the only site not currently being delivered by Akamai, having transitioned to Instart Logic in mid-2016.

Online Native

As businesses born on the Internet, one would think that they recognize the value of CDN usage, incorporating these services into their architecture from the start. However, in contrast to the online presence of brick & mortar retailers reviewed above, many of the “online native” sites have only come to rely on CDN providers in the last five years. (Except for one – iFixit is on the Alexa list, but not included here because we found no evidence of it being delivered through a CDN provider in our historical data set. However, it does appear to be using Amazon’s Cloudfront CDN service to deliver embedded page content.) Akamai has a strong presence among this set of sites as well, with a few exceptions. As part of its ongoing efforts to optimize site performance, Etsy moved to a multi-CDN configuration in late 2012, adding Verizon Digital Media Services/EdgeCast alongside Akamai, with Fastly joining the set of providers in early 2013. Humble Bundle has been delivered from Google since late 2010, although it is not using the Google Cloud CDN solution. Among this set of sites, Redbubble was the last to begin delivering its site through a CDN, waiting until early 2016 to integrate Cloudflare.

Automotive

Looking at AutoTrader, we see that its site has been delivered by Akamai since late 2011. CarGurus turned up CDN services from Verizon Digital Media Services/EdgeCast in early 2013, and shifted to a dual-vendor strategy with Fastly and Verizon in late 2015, before moving to use Fastly exclusively in early 2017. Cars.com held out much longer than its counterparts did, relying on origin infrastructure until activating Akamai’s services at the end of 2015.

Video

Sky and DirecTV have both been long-time Akamai customers, with Sky integrating the CDN services before the start of 2010, and DirecTV coming on board in late 2012. Netflix is well known as an Amazon Web Services customer, and its site has historically been hosted on Amazon’s EC2 service. Although not a CDN service, it appears Netflix used the cloud provider’s Elastic Load Balancing solution for a three year period between 2012 to 2015.

Publishing

The Oxford University Press site is found on the Alexa list, but is not included here because we found no evidence of it being delivered through a CDN provider in our historical data set. (The www hostname simply redirects users to global.oup.com, but there is no evidence of CDN usage, either for whole site or embedded content delivery, on that site either.) Wiley’s site was also historically served directly from origin, before shifting to delivery through Amazon Cloudfront in late 2017. However, Cambridge University Press has had a longer-term reliance on CDN usage, delivering through Akamai for three years, before shifting to CDNetworks for two years, and then to Cloudflare for the last two years.

Nutrition

Both sites in this category have had a long-term reliance on Akamai for delivery. However, iHerb briefly tested Amazon Cloudfront in early 2014. It also pursued a multi-CDN strategy with Akamai and Cloudflare starting in late 2015, before moving exclusively to Cloudflare two years later.

Amazon

Zappos is included in this grouping, along with Amazon’s US and UK sites, because it has been owned by Amazon since 2009. As seen in the chart, it has relied on Akamai’s CDN services since that time as well. In contrast, Amazon’s native sites have only been served through a CDN since late 2016, with www.cdn.amazon.com and www.cdn.amazon.co.uk being balanced between Akamai and Amazon’s own Cloudfront CDN solution.

Summary

In short, our Internet Guide data shows that Akamai has an extremely strong presence within top shopping sites, and for the most part, has held that position for a number of years. It also exposes the fact that more than sixteen years after CDN providers launched basic whole site delivery services, and more than thirteen years after dynamic site acceleration services were launched, there are still a set of shopping sites that are not taking advantage of those capabilities for performance and availability improvement, to say nothing of the security benefits from the WAF services also offered by these providers.

If you have other historical recursive DNS data research ideas for future blog posts, please comment on this post or e-mail us at internetintel_ww@oracle.com.


Share Now

David Belson
Whois: David Belson

David Belson is a Senior Director of Internet Research & Analysis at Oracle Dyn Global Business Unit, a pioneer in managed DNS and a leader in cloud-based infrastructure that connects users with digital content and experiences across a global internet.