The NCAA tournament is the highlight of many basketball fans’ season. With its constant action, bracket upsets, and young kids playing their hearts out, the tournament was made for the digital age. While the first few rounds are chaotic and full of upsets, eventually, the cream rises to the top and the schools with the best, consistent performance are the ones in the Final Four.
But it is the upsets and chaos that make the NCAA tournament what it is, and that is why millions of people around the world fill out a bracket. It is why they religiously check that bracket as the tournament unfolds to see how they are doing. In the old days, this was done with pen and paper. Today fans go to websites to check their status. One of the most popular websites for filling out a bracket is ESPN.com.
Since the NCAA tournament is a global phenomenon dedicated to performance, we decided to check out how ESPN.com performs. Is Espn.com winning the tournament or are they leaving room for a competitor?
Using our Gauge tool, we looked at how long it took to reach ESPN’s website in the respective regions that are represented in the tournament bracket – East, South, Midwest and West. We wanted to know how long it took for people to check scores, watch highlights, and cry over their bracket. We looked at the week from when the tournament started on Thursday, March 16, until Wednesday, March 22 (prior to the Sweet Sixteen round). This data represent the experience of the users of Dyn’s Gauge tool, which makes it a limited sample, however, it does tell an interesting story.
|Region||Average Page Load Time (In Seconds)|
The midwest was the clear winner in page load time. Not only did Michigan (2 seconds) and Indiana (2.5 seconds) have great times, there were no major spikes in the other states, which allowed the average to remain consistent. The west performed very solidly, especially in California and Colorado.
While the average page load time to ESPN.com in the east region was 5.65, two states performed well above the average. Residents in basketball hotbeds, New York and Pennsylvania, only had to wait on average 3.7 seconds and 3.5 seconds to load ESPN. Fans in Washington DC, however, had to wait, on average, 9.5 seconds.
That length of wait was not unusual in the south region where residents in Georgia had to wait, on average, 11 seconds for ESPN.com to load. The south was saved by South Carolina, which had one of the fastest load page times in the country, at just over one second. There appeared to be an absurd load time in Alabama, however, we didn’t have enough users located in Alabama to make it statistically viable and so I did not include it.
|State||Average Page Load Time (In Seconds)|
These rankings will make basketball fans in each of the above states happy, as each of them has at least one team that has played in either the men’s or women’s NCAA tournament.
Why does the performance vary?
As shown above, states in the midwest seemed to have better performance to ESPN than other regions. The variation in performance to ESPN.com across regions and states is a common issue. Let’s use some of Dyn’s global internet intelligence to look through the various internet infrastructure choices that ESPN.com uses to host their website, as well as how end users connect to the internet through their ISP to see how these choices may impact performance.
ESPN uses Amazon Web Services as its cloud provider. Currently, they are using AWS Ashburn (Zone A, Zone C, Zone D, and Zone E) and AWS Portland (Zone A, Zone B, and Zone C). The II tool allows me to examine average performance to major urban hubs and so I was able to look at the Ashburn Point of Presence and see if it played a large role in our findings. Turns out it seems to provide solid performance to much of the eastern, southern and midwestern portions of the United States.
As I wrote above, it took, on average, 11 seconds to reach ESPN.com from Georgia but only 1 second from South Carolina. There was also fast performance in Michigan and Indiana. Yet these discrepancies were not obvious when looking at the performance of AWS Ashburn.
|Location||Performance (in Seconds)|
|Columbia, South Carolina||.025|
Here you see that Columbia performed the same as Atlanta, which actually performed better than both Detroit and Indianapolis. As you can imagine, the locations closest to both Ashburn and Portland performed the best, which is why the fact that the midwest won is surprising since most cloud providers have their Points of Presence on the coasts and actually don’t have a lot of proximity to the middle of the American continent. So if it is not the cloud provider causing the delay, what else could it be?
Like many companies that serve rich content, ESPN uses a Content Delivery Network (CDN) to get as close to its users as possible. ESPN uses Akamai, which is the largest and most globally distributed CDN in the world. After some examination, we can say that performance benefited across the midwest from the use of a CDN. This does not explain, however, why the other regions were so variable.
To gain some understanding on why this could happen, we now must look at the Last Mile, or the actual Internet Service Providers (ISPs) that our Gauge users were using.
By digging deeper into this data, you can begin to see the impact an ISP can have on a user’s experience. South Carolina was the fastest performing state based on the Gauge data based on the 78 times ESPN was accessed from South Carolina by Gauge users during the aforementioned timeframe. Those 78 visits were all done through Time Warner Cable. This would suggest that Time Warner has a fast connection in South Carolina.
Another of our top performing states – Indiana – also only registered one ISP. Lack of diversity works the other way as well. Some of the states that performed the worst (Arkansas and Alabama) also only registered one ISP. States that had Gauge users who used multiple ISPs were, not surprisingly, incredibly variable. The lesson being: ISPs play a large role in the end user experience. We previously mentioned that it took, on average, 11 seconds to reach ESPN.com from Georgia. The majority of those visits were done through AT&T (Comcast was another popular ISP in Georgia. Disney’s network (ASN 8137) (Disney owns ESPN) peers to the general internet through two providers, Level 3 and AT&T. ISPs with better connectivity to these providers might show better performance considerations than others.
This would appear to be a pretty direct connection. However, what you cannot see in this chart is the level of ISP congestion. Dense urban areas may seem to benefit to their proximity to key infrastructure hubs. However, the reason those infrastructure assets were built there to begin with – proximity to large populations – can also hurt individual users because they may be experiencing congestion at the ISP.
Gauge is a cool tool because it reports the performance of Real Users. Understanding the experience of real users is crucial for any business because it does not always play out the same as one may architect in a spreadsheet. A company can build its architecture well, which ESPN has done, but if a user has a bad ISP, the whole experience can be shot. And, more often than not, the user blames the company not their local ISP.
Let’s use a basketball analogy to illustrate this point. ESPN had a wonderful game plan and they executed throughout the majority of the game. But basketball is often won in the final minute and a team must play to the buzzer. The same is said of a company, except for companies the buzzer is their users. This is why it is so critical for companies to have end-to-end visibility of their entire digital supply chain. Only when you have an end-to-end view can you understand your customers and give them the experience they deserve from beginning to end.