There was minor consternation in Internet engineering circles today, as the number of IPv4 networks worldwide briefly touched another magic “power of 2” size limit. As it turns out, 512K (524,288 to be exact, or 2-to-the-19th power) is the maximum number of routes supported by the default TCAM configuration on certain aging hardware platforms.
The problem is real, and we still haven’t seen the full effects, because most of the Internet hasn’t yet experienced the conditions that could cause problems for underprovisioned equipment. Everyone on the Internet has a slightly different idea of how big the global routing table is, thanks to slightly different local business rules about peering and aggregation (the merging of very similar routes to close-by parts of the Internet address space). Everyone has a slightly different perspective, but the consensus estimate is indeed just under 512K, and marching higher with time.
The real test, when large providers commonly believe that the Internet contains 512K routes, and pass that along to all their customers as a consensus representation of Internet structure, will start later this week, and will be felt nearly everywhere by the end of next week.
Enterprises that rely on the Internet for delivery of service should pay close attention to the latency and reachability of the paths to customers in the coming weeks, in order to identify affected service providers upstream and work around them while they perform appropriate upgrades to their infrastructure.
Putting This Event in Perspective: Don’t Panic
It’s important to put this all in proper perspective (and yes, friends from the media who cover Internet infrastructure issues, I’m especially hoping you read down to this paragraph).
This situation is more of an annoyance than a real Internet-wide threat. Most routers in use today at midsize to large service providers, and certainly all of the routers that operate the core infrastructure of the Internet, have plenty of room to deal with the Internet’s current span, because they were provisioned that way by sensible network operators.
Affected boxes cause local connectivity problems for the network service providers who still run them, so they will be identified quickly and upgraded as we pass the threshold. Their instability in turn causes some minor additional load on adjacent routers.
But the overall stability of the global routing system should be unaffected. In terms of a threat, this isn’t nearly in the same class as some poison-message scenarios we’ve described before, which combine router failure with contagion dynamics.
Origins of the Problem
This has been coming for some time. The Internet keeps growing, which is what it does best. There’s very little indication that the current shortage of IPv4 space has done anything to dissuade new autonomous systems (enterprises, universities, service providers, etc.) from connecting to the Internet and expecting to route some space of their own.
Ironically, exhaustion may be speeding up the growth, as enterprises and service providers learn to use tricks like carrier-grade NAT to get their jobs done in tinier and tinier fragments of the remaining IPv4 space.
And that means that 512K is right around the corner for everyone on Earth, as early as next week. Here’s a plot of the distribution of routing table size, marching forward, from May 2014 (red) through July 2014 (purple) and up to today (blue). This wave only propagates one way. Someday, sooner than you think, we’ll be facing the 1024K routing table challenge.
What Comes Next
This event won’t be over tomorrow; in fact, it has barely begun. As the routing table size distribution creeps to the right, the number of routers in the world who “see” 512K+ routes will steadily increase. Within a few weeks, nearly every piece of vulnerable gear will have been discovered, as 512K+ becomes the global consensus opinion. We don’t know how many machines that represents, and we don’t know what the net impact will be on local Internet connectivity before it all gets sorted out.
There is irony lurking here, of course, if you read the advisories. You can change the default configuration to reclaim more TCAM for IPv4 .. but only at the expense of support for IPv6, the “next generation” Internet addressing scheme that continues to struggle for widespread adoption. Sadly, this elderly gear was shipped at a time when the world was full of hope for the emergence of a real, live, flourishing IPv6 routing table. There’s far too much TCAM alloted to IPv6, as a result (in at least one case, 256K routes, when the current IPv6 routing table still requires fewer than 20K).
You can reclaim most of that precious router memory for IPv4, and you’ll be fine again .. at the expense of evicting your IPv6 routes from TCAM. That’s probably a decent bet, since anyone who failed to future-proof their deployment and is still running this older gear probably has very, very little IPv6 traffic on their network anyway. For IPv6 aficionados who are are tracking the continuing growth and robust good health of the “legacy” IPv4 Internet, that’s called “cold comfort.”