Bringing the idea to fruition took cooperation from the 11 companies that make up the Internet backbone, as well as recruiting some big-name sites such as Amazon.com as customers. But over the past four years, InterNap (Seattle) appears to have pulled it off.

Several companies, such as AboveNet Communications Inc. (San Jose, Calif.), use co-location facilities to link Internet service providers (ISPs) and particular Web sites in a "one-hop" fashion. InterNap is unique because of its relationship with all 11 Internet backbone carriers, chief executive Anthony Naughtin said.

Traffic across the Internet passes through multiple backbone carriers, such as UUNet or Digex, which forward the signals to one another for free under a system called peering. It's this system that makes the connectionless Internet possible, allowing messages to bounce from node to node en route to their destinations.

InterNap offered to change that by feeding traffic directly between the content provider and the appropriate network backbone. Customers such as Amazon.com have links directly to one of InterNap's private network access points (P-Naps), each of which connects to all 11 carrier backbones. When a customer sends a request to Amazon, the carrier detects it and shunts the message to the P-Nap, which in turn forwards it to Amazon. The reply from Amazon also goes to the P-Nap, which ideally will send the message using the customer's own backbone carrier, avoiding any peering.

With 20 P-Naps in place, InterNap has become the largest paying customer of the backbone carriers, Naughtin said. In turn, InterNap sells its service to content providers such as Amazon and the Nasdaq stock exchange.

InterNap minimizes the number of hand-offs required from backbone to backbone. "It has the effect of collapsing the 11 Internet backbones into a single hybrid service," Naughtin said. Also, "Carrier ISPs like a system where they can drop the traffic to the next backbone at the earliest convenience, because then they don't have to upgrade their networks [as often]."

In fact, InterNap's infrastructure uses only OC-3 (155-Mbit/second) or even DS-3 (45-Mbit/s) connections to the various backbones, not the massive OC-192 (10-Gbit/s) routers being installed at the network core.

The money issue is important because to date, no company is turning a profit at providing backbone connections. And InterNap itself is still losing money — the publicly traded company reported losses of $43.4 million on revenues of $22.5 million for the first six months of this year.

Optimal routes

InterNap pays carriers to configure certain routers with Local Preference Inbound, a proprietary derivative of the BGP4 routing protocol. It doesn't have to be ubiquitous — the protocol runs on less than 2 percent of the backbone networks, Naughtin said. This software recognizes traffic destined for an InterNap customer and forwards it directly to a P-Nap, with no peering.

IP routers send "advertisements" to the greater network, notifying one another of available routes. The P-Naps, each working independently, process that information through a software package called ASsimilator to determine best routes across the Net. The heavy engineering work comes in a part of ASsimilator called Cogitator, which chooses routes for packets in real-time — taking into consideration transient effects such as backbone bottlenecks and temporary line outages.

"You can't make those decisions independently because there's a fixed headroom on each backbone," said Oscar Stiffelman, senior software engineer for InterNap. That means the optimization has to consider not only the 85,000 address prefixes available to it, but also the amount of traffic going to each backbone at any given time.

Cogitator considers different routing options in terms of a cost function based on elements such as latency or packet loss, and runs an optimization algorithm to make its choice. Stiffelman was not willing to divulge details of InterNap's proprietary optimization techniques, which combine original research with known schemes for handling similar problems.

Determining the best path is probably an nP problem — that is, it's likely no computer algorithm could solve it in polynomial time. But InterNap isn't trying to solve the problem mathematically; Cogitator is more an engineering effort, Stiffelman said.

"This is a new area of computer science," he said. "You look at a problem and find it's just intractable. But you struggle with it and find in practice that it's quite manageable."

InterNap's operations are controlled from Seattle, but the company is in the process of building an Amsterdam site, and a third center is likely in either Sydney, Australia, or Singapore, Naughtin said.