market intelligence

Getting on the Internet

The world of the Internet bandwidth wholesale provider is much like the Internet itself: massive, lucrative, and potentially dizzying in its complexity. A fundamental question is, Who is your client and what does your client want?

First, you have the traditional Internet service provider (ISP), which is focused exclusively on providing Internet access to consumers or businesses either locally, regionally, or nationally.

Next, you have the traditional telecommunications-oriented companies that have purchased ISPs and integrated them into their businesses. As if in retaliation, some of the ISPs want additional bandwidth so they can sell or resell telecommunications services.

Then you have people coming into the ISP business who have never been in the communications or the telecommunications business. They have a vertical-market orientation that somehow includes Internet services.

Local loop de loop
Once it has selected a national backbone provider, an ISP then must determine how to get its traffic to the backbone. Because the Federal Communications Commission has not regulated the data-communications industry, ISPs that use the local telephone companies to link to other providers are exempt from paying access charges, making the incumbent local-exchange carrier a tempting choice for this role. But as other large business users have found, this road to the backbone is not without its bumps for ISPs.

"Most of the challenges don't come from the network service providers like GTE or MCI or Sprint, they come from the RBOC [regional Bell operating company]," says Mike O'Connor, a cofounder of go.fast.net Inc., an ISP in St. Paul, MN. "We are very reliant on US West for the transport between us and our service provider, and we have the same problems that anybody who is a big network consumer has: Things break, they aren't fixed as fast as they ought to be, the technical folks don't understand the problems. There are always service issues that are involved. There are often facilities issues, just because a lot of the time it is hard to get [fiber] pulled in any reasonable amount of time."

That said, careful planning can make regional transport more efficient. For example, the founders of go.fast.com established their company's facilities between two fiber loops that US West runs out of separate central offices. "It's really easy for us to do redundant SONET ring kinds of interconnections because we can go in one direction and get on one loop and go in the other direction and get on another," says O'Connor.

O'Connor plans to keep a wary eye on its local partner, particularly as Washington contemplates deregulation of the RBOCs. "One of the things that we are very concerned about here in Minnesota—in fact, we had a lawsuit about it—is congestion on US West's interoffice ATM [Asynchronous Transfer Mode] network, which we also ride," notes O'Connor. "In my case, the glass goes from my building to the loop, then it goes to the central office, and then it rides ATM interoffice over to Minneapolis—and congestion on those interoffice networks is a very big deal to me. Because if US West oversubscribes those networks, oversells them, and gets them jammed up with way too much traffic, then I am in a big world of hurt when it comes to serving my own customers. And to completely deregulate that and to leave US West with nobody looking over their shoulder as to how they're doing on delivering service quality is a big deal to me. We have to sort of tiptoe into that because, right now, if they deregulated all that, US West can do what they used to do on the 'Laugh-In' show: 'Well, we're the phone company, we don't have to talk to you.'"

Concludes O'Connor, "There are some real interesting few years to go through yet in terms of price regulation, deregulation of data services, and the FCC has a really important role to play in that."

In such an environment, the requirements—and level of sophistication—for each of your customers can differ significantly. Of course, this entire mix ignores the fact that the wholesale provider could be in the ISP business itself, which adds even more spice to the bandwidth recipe.

With so many factors to balance, it comes as no surprise that a survey of national bandwidth providers reveals that there are both common approaches and divergent viewpoints when it comes to the best way to define and serve the market. It is also less than shocking to discover that when it comes to wholesale Internet bandwidth, supplier and customer aren't always on the same page.

Connect the dots

Approximately 30 companies in the United States act as Internet backbone providers. They provide connectivity to the four Network Access Points (NAPs) in the United States, located in San Francisco, Chicago, Washington, DC, and Pennsauken, NJ. The NAPs are interconnected to form the primary backbone of the Internet. ISPs typically buy or lease backbone capacity from either national or regional suppliers, who in turn may acquire this capacity by buying or leasing it from each other.

"Traditionally, [the Internet backbone market] has been dominated by Sprint, UUnet, and Cable & Wireless," observes Courtney Munroe, director of research for business network and extranet commerce services at International Data Corp. (IDC—Framingham, MA). "Now, you also have companies like Qwest, Level 3, and Williams. But I would say the top three backbone providers still carry 80% to 90% of the traffic. So we haven't seen a dramatic impact of these new providers yet. But as their networks are completed, we're probably going to see the market share of the top three decline."

In 1998, wholesale ISP revenues in the United States reached $2 billion, demonstrating 66% growth during the year, according to IDC. This year, the U.S. wholesale ISP market is expected to grow 52% to more than $3 billion. By 2003, those wholesale revenues will reach $6 billion, which represents a 25.2% compound annual growth rate for the five-year period (see Figure).

More bandwidth

This impressive growth derives from the ravenous appetite for pipes—a demand that occasionally catches even backbone veterans off guard.

"This year, we've been very surprised by the demand for OC-3 service," says Sue Birkenheier, director of operations for GTE Internetworking's wholesale division, ISP Direct (Irving, TX). "It's similar to the past, when DS-3s were very far and few between. You had to be a very large ISP to be looking for that kind of bandwidth. That picked up speed, DS-3s became commonplace—and OC-3s hit the market the same way. So we're seeing a great demand for that."

She adds, "The bandwidth requirement is increasing at a quick pace to the point where we are trying to think ahead to OC-12s and get ahead of the curve."

National providers have been adding a substantial amount of fiber to their backbone networks to meet this demand, notes IDC's Munroe. In addition, new service providers such as Frontier, Qwest, Williams, Level 3, and IXC Communications also are building national fiber networks. And traditional telecommunications companies such as AT&T, MCI Worldcom, and Sprint also have increased capacity by implementing wavelength-division multiplexing (WDM) in their existing fiber networks.

"I believe there is a lot of capacity out there. But you also have to look at it from a perspective of, Where is the demand?" says Munroe. "On any given day in some markets—for example, Washington, DC, to New York or San Francisco to Los Angeles—demand might outstrip supply. Then it will take a few weeks before supply catches up with demand. Those are normal supply and demand issues."

However, competitive aspects also can affect the availability of bandwidth from national providers, particularly those who function as ISPs themselves. "The ISP is both their competitor and their customer, so that can create tension," says Munroe. "Sometimes, ISPs say they can't get a DS-3. But there could be two reasons for that: On any particular route, there is no capacity. Or ISPs, for whatever reason, don't want to sell capacity to other ISPs, even though a corporate user may be able to get a DS-3. That happens all the time.

I hear ISPs complain that the big backbone suppliers don't want to sell them capacity. They say they don't have it and the ISPs don't know whether it is true or not."

Mike O'Connor, a cofounder of local ISP go.fast.net Inc., which serves business customers in St. Paul, MN, expects bandwidth demand to continue to grow. "By most measures today, the Internet backbone, even with more traffic, is faster than it was two to five years ago for the individual. And that trend, I think, will continue. What that means is just continued voracious demand for raw pipe bandwidth."

06fexconcept.gif

High demand, competitive pricing

Pricing in the ISP wholesale market is very competitive. A DS-3, depending on the link distance, can cost $20,000 to $50,000 a month, according to IDC estimates. An OC-3 costs from $125,000 to more than $200,000 a month.

"You should expect prices to decline over time for most bandwidth capacity," says IDC's Munroe. "For the highest speeds, it will probably decline, but a lot less, maybe 2% or 5% over the next couple of years per year. Traffic demand is increasing so fast, that on the top end—OC-3, OC-12, OC-48—I really don't see prices declining that much. And that's because there is a lot more DS-3 available, comparatively, than there are OC-48 corridors. So if you want an OC-12 or OC-48, you have to pay full price—there is very little discounting going on that I know of."

Pricing for dedicated access lines to business customers, on the other hand, are dropping every year by 5% to 10%. "It is getting cheaper for the big service providers to add capacity without having to put new fiber in; they can implement WDM," says Munroe.

"Pricing is fairly stable," observes Harry Lalor, director of ISP services at PSINet, Inc. (Herndon, VA). "It is going down in some places where the wholesale customer serves only their local customers. You'll see those prices dropping as more and more of the facilities-based providers get in a 'ports-and-pipe' game. But once you go beyond a single market, the need for services to accompany ports and pipes drives the price higher."

"There are newly formed competitors that are trying to build up their customer bases in an effort to build on their peering agreements, so they've low-balled some pricing out there," asserts GTE's Birkenheier.

"The competitive marketplace right now is bringing some pricing down, but I don't foresee continuing. I see it leveling off," she says. "I see it as a temporary competitive environment that isn't consistent across the country, either. It is street-level competitiveness. In many ways, you can see that some of this pricing is near or below cost; there's no way that it can continue in the long run. But I also don't foresee a great jump in pricing either."

"Because the demand for bandwidth increases so quickly, ISPs buy as much as they can afford," says Lalor, "which means that the different Internet access methods become key. The most expensive way to access the Internet—and also the most reliable and the most valuable—is a direct connection from the customer's site to the Internet network. What we are seeing is still a strong demand in growth, mostly in the in sub-T1 range to T1. But the fast growth is where you have customers going through a competitive local-exchange carrier's data network or an RBOC's [regional Bell operating company's] network, where they get economies on the access costs and then come to an Internet provider's backbone.

The upside there for the ISP is that their costs are lower on the access side; therefore, they are able to be more aggressive on their price to users. The downside is that the performance that they get from that access circuit is not as reliable—the customer's data has to first go through an RBOC frame network or cloud network that's shared amongst a number of other customers and then finally gets to an Internet backbone network."

Divergent market views

Internet backbone providers sell bandwidth capacity on their national networks to other national, regional, and local ISPs, and in many cases, directly to corporate (retail) customers. Many providers like GTE Internetworking, which launched ISP Direct in October after acquiring wholesale provider Nap.net, have developed wholesale sales and service divisions. With so many suppliers in the market, it's not surprising to discover that there are several differences of opinion on how best to serve the market.

"ISPs need a different level of support," says GTE's Berkenheier, who cofounded Nap.net in July 1995. "They are looking to buy purely on network performance, service-level agreements [SLAs], quality-of-service issues, on price, but also support. ISPs need to have a single point of contact and not get folded in with other business and consumer types of support mechanisms. That's what the wholesale offer is all about. And we found that to be highly successful."

But PSINet's Lalor has a different view: "Price matters most and then service delivery. And service delivery broadly defined as 'Do they have a network where I am able to get consistent and reliable service for my customers? Am I able to have insights and view how my customers' data is performing on this network?"

Lalor contends that very few ISPs actually make business decisions based upon SLAs. "It's hard to make an apples-to-apples comparison—what UUnet means by their SLAs compared to PSINet compared to Sprint," he says. "There is so much market confusion out there. The customers say, 'I'm going to take a leap of faith that Worldcom or UUnet is just as good as Sprint is just as good as PSINet. What are the other decisions that I need to focus on in order to drive my business forward?"

Lalor believes that ISPs would rather focus on service delivery than back-hauling traffic to the nearest NAP. "Most ISPs that we've seen are moving to outsource what is becoming for their business a commodity and focus on delivering services to their customers. They are transitioning their network, which they have run in the past, over to wholesale carriers who will run it for them and provide the bandwidth. What I mean by managing the network, broadly, could be, in dial-access sense, managing the capacity and provisioning of traditional modems and telling the customer when their capacity or their modems are reaching peak and how many users are on the network and how many hours they're using. So they could create new price plans, or perhaps they are missing a segment of the market that they could adapt their pricing and products in order to meet [the needs of that market segment]."

The bottom line, says Lalor, is that ISPs will reward providers that address their primary requirements. "The companies that the customer doesn't believe will manage their network will automatically not make the cut for their business decision. So if reliability and a high-quality network is something that is important to these customers, they restrict the number of companies to whom they're talking to for services, rather than drive home an SLA. Unless an SLA is really important for their application, like for example with a gaming network, then they definitely want to negotiate something."

International expansion can be another draw for ISPs. The ability to launch an Internet service in the United States and then expand it into other countries is starting to drive backbone decisions. "The most visible in the market is AOL," says Lalor. "AOL is driving wholesale carriers to expand their networks in Europe and Asia and is providing new opportunities for companies like PSINet to provide services to a company like AOL in those different countries. The benefit to them is they have a single account name or company that can work with them here in the U.S., yet deliver services on several continents. Very few companies can do that. In the wholesale market when you get beyond the regional ISPs, the international capabilities are key, which is why PSINet in the last year has invested so much overseas because we understood that very quickly."

Mike O'Connor of regional ISP go.fast.net has a clear set of criteria with which to evaluate potential providers. "It's almost a poker game," he explains. "In order to stay in the game, you have to match your competitors, and the national backbone players are especially in that situation. There's almost an entry minimum that you have to be able to do, and these days it's fiber and ATM and multiple connections to the backbone and carefully managed networks that avoid congestion. You just presume that, and if you don't have that, then you're not a national player. An ISP like me is picking based on customer service, support, flexibility, and price."

"Just like my customers are very interested in how well I'm connected upstream, concludes O'Connor, "I am interested in my upstream providers' connections. And my criterion is just like the criterion that my customers use: How reliable are they? How well connected are they? How much excess capacity do they have? Where does their cost structure fit?"

The desire to meet customer requirements doesn't always mean that backbone providers see eye to eye with their ISP clients. For example, as quality of service for the Internet becomes more of an issue, how accountable should the backbone provider be when there is an interruption or degradation in service? "A lot of ISPs have to say there is a problem with my provider's network," says IDC's Munroe. "In some cases, the ISP may actually have to sign an agreement where they can't name that backbone provider, and that's an issue. It makes the ISP look bad. It makes the Web hoster look bad.

"So there have been some problems trying to figure out this issue. It's in everyone's interest, but the reality is that [some providers] just don't care. They do care about the quality of service, but because they're not facing the customer, it's not their immediate problem. To a certain extent, so much of capacity is concentrated in so few hands, it creates a very tense relationship at times."

Another issue is peering. While backbone provider PSINet offers free peering, national providers increasingly place restrictions on free peering connections requiring multiple locations, minimum bandwidth levels such as DS-3, and comparable traffic. "The term 'peering' implies that you want to exchange traffic with someone who is your peer in terms of the amount of traffic," notes Munroe. Should the smaller providers pay, and what level of support should they receive? There is even discussion of setting up a rival peering consortium. The issue remains unsettled.

"For regional and local ISPs, it is a life-or-death situation as they grow their network or invest in their network locally," says PSINet's Lalor. "They are then shut out of the global network by companies that don't believe in the concept of free peering."

The future

IDC believes that Internet bandwidth demand will continue to grow, with voice-over-IP the "killer application." Most national backbone providers have convergence strategies up their sleeves; some will approach the issue through overlay networks, others have already embraced the concept of IP-only services. Most are testing voice-over-IP somewhere in their networks.

Yet, the road to convergence remains unpaved. "Voice is far more difficult to provide as a service than data," says Fred Harris, director of network planning and design at Sprint (Kansas City, MO). "The IETF has spent years trying to follow the impact of voice on IP, and they haven't been successful yet."

More traffic, greater demand for bandwidth capacity, and increased competition will all play a role in the wholesaler's future. As the market matures, mergers and acquisitions will continue to consolidate the backbone competition. And service will play a greater role in regional and local ISP offerings.