Internet Engineers Seek Path of Least Resistance

MPLS Offers Routing Pluses, but Is Still Work in Progress

Multiple Protocol Label Switching (MPLS), the technique of labeling packets and routing them along a predetermined label switched path (LSP), is being touted as a great way to avoid network congestion, run packets along the quickest or least expensive routes, and create virtual private networks (VPNs).

“One of the true promises of MPLS is the ability to create end-to-end circuits, with specific performance characteristics, across any type of transport medium, eliminating the need for overlay networks or Layer 2 only control mechanisms,” writes Irwin Lazar, who maintains an FAQ site on MPLS at the MPLS Resource Center (http://www.mplsrc.com).

But MPLS—as long as we’re showering you with acronyms—could stand for “Maybe Perhaps Later Someday” when it comes to full implementation in the industry. For the time being, Tier 1 carriers and network hardware manufacturers use MPLS primarily to create VPNs for large business clients or to manage network performance through more efficient routing. Mistrust still exists in the telecom industry.

Azhar Sayeed, manager of product management for MPLS and QoS technologies at Cisco’s IOS Technologies Division, says mass-market use of MPLS is still on the horizon. “It’s been slow in being adopted,” he says. “Service providers are saying, ‘We’ll wait to see how the technology is going to be adopted in the market before we look at it.’ A few years ago, if you had asked providers questions about MPLS, they would say, ‘No, we don’t want MPLS.’ So adoption has been slow, especially in the area of traffic engineering. And it had negative effects, because traffic engineering is such a good technology and a good way of routing traffic on your Layer 3 network. It really brings the flavors of Layer 2 traffick engineering up into Layer 3.”

What Is MPLS?

In an MPLS-based network, incoming data or voice packets are assigned a label by a label edge router (LER). Packets are then forwarded along a label switch path (LSP) where each label switching router (LSR) makes forwarding decisions based solely on the contents of the label. At each hop, the LSR strips off the existing label and applies a new label that tells the next hop how to forward the packet. In many ways, LSPs are no different than circuit-switched paths in ATM or frame relay networks, except that they are not dependent on a particular Layer 2 technology.

This is important because in traditional layering of say, IP over ATM, mapping between the two layers is complex and time-consuming. “With the overlay network, you have the problem of two different domains of addressing and network management systems on both Layer 2 and 3,” says Tony Bogovic, director of computer communications research at Telcordia. “That means additional training and field support to maintain that overlay network. In MPLS, they are essentially integrated. As a result, your operational costs are reduced.”

Routers between traditional IP and ATM overlays have to be “meshed” or interconnected so packets can leap efficiently between the layers. But the network can suffer what is known as hyper-aggregation if something goes wrong between the layers and connectivity is lost. “When the ATM layer is fully meshed with the IP layer,” Bogovic says, “the routers want to be one hop away. The ATM layer gives that connectivity by allowing each router along that periphery to be one hop away from any other router along that periphery. If any of the routers fail, control information is sent to all the routers, so you get to the point where your routers are processing only control information, and not forwarding packets, or processing data.”

Using MPLS To Set Up VPNs

The most popular use of MPLS technology is in creating VPNs for corporate clients, and for routing data and voice packets to the shortest path available that has the available bandwidth, a key tool in providing different levels of quality of service (QoS). In traditional IP transmissions, the packets will find the shortest route possible to the termination IP address, regardless of where traffic is heaviest. It will eventually (in seconds of time difference) reach the intended IP address. For instance, it may not matter to a service provider’s corporate customer if the client’s e-mail takes a minute longer to reach its office complex in Cincinnati. That customer is willing to accept a “best effort” level of service.

But if you have a VPN built around MPLS, corporations have learned, you can create a faster, more secure path for more proprietary intra-company voice and data traffic—whether you’re sending it from New York to Los Angeles or to offices overseas. “Service providers have seen an immediate benefit in building network-based VPNs, and the adoption has been much faster now,” Cisco’s Sayeed says. “They are offering network-based VPN capabilities to the enterprise customers—particularly those customers that want global connectivity.

“And they don’t want to be setting up these permanent virtual circuits [PVCs] globally or on a very large scale. If you look at the early adopters of MPLS/VPN, they’ve had very good success marketing to multinationals or national players. It’s easier to provision; they can put in a new site in a matter of hours instead of a matter of days or months. Provisioning is no longer an issue.”

MPLS Versus ATM or Frame Relay

Provisioning an ATM or frame relay VPN is more labor-intensive, Sayeed argues.

“If you have a frame relay or ATM and you were trying to connect the three sites together, you would basically have a PVC that connects all three sides,” he explains. “You would have a PVC from Site A to Site B, and a PVC from Site B to Site C, and a PVC from Site C to Site A. You end up creating ‘n-squared’ number of PVCs. If you want to add a new site, Site 4, then you need to have a PVC from site A to Site B, Site B to Site D, Site C to Site D, and you have to completely mesh up to have proper adjacency. Otherwise you’ll always end up routing traffic via another site, which might not be desirable.

“If you build a network-based VPN using MPLS, then the VPN is actually built into the network. The network has knowledge of the VPN traffic, and where it’s coming from and where it’s going,” because the label edge router introduces the first labels to the packets. “So if you want to add the new site to this VPN, all you have to do is configure the new site and make the new site a member of the VPN. Nothing has to be pre-provisioned; you just go configure that new site and connect the port up.”

How does that MPLS network determine where there is no traffic congestion?

“Obviously this can’t be done by itself, without the router knowing where to put this label switched path,” Sayeed says. “In order for the router to know where to put this label switched path, the router needs to understand what amount of bandwidth is available on different links. So the interior gateway protocol [IGP] has to be extended to not only advertise the link availability, but the IGP needs to be able to advertise the available bandwidth on those links. And then the router can do some kind of constraint-based computation that says, ‘Eliminate all the links for me that do not have the bandwidth that I need.’ This is supported today in MPLS traffic engineering. ”

MPLS Versus IP Tunneling

Many providers use tunneling, the art of building virtual paths using IP Security Protocol (IPSec). The technology—under continuing development at the Internet Engineering Task Force (IETF), the IPSec Forum and other industry groups—is a security protocol in the network layer that provides cryptographic security to support combinations of authentication, integrity, access control and confidentiality. It is being used widely in the IP world and is designed to create a single, secure path or “tunnel” that can be used only by a single customer.

IPSec, however, has scalability limitations, according to Sayeed. “You can use IPSec to encrypt tunnels from your edge devices or from your customer’s edge routers, but … you can only support one tunnel per pair of sites or end points--and per-user in the dial integration environment. In network-based VPN you’re not building one label switched path per user, you’re probably building one label switched path per VPN. You can now potentially build hundreds of thousands of VPNs, with thousands of sites per VPN, and you wouldn’t run into any major scaling issues.”

Devil Is in the Protocols

Remember that MPLS is not ubiquitous across the Internet; that its use is limited so far to closed networks owned by large ISPs and such carriers as Cable & Wireless, AT&T, and WorldCom; and that it’s just beginning to emerge as a tool for Tier 2 providers. Though MPLS is considered a great way to mesh, or integrate, packet traffic between Layer 2 and 3 across different platforms, MPLS standards are still being worked out in the IETF and at various universities. Cisco and other hardware and softswitch manufacturers are running interoperability tests with carrier partners, throwing research into various signaling protocols, such as interoperability of Resource Reservation Protocol (RSVP) traffic engineering and so forth.

What happens, for instance, if your MPLS network needs to query servers in the traditional IP cloud? You have different protocols and applications that need to communicate and interact.

Do you use something like Network Address Translation? Or some of the other services, such as High Speed Resource Reservation Protocol? “These types of applications were once running in a corporate VPN environment previously using Layer 2 to provision such networks, such as frame relay or ATM, so it was transparent,” Sayeed says. “Now when you’re building a Layer 3 VPN using MPLS, these applications have to be enabled to work with Layer 3.”

In other cases, a router that’s sitting out on the edge of the MPLS cloud needs to be able to transmit a dynamic host configuration protocol (DHCP) query across, because the DHCP server is sitting at another site. “These are the areas that need a little bit more work, especially in Web caching protocols, that need to be VPN-aware,” Sayeed says. “If you want to do Web caching and you have Web content spread all over the place inside of a VPN, maybe you want to be using a Web caching control protocol (WCCP) enabled on the provider router, and that must be then VPN aware, because it needs to know where that content is going. So it’s in these areas that there is room for improvement and there is room for progress.”

Adoption of MPLS traffic engineering has also been slowed by the fact that there are two sets of standards issued by the IETF: RSVP-based traffic engineering and Constraint-Based Routing Label Distribution Protocol (CR-LDP). “Adoption has been a little bit slow, especially in the area of traffic engineering,” Sayeed says.

Management Platform Integration Is Weak

“Providers don’t make a service a service until it’s ready, until they have all the data and they’re ready to bill,” Sayeed says. “To that effect, a lot of packet integration still needs to take place. Equipment like routing devices and switches can provide only broad data in terms of what type of packets [are on the network] and where they are going, what business is defined and what’s not defined.

“The packet integration of management platforms with OSS and billing capabilities is a little weak … because the amount of data that’s available from the network event wasn’t the way people want it,” he says. “Whenever you build a service, you want to be able to bill for it, and if you can’t bill for it, it’s not a service. They are not able to get that information to the back office very well.”

This doesn’t mean that no one’s able to do it yet, Sayeed says. “What I’m saying is, it’s not ready for mass market. There will always be those early adopters and those providers who have their own homegrown systems with respect to back-end integration and back-end OSS billing inquiries, and have gone ahead and deployed and are offering their customers services based on QoS and MPLS, and are finding ways to bill for it.”

QoS and MPLS Interoperability

“We’re trying to solve a little different problem with QoS,” says Tony Zeis, CTO of CoreExpress. “The rest of the industry is trying to solve QoS within their own network, for their own closed groups. Our interest is how we solve IP QoS between networks and their many implementations. Our challenge is to define an MPLS QoS standard that operates with all kinds of edge ISP networks and their implementations of QoS.”

This is a tall order. Because QoS is not standard across the Internet, packets run across networks using various kinds of routing and switching hardware and myriad service quality determinants. In essence, it’s a free-for-all of service level agreements and QoS measurements.

“MPLS is one kind of QoS you can implement,” Zeis says. “Within IP Type of Service, you have IP precedence, about eight levels of quality of service; in DiffServ, you have about 64 levels of quality of service. Then within MPLS, you have two implementations of MPLS for QoS: Experimental, eight levels you can define; [and] Label Interpretation, which has 2 to the 20th power, or about a million different levels of QoS.”

Network providers, for simplicity reasons, usually designate four levels of QoS, but may have different ways of coding or defining them. “Although IP DiffServ can have 64 levels of quality, versus a million levels,” Zeis says, “how would you differentiate QoS between service level 1 and level 2, versus level 62 and 63? Providers usually choose a few numbers of levels of QoS and prioritize the traffic within those levels of service.”

Latency Versus Packet Loss

Some fear that label switching can create latency in packet processing, but MPLS is designed to decrease latency in packet traffic. “At the core of the network you swap one label for another,” Sayeed says. “The label tells the packet where to go next. The imposition/disposition and swapping functions are not very intensive functions in terms of either the CPU or mapping. It really doesn’t introduce a latency.” Routers can stack five or six labels at a time, or remove five or six labels at a time. “Adding four bytes or adding five labels is a very small amount when compared to the average Internet package of 512 bytes,” Sayeed says. “That’s very minor. In fact, by doing just label switching at the core, you actually make it much faster, because you’re not doing full IP routing.”

Packet loss, however, is a different matter. Packet loss can occur when packets hit those networks with various QoS standards and service level agreements. “The variability of latency isn’t as important as the amount of tolerance of packet loss,” Zeis says. “The application’s sensitivity under guaranteed bandwidth QoS implementation allows it to accept variability within latency, but cannot tolerate loss of packets. Customers will have a challenge to map their applications into QoS levels. It requires an access control list at the router that would look at anything coming in. I can see an ISP that would have a template, and know that at port number 23, telnet might get mapped into ‘best effort,’ or ‘low latency,’ measurements of application performance sensitivity.”

But QoS distribution is still easier in an MPLS-based network, Zeis says. “In normal non-MPLS router addressing, you might have a little over 100,000 router entries; you may have to make a decision between 100,000 places to send it. The benefit of MPLS is that it uses less memory, fewer routing entries, and can switch packets more quickly. In the core of the network, it doesn’t look at the IP address, just the label. The second thing it can do is look at the incoming QoS, read it the right way and assign it to the right queue.”

“There have been extensions to MPLS to actually go look at the class of service in the IP and maybe map it into the label value itself, rather [than] just the bits in that label header,” Sayeed says. “Which means if you distribute a label ‘20,’ it’s a higher class of service than a label ‘40.’ If you do this, you have a much larger capability than IP to do more classes of service. Though every label can now be treated differently, you need to have the hardware to handle the thousand or two thousand queues to be able to do something like that. Most deployments today, however, need 3-4 classes of service, which can be easily provided by the current MPLS implementation using the MPLS EXP field.”

Are Optical Systems Next?

As software engineers and carriers run interoperability tests and otherwise tweak MPLS for managing networks and building dedicated packet networks like VPN and others are looking for ways to use it to manage optical networks. “The idea is to extend it for the control of multiple switches,” says Andrew Malis, CTO for Vivace Networks and a member of the MPLS Forum. “MPLS would be used to set up switches at Layer 1, controlling the allocation of the wavelength when you do dense wavelength division multiplexing. It puts more traffic on multiple fiber by using different wavelengths of light. It’s in standards groups right now. It will be a year or two before you see optical products based on MPLS.”

Maybe, Perhaps, Later, Someday, may come sooner than we think—once standards are settled and the majority of service providers are convinced the industry is adopting the technology on a larger scale.