Introduction
Since the early inception of packet-based networking, there has been a debate between switched architectures (Ethernet/Layer 2 functionality) that dominated local area networks versus routed architectures (IP/Layer 3 functionality) that dominated wide area networks. This debate is surfacing again when utilizing MPLS / MPLS-TP technologies at metro area networks, for the delivery of enterprise VPN, residential triple-play and mobile backhauling services.
Different types of enterprises require different types of VPN services Connecting remote sites is a critical telecom service for enterprises. Different types of enterprises (in their size, sensitivity to cost, existing equipment, etc.) require different types on VPNs (by their connectivity model, QoS, security, management and availability capabilities).
The Metro Ethernet Forum (MEF) is giving an example for different types of enterprises with their different needs and applications that Packet Transport Networks (PTN) can provide. This is being described in the following table:
|
Enterprise |
Needs |
Applications |
|
Finance |
|
|
|
Education |
|
|
|
Healthcare |
|
|
|
Government |
|
|
|
Media |
|
|
Table #1: Enterprise needs and applications (source: MEF)
MPLS basic mode of operation
Multi Protocol Label Switching (MPLS) is the de-facto technology for the delivery of VPN services in metro area networks. Virtual Private LAN Service (VPLS) and Virtual Private Wire Service (VPWS) became popular terms for VPN services that are based on MPLS technology.
MPLS is a field-proven, scalable, standard and interoperable technology, which was originally developed to provide a connection oriented and faster packet forwarding than the traditional IP routing. In an MPLS network, Ethernet packets are assigned with labels and packet forwarding decisions are made solely on the content of these labels, without the need to de-encapsulate and process the entire packets' contents.
Traditional routers forward packets by examining the IP header, searching for the longest matching entry in the routing table, and forwarding the packet through the specified interface and port to the next hop router. This process is consuming resources and time, and is repeated for each packet at each router along the path. Since no state is maintained from packet to packet, the system is highly scalable, but inefficient for large capacity transport purposes.
In an MPLS network, each packet encapsulates and carries its MPLS labels during the way from source to destination. High performance switching of data is achievable due to the fixed length labels that are inserted at the beginning of the packet and can be used by relatively simple hardware to switch packets.
How this is being done - a Label Edge Router (LER) assigns a label to each Incoming packet (also known as ingress). Packets are forwarded along a Label Switch Path (LSP) where each Label Switch Router (LSR) makes forwarding decisions based solely on the contents of the label. At each hop, the LSR strips off the existing label and apply a new label, which tells the next hop how to forward the packet.
Label distribution can be done using a standard LDP procedure. Labels are assigned according to multiple options, allowing flexible networking services such as VPN membership, hard QoS, traffic engineering, technology emulation, and more.
Layer 2 MPLS approach
Layer 2 MPLS is a carrier grade and connection oriented Ethernet network. When Layer 2 MPLS VPN solution is implemented, Ethernet traffic is encapsulated in MPLS frames and sent over MPLS tunnels.
Layer 2 is used both as a service and with MPLS as an infrastructure. As shown in figure #1, it is possible to offer a Layer 2 VPN service using Layer 2 or Layer 3 networking infrastructure.
From the enterprise perspective, Layer 2 VPN is an Ethernet network that connects remote offices.
From the service provider perspective, implementing Layer 2 VPN service over Layer 2 network (case "a") is intuitive, simple and cost efficient. However, implementing Layer 2 VPN service over Layer 3 network (case "c"), or in other words - emulating Ethernet over IP, is a complex solution with a much higher Capex and Opex.
Since the VPN is based on Ethernet, the enterprise is actually managing a regular LAN. Therefore, "how to get from point X to point Y" - is an issue that is being configured by the enterprise according to its internal policy. This is a preferred option for medium and large enterprises that have IT departments and manage their own networks.
Since the enterprise is getting an Ethernet network, there is a clear demarcation between the enterprise VPN and the service provider network.
Demarcation is separation in Layer 2 MPLS VPN
The clear demarcation is significant both for the enterprise and the service provider. From the enterprise's perspective, he is taking care for his premises gear and doing the administration for his own network. And from the service provider's perspective, he is taking care for providing connectivity services at the backbone.
This separation is reducing the service provider's OPEX and increasing the service uptime.
Figure #2: Layer 2 MPLS network - clear demarcation
Layer 3 MPLS approach
Layer 3 MPLS is a carrier grade and connection oriented IP network. When Layer 3 MPLS VPN solution is implemented, IP traffic is encapsulated in MPLS frames and sent over MPLS tunnels.
Layer 3 is used both as a service and with MPLS as an infrastructure. As shown in the figure, it is possible to offer a Layer 3 VPN service using Layer 2 or Layer 3 networking infrastructure.
From the enterprise perspective, Layer 3 VPN is an IP network that connects remote offices.
From the service provider perspective, implementing Layer 3 VPN service over Layer 2 network (case "b") is using Ethernet as cost effectively transport layer that inter-connects routers. In this solution, CE router will need to pass its traffic to a remote and centralized PE router.
Implementing Layer 3 VPN service over Layer 3 network (case "d") is intuitive step that is reflecting an All-IP approach.
Layer 3 MPLS VPN is a routed solution - In order to acquire reachability information about a given enterprise network, the PE routers exchange routes with the CE routers. Since the VPN is based on IP, there are no issues regarding "how to get from point X to point Y" - this is being solved automatically by the IP network. But on the other side, there is no clear demarcation between the enterprise VPN and the service provider network.
Unclear demarcation in Layer 3 VPN - there is no such thing "my/your problem" in IP network
Since the enterprise and the service provider are building together an IP network, there are preparations and decisions that are done by both sides, such as:
- Coordinating IP addresses and static IP
- Coordinating IP subnets
- Coordinating routing protocols
- Coordinating VRFs and route targets
- Coordinating security levels
This joint network is bringing maintenance and OPEX challenges. In practice:
- Add/remove enterprise's CE may cause difficulties
- Add/remove service provider's PE may cause difficulties
- Equipment upgrades may cause difficulties
The result is higher OPEX from the service provider, since service uptime and SLA assurance require professional and dedicated customer support.
Figure #3: Layer 3 MPLS network - flat IP network (unclear demarcation)
Solution comparison: reasoning for VPN offering based on Layer 2 and Layer 3 MPLS
Layer 2 MPLS and/or Layer 3 MPLS solutions should be considered according to current and future service provider's requirements based on the offered services, the existing infrastructure, and the costs involved.
Protocol transparency
Layer 2 MPLS VPN is based on Ethernet infrastructure and Layer 3 MPLS VPN is based on IP infrastructure. Therefore, Layer 2 VPN carries any protocol over Ethernet while Layer 3 VPN is aware to IP packet types. This is why Layer 2 VPN is considered as "transport" layer for the end-user protocols (such as Ethernet, IPv4, IPv6, IPX, DECNet, OSI, etc.).
This difference is having implications for IPv6 packets, security mechanisms and the selected control plane.
- IPv6: Layer 3 VPN might face compatibility issues with IPv6 packet types while Layer 2 VPN can interconnect IPv4 and IPv6 routers and deliver the service.
- Security: coordinating security levels should be done in Layer 3 VPN and this is not an easy task. In addition, in Layer 3 VPN the enterprise and the service provider share the same routing tables. Those issues are not relevant in Layer 2 VPN.
- Control plane: in Layer 3 VPN the control plane is determined by the service provider, this is a restriction to the enterprise's mode of operation. In Layer 2 VPN the enterprise is free to select his preferred control plane since the service provider is providing only transport.
Connectivity scenarios
Both approaches can be used to implement point-to-point services (such as E-Line or VPWS), point-to-multipoint services (such as E-Tree or Hun-and-spoke), and multipoint-to-multipoint services (such as E-LAN or VPLS).
White technically both approaches are feasible, it is important to note that:
- No need for multiple connections per site: In both options connecting multiples sites requires only one connection per site.
- Layer 2 is simpler: and therefore there is no reasoning to build an entire IP network for small VPNs or simple point-to-point connections.
- Multicast: unlike Ethernet bridging, there is no native multicast in IP.
- Enterprise perspective: "How do I connect site X with site Y": in Layer 3 VPN the enterprise should care about this question while in Layer 2 VPN the enterprise is managing his network and takes this decision. Large enterprises with their own IT departments may want to manage their VPN while smaller enterprises do not want to care about this topic.
Subscriber's equipment
The subscriber existing equipment may be a decision factor for Layer 2 or Layer 3 VPN.
- CPE: in Layer 2 VPN the CPE can be a router or a switch. In Layer 3 VPN the CPE must be a router.
- Interfaces: in Layer 2 VPN it is possible to use only Ethernet interfaces white Layer 3 VPN is capable to handle any interface. For example, POS interface requires router - Layer 3 VPN.
Carrier grade building blocks
- Scalability
Each solution has its own challenges when scaled into a large network.
Layer 2 - MAC addresses: In a Layer 2 solution, the maximum number of MAC addresses and Layer 2 forwarding table entries supported on a PE router may be a constraint. This can be solved by requiring that CE devices be routers, and by limiting the number of (MAC) entries created for each VPN.
Layer 3 - VRFs: In a Layer 3 solution, the maximum number of VRFs that allows multiple instances of a routing table to co-exist within the same PE at the same time may be constraint. To alleviate the impact of this factor route summarization (consolidating selected multiple routes into a single route advertisement) can be used whenever possible.
Other factors that impact scalability: additional parameters such as number of protocols, configuration scripts and ease of deployment / provisioning, and management system impact the scalability of the solution. - Availability and reliability
In both approaches high availability and reliability are achievable for service level and equipment level. Sub 50mSec protection is possible in both approaches and MPLS based multilayer OAM brings rich set of options for failure detection and fault isolation. - Hard QoS
In both approaches hard and well defined QoS is achievable. Both switches and routers implement traffic management components that are performing queuing and shaping for the traffic to assure the desired QoS. Since this implementation is at the network equipment, it is not associated with Layer 2 or Layer 3 solution. - Management
In practice, Layer 2 MPLS / MPLS-TP VPN can separate the control plane from the data plane, while Layer 3 MPLS VPN is having inherent control plane. Since Layer 2 is considered as "transport" layer and Layer 3 is considered as "service layer", transport-class element and management systems where developed for Layer 2 solutions while traditional CLI and text scripts are used in Layer 3 solutions (Layer 3 management with a GUI is usually considered as orthogonal and expensive product). Transport-class and GUI-based management increase the scalability of the solution and ease the ability to deal with upgrades and errors.
Complexity and total cost of ownership
Ethernet is considered simpler and therefore cost-effective in both CAPEX and OPEX.
- CAPEX: Although Ethernet and IP are mature, standard and interoperable technologies, in most cases switch has a lower price than a router and GE/10GE port on a switch has a lower price that GE/10GE port on a router.
- OPEX: IP know-how and therefore training costs and human errors are affecting OPEX and consider higher than Ethernet networks. Another significant OPEX item is power consumption: routers consume more power than switches. For the specific case of VPNs, the clear demarcation in Layer 2 VPN between the enterprise network and the service provider network means OPEX saving. Since the service provider needs only to take care for providing connectivity, the service uptime is higher and customer support is needed in lower scale.
Summary
A real-life and typical requirement for a telecommunication service provider is the ability to deliver both Layer 2 and Layer 3 VPN services.
Successful delivery needs to consider the enterprise's specific requirements and restrictions, as well as the service provider's economical and technical considerations.
Both Layer 2 and Layer 3 technologies are used as an infrastructure for VPN connectivity services. Layer 2 and Layer 3 technologies are mature and standard and therefore fit for mandatory requirements such as QoS, OAM, and synchronization.
However, in-depth understanding of Layer 2 and Layer 3 technologies exposes differences in the demarcation point, protocol transparency, security and end-point equipment and interfaces.
Layer 2 MPLS / MPLS-TP network introduces substantial advantages due to its lower cost and complexity, ability to carry any protocol, and separation between the enterprise's network and the service provider's network.
A large enterprise with his own IT department that needs large capacity transport to connect his remote offices will probably prefer Layer 2 VPN while other enterprise that is not having Ethernet interfaces will ask for Layer 3 VPN. The following figure and table provide a summary for different enterprise's requirements and the abilities of Layer 2 MPLS and Layer 3 MPLS technologies to fulfill those needs:
Figure #4: Segmenting enterprises for the selection of Layer 2 or Layer 3 MPLS VPN
|
The enterprise |
Layer 2 MPLS VPN |
Layer 3 MPLS VPN | |
|
Size |
Large |
Go |
|
|
SMB |
Go |
Go | |
|
SOHO |
Go | ||
|
IT department |
Strong |
Go |
|
|
No IT / don't care |
Go | ||
|
Connectivity type |
Point-to-point |
Go |
|
|
Point-to-multipoint |
Go |
Go | |
|
Multipoint-to-multipoint |
Go |
Go | |
|
Sensitivity to cost |
High (cost is critical) |
Go |
|
|
Low (cost is less critical) |
Go |
Go | |
|
Network stability |
Dynamic (many changes) |
Go |
|
|
Static (rarely changed) |
Go |
Go | |
|
Premises gear |
Switch |
Go |
|
|
Router |
Go |
Go | |
|
Complex protocols |
Go |
||
|
Non-Ethernet interface |
Go | ||
|
Service requirements |
High availability/reliability |
Go |
Go |
|
Hard QoS |
Go |
Go | |
|
TDM |
Go |
| |
Table #2: Layer 2 MPLS VPN and Layer 3 MPLS VPN fit for different enterprise's needs