13.3. L2TP tunnelling

Layer 2 tunnelling protocol is a standard way to pass IP packets via a tunnel. It was originally used for dialup connections, connecting the RAS (Remote Access Server) to the LNS (L2TP Network Server), and is still used today within Internet Service Providers for broadband connections. L2TP has the advantage that it is very standard and interacts with a variety of equipment.

The FireBrick provides a means to accept static L2TP connections with configured username, password, and IP routing.

The FireBrick also provides a means to make an outgoing L2TP connection as if it was a RAS, connecting to an LNS. This can be used with ISPs that offer this service, or could be used, for example, to connect to another FireBrick.

Note

Some systems may refer to L2TP when they actually mean L2TP over IPsec, which was used more commonly before IKEv2 was available in IPsec, and is not supported.

The FireBrick can act as full LNS, designed to allow thousands of connections to be handled by an ISP. See Chapter 18 for more details.

L2TP is usually used by Internet Service Providers, but it can also be used to create point to point tunnels connecting, say, two FireBricks together to pass IP traffic. The configuration for this can be a little confusing because L2TP is designed for multiple connections over separate tunnels between Access Controllers and L2TP Network Controllers, so the protocol involves more than one layer and authentication, and a lot of detailed settings.

Once upon a time the Remote Access Server (RAS) would have had a bank of modems or ISDN links, but these days it would be broadband connections over PPPoE. A RAS would create tunnel to an L2TP Network Server (LNS). Once a call comes in, the RAS would then create a session over that tunnel connecting the LNS to the end user. Each tunnel can hold multiple sessions all to the same LNS. The LNS then talks PPP over the tunnel/session to the end user and negotiates login details and IP addressing and so on. It then uses PPP to pass IP packets allowing a connection to the Internet.

If you make a point to point link, one end acts as the RAS, and makes a tunnel to the other end acting as an LNS, and then it creates a session over that tunnel. The settings needed for this are often a lot simpler than an ISP would use, so there is a lot of the FireBrick configuration you can ignore. It is, however, useful to understand some of the terms and settings, especially the authentication.

13.3.1. Incoming tunnel

The FireBrick uses a static configuration to decide if it will accept an incoming L2TP tunnel. This is the Incoming L2TP configuration. An incoming tunnel has very few credentials - the main one being the hostname it sends. This is matched against the remote-hostname (which allows a wildcard, even). You can also match to allow specific RAS IP addresses, as well as the routing table used. The secret needs to be the same on the RAS and LNS.

Once the first match is found, the L2TP tunnel can be accepted. Most of the remaining configuration settings are not related to the tunnel as such, but define the default settings for any session that comes in over the tunnel. These settings can normally be omitted as they will have sensible defaults anyway.

A single tunnel config allows multiple instances of that tunnel to be created. If you are making multiple connections you may want to create different tunnel hostnames to allow fine control of specific features. Alternatively you may want to have one incoming tunnel configuration, even matching any hostname, and set specific session details using RADIUS. As an ISP, you will typically have an incoming tunnel config for each carrier, or possibly service type.

13.3.2. Incoming session

Once the RAS has a tunnel to the LNS it can create a session. The FireBrick acting as the LNS has to decide to accept the session or not, and what settings to use for the session.

The FireBrick acting as an LNS can use a static configuration to accept a session, or use RADIUS. An ISP would typically use RADIUS which allows a separate authentication service to decide if to accept the session and the settings that apply.

The static configuration consists of match settings. These are part of the relevant Incoming L2TP configuration for the tunnel being used. The main criteria is username (which can be wildcard, even), but can also match calling-station-id and called-station-id which would have once been phone numbers, but are typically some sort of circuit ID and service ID in broadband. The password has to be correct, if specified. Making a point to point L2TP link allows all of these to be set on the outgoing connection.

Having found the first match, there are a few settings which can be controlled. This is not the full set of session settings, but most can be set as a default in the Incoming L2TP tunnel configuration if required on a point to point static configuration. The main setting is remote-ppp-ip which defines the IPv4 endpoint for the far end. If this is set then the session is accepted and routing for this IP is negotiated and routed. Additional routes can also be set.

If the remote-ppp-ip is not defined, then relay settings can be defined to allow the session to be relayed down another L2TP tunnel. These include the hostname, IP, and secret to use.

If not static match is made them RADIUS is used. As an ISP it will be unusual to have any static incoming session matching apart from special test cases.

If a profile associated with an incoming connection is disabled then existing connections will be allowed to persist, however no new ones can be established. This is useful if you need to gracefully drain a particular set of connections.

13.3.3. Outgoing connection

An outgoing L2TP connection means the FireBrick is wanting to establish a session with IP routing. To do this it is necessary to create an L2TP tunnels first, so the outgoing configuration includes details of both the tunnel (local-hostname, secret, server IP), and the session (username, password, calling-station-id, called-station-id).

If a tunnel exists with the right credentials, it is re-used, so having more than one session, although typically you do not do this as tunnels to the same LNS will normally be via different routing tables and backhaul links. If no tunnel exists, one is created first and then the session created over that tunnel. If the session drops, it may be re-established over the same tunnel. If the tunnel drops (which drops all sessions in it) then it hangs around for a few seconds before a new tunnel can be made (fail-timeout).

Note

The config makes use of prefixes like local-… and remote-…. These are relative to the device you are configuring. So a config to link two FireBricks will have local-hostname one end matching the remote-hostname the other. Similarly settings like tx-… and rx-… are relative, tx being transmit/send from this device, and rx being receive to this device.

Note

As with other tunnels, it is often useful to set the payload table to a different routing table to the tunnel traffic, to avoid sending tunnel traffic down the tunnel. This is particularly important for an outgoing tunnel as the default is to route all traffic (i.e. 0.0.0.0/0 and ::/0) down the tunnel when connected. You can configure specific routes to be passed down the tunnel if you wish. Routing in to this tunnel depends entirely on the configuration at the other end.

Tip

The debug level logging is ideal for showing the details of the low level connection and PPP negotiation. This is recommended when first setting up such tunnels, and should be checked at both ends to identify any errors or problems.

13.3.4. High availability L2TP

The L2TP configuration has an optional ha-set setting allowing you to define that sessions in the specified tunnel are used for High availability.

This means any packets sent to the sessions are in fact sent to each session (duplicated). At the receiving end the first of each duplicate is passed on as normal and any subsequent ones discarded.

This sacrifices bandwidth for high availability and reliability.

You need to ensure packets are routed to the multiple sessions in the set, in the same way as bonding L2TP, and also that they are put in the same ha-set.

Note

The receive byte counts and any graphs on the L2TP tunnels will show the duplicate packets, as this shows the process is working. However, a packet dump of L2TP session content traffic does not include the duplicate packets. A packet dump of the L2TP wrapper traffic will, of course, show the wrapper duplicate packets received.

13.3.4.1. Interpreting HA statistics

If ha-stats logging is enabled then when HA sets are active you will get lines like this each second at the specified log target:
HAL A: (tick)4 (cross)12 o0 (arrows)0 -2 +0

These correspond to the following for each sample interval:

Table 13.3. HA statistic definitions

SymbolMeaning
Count of unique packets received and forwarded.
Count packets classified as duplicates and dropped.
oCount packets classified as too old for the deduplication window and dropped.
(arrows)Count of out of order packets received.
-Maximum distance behind the current sequence number received.
+Maximum distance ahead of the current sequence number received.

In the example above there were 4 tunnels in HA set A.

During the sample interval 4 packets were transferred to this FireBrick via the HA set.

It can be seen that the expected number for dropped duplicate packets d is given by d = (t - 1) × p where t is the number of active tunnels in the HA set and p is the number of packets transferred. In this example that comes out as 3 × 4 = 12. Sometimes this won't be an exact multiple due to loss or packets sent by different paths landing in different sampling intervals.

Some reordering is to be expected if there is loss on the lowest latency path.

If you see large values for the distance behind (greater than the number of packets transferred in the interval is a good rule of thumb, but certainly anything more than 10,000), or if you see any "too old" packets you should consider reducing your egress shaper speed on the far end.

13.3.4.2. HA best practice

The aim of the HA sets is to allow traffic to flow via different paths for redundancy, therefore it follows that paths with minimal overlap are preferable from an HA point of view.

Within an HA set, traffic (including TCP) will flow at the speed of the fastest link. However the slower links may be overwhelmed, which will make the duplication of paths ineffective as a redundancy mechanism. Therefore it is recommended to limit the speed of traffic sent via HA tunnels (usually this is achieved with a shaper on the ingress traffic).

Adding more paths to the HA set will make the set more tolerant in general, however transmitting and receiving all the duplicate packets has an overhead. This is especially noticeable if a lot of the packets are fragmented. For instance if you have 5 tunnels and every packet is being fragmented then 10x the number of packets are being sent and received as are traversing the tunnel. This has a considerable impact on throughput.