Providing internet access using broadband or similar systems requires the use of an L2TP Network Server (LNS) to take connections from a carrier and connect them to your network. Obviously being an ISP also requires a number of other components such as core routers, RADIUS, DNS, etc, and so on. The FireBrick provides an LNS solution for small corporate connections through to multi-gigabit large scale internet providers.
In the UK, carriers such as BT Wholesale will connect ADSL/broadband, fibre to the cabinet (FTTC), and fibre to the premises (FTTP) via an L2TP handover. Aggregators such as Fluidata can connect BT, Be Wholesale, Rural broadband, and even mobile data. Mobile integrators such as AQL can connect 3G data SIMs. The same is true in many countries. For all of these you need a connection to your carrier, and an LNS to handle the L2TP handover.
L2TP was originally devised to handle modem dialup connections, and such services are still available for L2TP handover. The same protocols used for dialup (PPP, L2TP, RADIUS) are still used for modern fibre links.
For small installations such as a corporate or even home connection, e.g. connecting 3G data SIMs to your LAN, the FB2500 can handle a limited number of L2TP sessions and up to 100Mb/s total throughput. For small/niche ISPs the FB2700 can handle a limited number of sessions and up to 350Mb/s total throughput. For larger ISPs the FB6202 can handle tens of thousands of simultaneous connections and a full gigabit download plus corresponding uplink per box.
For up to a gigabit of capacity the recommended method of operation is to use two FireBricks in a master/slave mode. For more than a gigabit a pool of FB6202 is used.
It is also recommended that you have dual links to your carrier, one per FireBrick. This is standard for BT and Fluidata. Normally the carrier will use an initial RADIUS connection (AKA Platform RADIUS) to determine which LNS you would like each connection to be made. This allows traffic to be steered to the "master" LNS, using the "backup" only if there is a failure.
Having all connections on one LNS with the other as backup is easier to manage; makes software upgrades easier; and also ensures bonding multiple lines works seamlessly.
Where handling more than one FireBrick worth of connections (e.g. more than a gigabit using FB6202 or if using a set of FB2700s on a smaller scale), you have connections coming in to more than one LNS. Again, one link to a carrier per LNS makes sense.
When working in this way you may want to have an N+1 setup so that one is always not currently in use to allow software upgrades or testing. You also need to allow for the fact that load will not necessarily be evenly balanced. e.g. for 2 gigabit you may want to have 3 FB6202s in use and one spare.
There is still the need to ensure multi-line sites that use bonding are on the same LNS - this can be achieved by using the built in platform RADIUS system which can allocate connections to an LNS using a hash of all or part of the login details. Lines at the same site can have the same prefix on their login to ensure the go to the same LNS.
As the routing for each LNS is announced using BGP to your core network, this system can scale up indefinitely.
A carrier will usually use a /30 per link and BGP to announce its BRASs to your network and accept your LNS's addresses. The FireBrick can provide BGP to do this. In some cases the platform RADIUS and L2TP endpoint cannot be on the /30 link addresses (BT!) so you will want to announce a small block of IPs and use these on your FireBricks (as loopback addresses) for platform RADIUS and L2TP/LNS addresses.
The LNS also uses BGP to your core network to announce routed IP addresses for connected links. This means that it does not matter to which LNS a line connects. Obviously this is important if you have more than one LNS. The FireBrick FB6302 makes a good core router which is happy to accept a large number of /32 route announcements from the LNS.
Your carrier will typically expect you to provide a RADIUS server to steer sessions. The FireBrick includes a Platform RADIUS service to respond to these requests. We recommend you allocate two addresses specially for this and set these as BGP announced /32 loopback addresses, one on each FireBrick. This allows you to move where the RADIUS server is located without contacting your carrier. In the case of BT links you may also have to BGP announce a /32 in their address space as they 1-1 NAT RADIUS access to you. See BT Platform RADIUS for specific details on working with BT.
There are two main ways to operate the RADIUS. (a) Have each RADIUS return tagged responses listing your master LNS first and backup LNS second. This can then be used with round-robin RADIUS servers. (b) You can use master/slave RADIUS configuration and have each LNS return its own L2TP IP so using RADIUS to handle the backup if the first LNS fails. We recommend using the session steering features now that BT has SIN502 steering for 20CN, and this also works well for a pool of LNSs rather than just a master/slave system.
The FireBrick can also handle master/slave where the carrier is not handling session steering. This uses the require-platform setting on the L2TP incoming configuration as this rejects L2TP unless a platform RADIUS was received. Now that BT do support session steering this is deprecated. Our experience is that relying on RADIUS master/slave to manage sessions is not 100% and so some RADIUS and hence sessions may end up on your backup LNS requiring a periodic mop up operation.
You may wish to configure both RADIUS IP addresses on two FireBricks, using BGP MED to make one IP per FireBrick the normal routing and so allowing fall-back where both IPs go to the second FireBrick in the event of a failure. This ensures the carrier RADIUS server is not failing 50% of the time.
The platform RADIUS is quite sophisticated allowing tagged responses to direct sessions to the LNSs you want - whether a master/slave arrangement or a load balance of sessions randomly or using a hash of the username. In any case it makes sense to ensure all RADIUS servers are giving consistent tagged responses.
For BT lines in particular it is necessary to manage aggregate bandwidth by setting a shaper (graph) on 21CN and 20CN incoming L2TP configurations. If you do not then you will be paying BT burst charges on 100th percentile. Typically you will request BT to allow a small burst (5%) so as to ensure all shaping is done on the FireBrick rather than at BT as the FireBrick is in a much better position to manage capacity fairly between customers.
Obviously if you only have one live LNS this is relatively simple, though there is a risk if any sessions are on the wrong LNS at times of high load. The FireBrick can, however, share any shaper via an Ethernet (LAN) link between them so that the aggregate of the link(s) to BT are managed as a whole over all of the LNSs at once. The balancing is handled by per second stats shared between all of the LNSs, sharing all spare capacity automatically. Simply set up cqm sharing and define the shapers to the carrier as shared (and using the same name on every LNS).
If you operate a master/slave LNS arrangement, then it is recommended way to handle a software update is to upgrade the backup LNS which has no traffic, and then switch all connections from the master (i.e. swap master and slave operations). This also ensures that each LNS is regularly tested. It is recommended that you leave the other LNS on the older s/w until next upgrade so as to cater for the risk of any software related bugs in the new release. If you are not sharing the traffic shapers you will want to do this when not fully loaded.
With a pool of LNSs and a spare, you can upgrade the spare and adjust the pool (in platform RADIUS config) to now include the spare in place of one of the live LNSs. You can then clear sessions from the replaced live LNS moving the session to the upgraded LNS.
If you are sharing the traffic shapers, as we recommend, then you don't actually have to force sessions on to the newly upgraded LNS if you don't want to. They will move when they next connect as a result of the new platform RADIUS config. However, broadband lines can stay up for weeks, so this is not necessarily sensible. If you do line bonding and one lines moves while the other does not then you get lower bandwidth on the bonding. As such it is generally a good idea to clear sessions from what is now the spare or backup LNS to the live one. This can, of course, be something you scheduled over night to minimise disruption.
There is a telnet command clear l2tp all to clear all sessions. Obviously ensure any platform RADIUS is completed on all RADIUS servers first. You may want to clear one session first to confirm it moves to the newly upgraded LNS.
Note: If you are using an FB2500 or FB2700 as an LNS you may wish to turn off automatic software updates in the configuration, as an automatic update will clear all sessions.
The FireBrick support team have experience setting up LNSs for all sizes of ISPs and the main UK carriers including BT and BE. Please ask for advice and example configurations.
