The FireBrick 105 has an optional bonding
feature which allows both load
sharing
(typically to share downlink on multiple internet feeds) and
multiple gateway operation
(typically to share uplink on multiple internet feeds).
Uplink bonding
Uplink bonding means making use of multiple internet routes to provide
a true aggregate uplink. This applies even to a single session or file
transfer as the bonding is done on a per packet basis.
To provide a mechanism for sending packets to multiple routers the
FireBrick uses a pseudo
router address. This is a normal IP address that could be on the same
interface as your routers. Whenever the FireBrick uses this as the
gateway for traffic, it will follow all of the normal rules for any
gateway address and determining which interface to send traffic.
At the last moment when it would get the MAC address for this gateway
(checking the ARP cache or sending an ARP), it sustitutes one of the
real gateway addresses. It then uses the corresponding MAC address for
the real gateway and sends the packet. The packet still goes on the
interface it has decided based on the pseudo gateway address defined in
the routing.
The FireBrick can be configured with one or two pseudo gateway
addresses, and up to six real router addresses. When it tries to send
traffic to either gateway address it will in fact send to one of the
real addresses to share out the load.
The real addresses can be specified as an IP address, or by reference
to a subnet which has a gateway address defined. Using a gateway on a
subnet avoids dublication and also use with DHCP subnets.
The reason there are two pseudo addresses is that you may want to use
NAT with separate external addresses. When you route NAT traffic to a
gateway the gateway address is used to work out which subnet applies
and hence which of the FireBricks own IP addresses to use for NAT.
Having two pseudo addresses allows for NAT via two different subnets
and hence two different source addresses whilst still sending traffic
via multiple actual gateways.
Without NAT, it is normal to have a separate block of addresses on the
LAN routed via one or both of the external internet connections. It is
sensible for any FireBrick originated traffic (time setting, emails,
etc) to be sent using an address in this separate routed block. As such
it is normal to have this subnet on the LAN and the WAN. The LAN first
allowing use of the addresses for PCs, and the WAN so that the
FireBrick has an address it can use when talking to the WAN. Then, one
of the routed block of addresses is used as a pseudo gateway address.
The pseudo address could be any address which the FireBrick can route
to the interface, but is ideally one of the addresses on a WAN subnet
to allow the FireBrick to use a sensible address itself. However, this
would waste one otherwise usable address purely for internal operation.
This is particularly annoying if the external subnets are /30s which
would not leave space for a third address for this purpose. For this
reason it is recommended that the otherwise unusuable
network address (first address in
the subnet) is used as the pseudo gateway. The FireBrick will treat a
network address as a host and so allow it to be a valid gateway
address. Remeber, this pseudo address is purely internal and not
actually used outside, so the router does not have to be configured
specially.
Note: Some routers seem to take
offence to an ARP being sent for their IP apparently
from an IP not on their subnet.
This can be avoided by using the subnet with gateway to set the real
address as the ARP is from
the subnet address. It can also be avoided by making the routers think
they are all part of the same larger subnet even if they are in fact
using adjacent /30 blocks.
Different speed uplinks
There is no allowance for different speeds of uplink, but this can be
accomodated to some extent. e.g. if uplinks of 250K and 500K are
available, list the 500K gateway twice and the 250K gateway once, and
this will cause the traffic to be sent in the right proportions. Mixed
speed links may not give the performance you expect though as there
will be mre packet reordering.
Packet reordering
IP does not guarantee packet order (or packet route, reliable delivery
or non duplication of packets). It is the stack, e.g. TCP, which
ensures correct order and integrity of streams of data. As such TCP has
to cope with dropped, repeated and reordered packets. Sending packets
via two paths could result in packet re-ordering if the latency on the
two paths is different.
Unfortunately, some TCP stacks are less efficient when there is packet
re-ordering and so it is not always possible to achieve the full speed
of the two links combined. In practice we find that this is generally
not an issue and bodning uplink works well.
Packet reordering has also been seen to have some detrimental impact on
voice over IP allocations, although they should be able to cope within
the jitter delay. It also has impact on some RFC2833 handling where it
can appear to indicate new DTMF digits as a result of duration going
backwards. As such you may want to force VoIP traffic via one router or
use load sharing to send individual call consistently via one route.
More than 6 gateways
In theory you could cascade multiple FireBricks to allow more than 6
final gateways to be used. If you need more than 6 links, tunnel
bonding may be more appropriate. However, more links will probably
start to make packet reordering a major factor.
Fallback
With the profiles feature, bonded uplink can have a profile set for
each gateway allowing a graceful fallback to fewer gateways.
Downlink load sharing
If using downlink load sharing (see below), you typically NAT sessions
to be from one of two separate FireBrick addresses via two internet
connections. If uplink bonding is also required then you can use two
pseudo gateways - one for each of the subnets for each NAT session, and
list both as pseudo gateways. This means the uplink bonding applies to
the traffic regardless of which IP it is NATed via. Again, profiles
allow fallback to only use the correct real gateway instead of a pseudo
gateway in the event of one link failing.
Downlink bonding
Obviously an ISP could offer a service of downlink bonding, either
using a FireBrick at their end of multiple links or some similar
device. This has no impact on the FireBrick as it simple receives the
packets.
Downlink load sharing
The FireBrick can be very useful in a large office to allow use of
aggregate downlink capacity on multiple internet feeds. This is applied
on a per session basis, so the capacity on any one session will be
limited to the feed it uses, but the overal capacity for all sessions
can reach that of all feeds combined.
Probability based routing
The typical usage is with two internet feeds, each with a small block
of IPs so that the FireBrick has a real IP. The LAN then uses private
IPs. The FireBrick NATs traffic to the internet.
To achieve the load sharing, the routing rules are used instead of a
default gateway. Two rules are defined setting the gateway to each of
teh external routers with a 50% loading. This means that each session
may use one or the other route. NAT is selected on the route, and the
IP used will depend on the FireBricks IP for the subnet associated with
the gateway selected. The reply traffic will then arrive for that IP
via that gateway, and so half the sessions will have data arriving via
one route and half via another.
Fallback
It is recommended that the profiles feature is used to monitor gateways
and take out the routes used for any that are not working. This can
distort the load sharing on the remaining routes when there are more
than 2 gateways remaining, although suitable use of 50% routes as 33%
routes, and then default routes can allow for this if configured
carefully.
Tunnel bonding
The FireBrick can be used to establish tunnels to other FireBricks
(tunnel feature). With the bonding feature as well, the tunnels can be
linked in to a set allowing multiple tunnels to be used as one.
Normally this would be configured so that each tunnel is sent via a
different physical link.
Balancing different speeds
The tunnels in a set are equally weighted for traffic sent down them.
If you have a mixture of physical links you can make multiple tunnels
down the same link to bias the traffic to the faster links.
Fallback
Using tunnel send and expect keep alives, the tunnels are automatically
removed from the set when not active both ways. This allows the
remaining tunnels to share the load automatically without the need for
the profiles feature. Tunnels can also be
used with profiles to control if they are active or inactive.
Packet reordering
Packet reordering has also been seen to have some detrimental impact on
voice over IP allocations, although they should be able to cope within
the jitter delay. It also has impact on some RFC2833 handling where it
can appear to indicate new DTMF digits as a result of duration going
backwards. To help remove this problem tunnels have a QOS option which
will force packets with TOS bits 7 or 4 set (as is common with VoIP
traffic) to be sent consitently to the selected tunnel rather than
bonding. If the selected tunnel is not available then this flag is
ignored. You can use load sharing to send sessions to different
starting points in your set of
bonded tunnels to share out VoIP calls between them.
