19.2. BGP Setup

19.2.1. Overview

The FB2700 series router provides BGP routing capabilities. The FB2700 cannot handle a full table. The aim of the design is to make configuration simple for a small ISP or corporate BGP user - defining key types of BGP peer with pre-set rules to minimise mistakes.

Caution

Misconfiguring BGP can have a serious impact on the Internet as a whole. In most cases your transit providers will have necessary filtering in place to protect from mistakes, but that is not always the case. If you are an ISP and connect to peering points you can cause havoc locally, or even internationally, by misconfiguring your BGP. Take care and get professional advice if you are unsure.

Note

The FireBrick has some defaults and specific types of peer that set some of these defaults. These are designed so that beginners do not make the more obvious mistakes with BGP and can be a good way of working. However, if you are already familiar with BGP you may wish to set all peers to type peer and define your own community tags and filters rather than using the defaults provided by FireBrick peer types.

19.2.2. Standards

The key features supported are:-

  • Simple pre-set configurations for typical ISP/corporate setup
  • RFC4271 Standard BGP capable of handling multiple full internet routing tables
  • RFC4893 32 bit AS number handling
  • RFC2858 Multi protocol handling of IPv6
  • RFC1997 Community tagging, with in-built support for well-known communities
  • RFC2385 TCP MD5 protection
  • RFC2796 Route reflector peers
  • RFC3392 Capabilities negotiation
  • RFC3065 Confederation peers
  • RFC5082 TTL Security
  • Multiple independent routing tables allowing independent BGP operations
  • Multiple AS operation

19.2.3. Simple example setup

A typical installation may have transit connections from which a complete internet routing table is received, peers which provide their own routes only, internal peers making an IBGP mesh, customers to which transit is provided and customer routes may be accepted. To make this setup simple the <peer> definition contains a type attribute. This allows simple BGP configuration such as:-

<bgp as="12345">
 <peer as="666" name="transit1" type="transit" ip="1.2.3.4"/>
 <peer as="777" name="transit2" type="transit" ip="2.3.4.5 2.3.4.6"/\>
 <peer type="internal" ip="5.6.7.8"/\>
</bgp>

This example has two transit providers, the second of which is actually two peer IP addresses, and one internal connection. Note that the peer AS is optional and unnecessary on internal type as it has to match ours.

The exact elements that apply are defined in the XML/XSD documentation for your software release.

19.2.4. Peer type

The type attribute controls some of the behaviour of the session and some of the default settings as follows.

Table 19.1. Peer types

TypeMeaning
normalNormal mode, no special treatment. Follows normal BGP rules.
transitUsed when talking to a transit provider, or a peer that provides more than just their own routes. Peers only with different AS. The community no-export is added to imported routes unless explicitly de-tagged
peerUsed when talking to a peer providing only their own routes. Peers only with different AS. The community no-export is added to imported routes unless explicitly de-tagged allow-only-their-as defaults to true
customerUsed when talking to a customer's routers, expecting transit feed and providing their own routes Peers only with different AS allow-only-their-as defaults to true allow-export defaults to true The community no-export is added to exported routes unless explicitly de-tagged
internalFor IBGP links. Peers only with same AS allow-own-as defaults to true
reflectorFor IBGP links that are a route-reflector. Route reflector rules apply Peers only with same AS allow-own-as defaults to true
confederateFor EBGP that is part of a confederation. Confederation rules apply Peers only with different AS
ixpMust be EBGP, and sets default of no-fib and not add-own-as. Routes from this peer are marked as IXP routes which affects filtering on route announcements. Only announced on EBGP not IBGP.

19.2.5. Route filtering

Each peer has a set of import and export rules which are applied to routes that are imported or exported from the peer. There are also named bgp-filter which can be used as import-filters or export-filters.

The objects import and export work in exactly the same way, checking the routes imported or exported against a set of rules and then possibly making changes to the attributes of the routes or even choosing to discard the route.

Each of these objects contain:-

  • Cosmetic attributes such as name, comment, and source.
  • Route matching attributes allowing specific routes to be selected
  • Action attributes defining changes to the route
  • A continuation attribute stop defining if the matching stops at this rule (default) or continues to check further rules

The rules are considered in order. The first rule to match all of the matching attributes is used. If no rules match then the default actions from the import/export object are used.

In addition, the top level import/export has a prefix list. If present then this will limit the prefixes processed at a top level, dropping any that do not match the list without even considering the rules.

19.2.5.1. Matching attributes

The actual attributes are listed in the XML/XSD documentation for the software version. The main ones are:-

  • A list of prefixes filters defining which prefixes to match
  • It is also possible to match by checking presence or absence of specific community tags.
  • It is also possible to match by checking presence of specific AS in the AS-Path, or check the last AS in the path (originator).

You can have a rule with no matching attribute which will always be applied, but this is generally pointless as no later rules will be considered. If you want to define defaults then set them in the top level import/export object.

19.2.5.2. Action attributes

The actual attributes are listed in the XML/XSD documentation for the software version. The main ones are:-

  • Adding specific community tags
  • Removing specific community tags, including defaults added by the peer type.
  • Dropping the route completely
  • Changing the MED
  • Changing the localpref
  • Padding the AS list

The logic works by creating a set of actions that are applied, and these are based on top level settings in the peer (such as set-med) followed by the list of import or export named filters from which one matching action is picked, and then followed by the peers indivdiual import and export rules from which one mathcing action is picked. The matching action causes each of the settings that are present to replace what is currently picked. E.g. if a MED is set in the top level and a named rule set the named rules set replaces the top level setting.

Important note - adding or removing community tags does not compound. For each setting (e.g. tag, untag, med and localpref and any added in future) the latest that was found after checking top level peer settings, the ordered list of filters, and then the local filters, is what applies. Multiple tag do not cause all the tags to be added, just the latest listed tags in the action. There are plans to improve this in the future to work step by step and even allow MED and localpref adjustments to compound.

You can have a rule with no action attributes. If matched then this means none of the actions are taken and communities, localpref, MED, etc., are all unchanged.

19.2.6. Well known community tags

Specific well known communities are supported natively. Some of these are set automatically based on peer type and can be explicitly removed using the detag action. These rules are automatically checked for exporting routes unless overridden on the peer attributes.

Table 19.2. Communities

CommunityNameMeaning
FFFFFF01no-exportThe route is not announced on any EBGP session (other than confederation or where allow-export is set).
FFFFFF02no-advertiseThe route is not considered part of BGP. Whilst it is applied and used for routing internally it is not announced at all or considered to have been received for the purposes of BGP.
FFFFFF03local-asThe route is only advertised on IBGP (same AS) sessions.
FFFFFF04no-peerThis tag is passed on to peers but does not have any special meaning internally

19.2.7. Announcing black hole routes

The FireBrick allows black hole routes to be defined using the the blackhole object. Routing for such addresses is simply dropped with no ICMP error. Such routes can be marked for BGP announcement just like any other routes.

It is also possible for L2TP announced routes to be marked as black hole routes using the D filter. If L2TP is marked to BGP announce such routes they are set to be bgp="true" rathed than the bgp setting defined.

In order to ensure that your internal BGP network sees such routes as a black hole, and not simply as a route to the router than has the black hole defined (where the packets will be dropped), you can ensure all black hole routes are announced using a suitable community tag. In many cases an EBGP peer will even allow you to announce black hole routes to them with a suitable community tag.

The top level bgp object includes a blackhole-community attribute which can be set to a tag that is used to mark routes as a black hole within your network. Any route received on a BGP peer within that config object which includes the specified community is treated as a black hole route. It is installed in the BGP routes and propagated as normal but it is internally set as forwarding to nowhere and packets dropped as a black hole.

Each peer object also has a blackhole-community tag. If set then this is added to any black hole routes announced. If not set, then black hole routes are not sent on EBGP links. On IBGP links, if not set, the blackhole-community from the parent bgp is added if present. Black hole routes are always announced on IBGP (subject to normal rules for announcement).

To use this, define a suitable blackhole-community for your network, such as YourAS:666 and set in all bgp objects. For all EBGP peers, set the peer object blackhole-community attribute with the tag they expect for black hole routes.

It is unlikely you would want to announce a black hole route to an EBGP peer without an agreed tag as you will be drawing traffic from them only to be discarded. If you want to do this, you have to specify a blackhole-community to add, but this could simply be your own community tag for black holes.

19.2.8. Grey holes

Sometimes you may want to inject a blackhole route into your network, but not actually blackhole within your network, simply ensuring that the routes propagate via BGP until edge routers where they are announced to transit/peering as blackhole community tagged routes (as above).

For this reason, blackhole route objects can be tagged no-fib which creates the routing in the routing table but does not impact packet forwarding. By defining a greyhole-community on your bgp settings, this will be used for IBGP to pass the route around as a blackhole route but with no-fib set.

This is useful where injecting a black hole is needed, but you want to ensure internal routing to externally blackholed routes.

19.2.9. Announcing dead end routes

The top level bgp object includes a dead-end-community attribute which can be set to a tag that is used to mark routes as a dead end within your network. Any route received on a BGP peer within that config object which includes the specified community is treated as a dead end route. It is installed in the BGP routes and propagated as normal but it is internally set as forwarding to nowhere and icmp errors generated (rate limited as usual). Any route installed as network are announced with this community. Note, this is not set automatically on a nowhere route, allowing a route to be announced to get to this FireBrick to be propagated via IBGP.

The effect of this is that your network can include one (or more) source of top level network routes which, within your network, are installed as dead ends at each point. Without this these would be announced to your internal network so traffic is sent to the originating router and it has to handle all dead end traffic. Using this system you can ensure dead end traffic is handled at your borders instead.

19.2.10. Bad optional path attributes

The BGP specification is clear that receipt of a path attribute that we understand but is in some way wrong should cause the BGP session to be shut down. This has a problem if the attribute is one that is not known to intermediate routers in the internet which means a bad content is propagated to multiple routers on the internet and they will drop their session. This can cause a major problem in the internet.

To work around this ignore-bad-optional-partial is set to true, by default. The effect is that if a path attribute we understand is wrong, and it is optional, and the router that sent it to us did not understand or check it (partial bit is set), we ignore the specific route rather than dropping the whole BGP session.

19.2.11. <network> element

The network element defines a prefix that is to be announced by BGP by default, and tagged with any dead-end-community, but otherwise treated the same as a nowhere route.

Table 19.3. Network attributes

AttributeMeaning
ipOne or more prefixes to be announced
as-pathOptional AS path to be used as if we had received this prefix from another AS with this path
localprefApplicable localpref to announce
bgpThe bgp mode, one of the well known community tags or true (the default) which is announced by BGP with no extra tags

19.2.12. <route>, <subnet> and other elements

Subnet and route elements used for normal set-up of internal routing can be announced by BGP using the bgp attribute with the same values as the well known community tags, or with true meaning simply announce with no tags, and false meaning the same as no-advertise.

Many other objects in the configuration which can cause routes to be inserted have a bgp attribute which can be set to control whether the routes are announced, or not.

19.2.13. Route feasibility testing

The FB2700 has an aggressive route feasibility test that confirms not only routability of each next-hop but also that it is answering ARP/ND requests. Whenever a next-hop is infeasible then all routes using that next-hop are removed. When it becomes feasible the routes are re-applied. This goes beyond the normal BGP specification and minimises any risk of announcing a black hole route.

Note

There is an option relating to imported routes reduce-recursion which, when set, changes any received next hop to the peer address unless the next hop relates to a locally connected Ethernet subnet. This helps reduce the recursion involved, and is important in some cases for route reflectors if they pass recursive routes on to routers that do not handle BGP recursive routes properly (such as BIRD).

19.2.14. Status

The rows in the BGP status page are coloured grey when idle (or incoming), yellow when connecting, teal until the EORIB marker is received and green thereafter. Red connections are waiting for a backoff timer before retrying an outgoing connection.

Under some circumstances (for instance during graceful restart), the FB2700 will hold routes received from the peer for a time while renegotiation occurs, these routes are included in the inbound route count.

Note

Since held routes are included in the count, it is sometimes possible to prematurely hit the prefix limit when refreshing (for instance when changing filters in the config). This means that if a prefix limit is configured, fewer routes than expected may be picked up. However, the held routes will time out shortly and the count will drop, after which a refresh will pick up as many routes as the prefix limit allows. A refresh can be manually triggered from the BGP status page for an individual peer.

19.2.15. Diagnostics

The web control pages have diagnostics allowing routing to be shown, either for a specific target IP (finding the most specific route which applies), or for a specified prefix. This lists the routes that exist in order, and indicates if they are suppressed (e.g. route feasibility has removed the route). There are command line operations to show routing as well.

It is also possible to confirm what routes are imported from or exported to any peer.

The diagnostics also allow ping and traceroute which can be useful to confirm correct routing.

Routes that are rejected during import (either due to hitting the max prefix limit or due to configured filtering) are logged to the debug target on the corresponding peer.

19.2.16. Router startup and shutdown

On router shutdown/reboot (e.g. for software load) or on profiling off a peer all established BGP sessions are closed cleanly. Before the sessions are closed all outgoing routes are announced with a lower priority (high MED, low localpref, prefix stuffed) and then a delay allows these to propagate. This can be configured per peer with the clean-shutdown-wait option. Setting to zero will not do the low priority announcement. A special case of setting this delay to a negative value on a peer causes routes to be specifically withdrawn before the delay rather than announced low priority.

On startup, each peer can be configured with a startup delay (clean-startup-wait) which will stop BGP announcments being sent within a period of a reboot. This allows a FireBrick to connect BGP sessions and receive routes before announcing routes to peers. This allows a cleaner startup when used as a pair of BGP routers.

19.2.17. TTL security

The FireBrick supports RFC5082 standard TTL security. Simply setting ttl-security="1" on the peer settings causes all of the BGP control packets to have a TTL of 255 and expects all received packets to be TTL 255 as well.

You can configure multiple hops as well, setting ttl-security="2" for example still sends TTL 255 but accepts 254 or 255. This works up to 127.

You can also configure a non standard forced TTL mode by setting a negative TTL security (-1 to -128) which forces a specific TTL on sending packets but does not check received packets. For example, setting ttl-security="-1" causes a TTL of 1 on outgoing packets. This simulates the behaviour of some other routers in IBGP mode. Using -2, -3, etc, will simulate the behaviour of such routers in EBGP multi-hop mode. This is non standard as RFCs recommend a much higher TTL and BGP does not require TTLs to be set differently.

Without ttl-security set (or set to 0) the RFC recommended default TTL us used on all sent packets and not checked on received packets.