Using BGP To Manage Dual Internet Connections

Using BGP to Manage
Dual Internet
connections
SDCUG
Sept 10, 2014
Meredith Rose, CCIE#4617

Copyright © SIGMAnet ® 2012. All rights reserved. Proprietary & Confidential.
2
Intro
• Meredith Rose, CCIE#4617 Emeritus
• Currently a Solutions Architect for SIGAMnet
• Disaster Recovery and Redundancy are
recurring themes requested by customers
striving to improve their network uptime
• Internet access has become
better/faster/cheaper, causing more
companies to rely on it and expect 5-nines
uptime.
• Not planning on reviewing the BGP protocol
details, but please ask questions any time.

High Level Agenda
• The need for Corp Internet x 2
• What you need to use BGP
• Key considerations
• BGP routes offered by ISPs
• Influencing traffic flows
3

Does a Company Need 2 Connections to
the Internet?
• Internet access is business-critical
• Apps, data exist in the cloud (ie AWS, WebEx)
• Branch connectivity via VPNs over the Internet
• Remote access, B2B connections
• eCommerce hosted at Corp data center
• Redundancy is a must; the less $ the better
• BGP can give you tools for utilizing the bandwidth
of both Internet connections simultaneously
and/or dynamic failover with 1 connection
backing up the other
4

“I want to use BGP to Load Balance my
Internet Connections”
• The BGP protocol does NOT know how to “load balance” your Internet traffic!
• BGP’s job is to select the single best path to a destination among the BGP
paths that are learned from different sources/ISP’s.
• BGP is not aware which link is “full” (oversubscribed) or “faster” (lower latency)
• Load sharing across your redundant Internet connections is a manual process
done on a per prefix basis that takes some TLC.
• Inbound and Outbound traffic loads of each link are tuned separately by
5
manipulating BGP attributes

One Internet Connection
ISP Router
Global Internet
6
Internet Connection
Corporate LAN
Corp Router
ISP
• Static routes to Corp on ISP router
• static default route to ISP on Corp router
• No need for BGP

Redundant Internet Connections
ISP#1 Router
7
InternetConnection#1
Corporate LAN
Corp Router#1
ISP#1
Global Internet
ISP#2 Router
Corp Router#2
ISP#2
eBGP eBGP
iBGP
L3
FHRP/OSPF/etc

Review of Recovery from Failure
8
• ISP failure
– Internet handoff
– Router failure
– Upstream peering issues
• Corp Router failure
– Internet handoff
– Router failure
– Connection to Corp LAN

Getting started with BGP to the Internet
• You will need an ASN (Autonomous System
Number). AKA “AS number”
– This can be private if using redundant
connections to the same ISP.
 Obtain from ISP
 Will be removed by ISP before being advertised to
global Internet
 Note: impacts ability to influence inbound traffic with
as-path pre-pending
– This will be a public ASN if connecting to diverse
ISPs.
 Obtain from ARIN
 More flexibility, ISP-independent
9

Getting started with BGP to the Internet
• You will need a public IP address “block” to advertise
• /24 minimum
– This can be assigned/leased to you from your ISP
 Easy if both Internet connections are from same ISP
 Make sure the ISP that allocated the block to you advertises your
specific subnet (ie /24) and not just their supernet block.
 If using diverse ISP’s, must check with both to make sure it is ok to
advertise IP block from ISP#1 IP space through ISP#2.
 More convenient, but less portable
– This IP block can be owned by your company.
 You can advertise your block to both ISPs.
 More mobility if change ISP’s
– Make sure you only advertise your assigned, routable IP
10
address space!
– You will advertise the SAME IP block out to BOTH ISP’s
 Can do some tricks with splitting into sub-prefixes and advertising
smaller, more specific chunks. Always >=/24

Key Considerations
• Ingress and Egress “traffic engineering” managed separately
• OUTBOUND traffic influencing
– Get your Corp traffic to its destination on the Internet
– Want to send traffic out the “best” ISP
 Shortest AS path is usually best
– Want to avoid oversubscribing a link
• INBOUND traffic influencing
– Packets from everywhere on the global Internet have to find
your Corp network. ISP advertises your IP block(s) to global
Internet
– Asymmetric is usually OK here (out one ISP, in the other)
 Caveat: not ok if you have non-stateful firewalls
– Want to take “best” route from global Internet to Corp
 Shortest AS path wins in most cases by default
– Want to avoid oversubscribing a link
11

Key Considerations (Continued)
• Redundancy protocols on Corp routers.
– HSRP/VRRP if L2 connected
– Or use L3 dynamic protocol like OSPF. Internet
routers can be in different Corp locations, L3
connected. Each Corp BGP router can originate a
default route in Corp-wide OSPF.
• Corp routers need to know how to get to ISP
router’s peering IP address (or use next-hop-self
on iBGP session). If iBGP routers peer on
loopback, must be reachable (use IGP + update-source
12
loopback0)
• Get Corp traffic destined for Internet to one of
the Corp Internet routers. It doesn’t really matter
which one. BGP will take it from there.
• It’s about manual traffic load distribution; BGP
does not know how to do dynamic Load
Balancing to multiple ISPs on its own
• You do not want your Corp to be come a
“Transit” path between your two ISP’s!

“Transit” - What’s the big deal?
Corporate LAN
13
ISP#1
Global Internet
ISP#2
ISP#1
Routes
ISP#2 says “Hey
Global Internet!
Here’s a quick way
to reach ISP#1
customers!”

Don’t be a Transit!
Corporate LAN
14
ISP#1
Global Internet
ISP#2
ISP#2 Routes ISP#1 Routes
Only send
routes
originating
from your
Corp ASN to
each ISP
iBGP full route
exchange

What Routes to Take in from ISP
• Remember: this affects OUTBOUND decisions (not inbound),
ie which ISP your Corp will use to make a connection to a site
on the Internet. Most common options:
• Option#1: Full Internet routes from each ISP
• Option#2: Default/0.0.0.0 only from each ISP
– Tune so use one link as primary, other as backup
• Option#3: ISP’s Customer Routes Only
– AKA “Partial Routes”
– Get each ISP’s local customer routes only. Use a
default route to put the rest of the outbound traffic on
one ISP’s link, backup by other ISP.
– Or use just one ISP link to receive that ISP’s directly
connected customers, use default route to put the rest
of the outbound traffic on the other link
15

Option#1: Full Routes from Both
Corporate LAN
16
ISP#1
Global Internet
ISP#2
Full
Routes
from
ISP#1
iBGP full route
exchange
Full
Routes
from
ISP#2

What Routes to Take In from ISP
• Option#1: Full Internet routes from each ISP
– Need a lot of memory for this. Each router will have
2xfull Internet routing table (table>450k routes)!
– Let it play out and monitor for over-utilization of one
17
link
– Tune to balance links better if necessary
– Use route-map + as-path access list to make sure you
do not become a transit between ISP’s.
 Do not advertise routes to ISP#2 that you learned from ISP#1
and vice versa
 apply a similar route map outbound to each ISP neighbor so
that only locally originated BGP routes are advertised
– route-map localonly permit 10
– match as-path 10
– ip as-path access-list 10 permit ^$
– Not a bad idea to take a default from each ISP as well

Option#2: Default from Both
Corporate LAN
18
ISP#1
Global Internet
ISP#2
0.0.0.0
iBGP exchange
default received
routes
w/preferences
Configure iBGP to
prefer default
route from ISP#1
0.0.0.0
iBGP will agree to
prefer 0.0.0.0 from
ISP#1 over ISP#2

• Option#2: Default only from each ISP
– Tune BGP (local pref is common) so use one link
as primary, other as backup (again, only applies to
OUTBOUND traffic)
– Tell your ISPs you only want them to send you the
default route
– Use an inbound prefix-list on route-map inbound
on the ISP neighbor statement or similar filter to
make sure to drop every route except default just
in case
 ip prefix-list default-only seq 5 permit 0.0.0.0/0
– Still only advertise prefixes originated by your AS
to ISP#1 and ISP#2 (by default, BGP won’t send
them each other’s 0.0.0.0 that you learned –
phew!)
19

Option#3: ISP Local Routes Only
Corporate LAN
Routes
from ISP#2
customers
+ 0.0.0.0
20
ISP#1
Global Internet
ISP#2
Routes
from ISP#1
customers
+ 0.0.0.0
iBGP will naturally
send traffic for
local routes to its
corresponding ISP
Configure iBGP to prefer
default route from ISP#1
to catch routes not local
to either ISP
iBGP will agree to prefer
ISP#1 for everything not
local to ISP#2

• Option#3: ISP’s routes only + Default
– Only receive routes from an ISP of that ISP’s directly
connected customers (think of how many big companies
host with ATT, etc)
– You can ask your ISP to send you just their customer routes
– Filter routes not sourced from that ISP just in case (in this
example, ISP = AS100, route-map is inbound on neighbor
statement to ISP):
 ip as-path access-list 20 permit ^100$
 route-map as100only permit 10
 match as-path 20
– Use one link for one directly connected ISP’s customers
(more local provider), use default route to prefer to put the
rest of the outbound traffic on the other link or similar combo
21

Influencing Traffic Flows: OUTBOUND
• OUTBOUND Traffic Control is easier than INBOUND.
It’s all on you.
• All you have to control is how attractive a destination
looks to your Corp BGP routers.
• You can only control the next AS in the path (ie ISP#1
vs ISP#2), not the entire path through the global
Internet to the destination.
• Most common OUTBOUND:
22
– Local preference
 Outbound traffic flows to one of your Corp BGP routers. BGP will
have used the “local preference” attribute to tell that router which
route to take (ISP#1 vs ISP#2) to reach the destination.
 Monitor regularly and tweak/tune local pref of prefixes as desired
 Look for popular, heavily-used prefixes to influence to get the
most bang for your buck (or increase local pref of big /4 chunks)

Influencing Traffic Flows: INBOUND
23
• Most common INBOUND:
– AS-Path prepend
 Backup Path: If you don’t want traffic to come in on a link for a prefix (or the entire IP block), use
Prepending feature to add AS Path length to your outbound advertisement, making this link the less
preferred path for traffic to your IP block. Aka “padding”. 4xAS# is generally sufficient
 Primary Path: Use standard advertisement (no prepending) for the link you prefer to use for inbound
traffic to your company
 Still have (pre-pended, valid) advertisement from backup path if primary path fails.
– Example: set as-path prepend 130 130 130 (add to route-map and apply to neighbor statement to backup ISP)
– Communities
 Community = instructions from you to your ISP on how to tweak what you advertise
 ISP will let you know definition of communities they honor
 You will attach a community to a prefix that you are advertising to your ISP(s)
 Consists of a series of numbers that correspond to handling instructions for that prefix (such as set
local pref within provider’s AS)
 Communities can also be used internally to identify routes. For example, you can assign all routes that came
from ISP#1 with a community and routes that came from ISP#2 with a different community. That community
identifier can then be used by your company to assign preferences to routes advertised internally via iBGP. For
example, I want all traffic destined for YouTube’s /16 IP Block to use ISP#2, even though ISP#1 has a shorter
AS-Path for the YouTube subnet (perhaps bandwidth is greater to ISP#2). So use the community to set a
better metric on that route when it comes in from ISP#2. Remember, weight and local pref take precedence
over AS-path length.
– Prefix-splitting
 ie /192x/20 subnets. Advertise one to each ISP, both also advertise complete /19 aggregate as a
safety-net to cover failure of one ISP. Remember: most specific advertisement always wins!
 Works best when you own your IP Space (splits still >=/24)
 Use a BGP Looking Glass or Route Server to see how to get to your Corp AS’s prefixes

Thank You!
SDCUG
Sept 10, 2014
Meredith Rose, CCIE#4617

Redundant Internet Connections
ISP#1 Router
25
Corporate LAN
Corp Router#1
ISP#1
Global Internet
ISP#2 Router
Corp Router#2
ISP#2
eBGP eBGP
iBGP
L3
FHRP/OSPF/etc

Using BGP To Manage Dual Internet Connections

More Related Content

What's hot (20)

Similar to Using BGP To Manage Dual Internet Connections (20)

Recently uploaded (20)

Using BGP To Manage Dual Internet Connections