Advanced enterprise campus design. routed access (2015 milan)

© 2015 Cisco and/or its affiliates. All rights reserved.BRKCRS-3036 Cisco Public
Some loops are fun ...

Advanced Enterprise
Campus Design: Routed Access
BRKCRS-3036
Mark Montañez, CCIE #8798
Architecture Lead, Enterprise Segment
Distinguished Consulting Engineer
@MarkMontanez or Montanez@Cisco.Com

Agenda - Enterprise Campus
Design: Routed Access
• Introduction
• Cisco Campus Architecture Review
• Campus Routing Foundation and Best
Practices
• Building a Routed Access Campus Design
• Routed Access Design and VSS
• Impact of Routed Access Design for Advanced
Technologies
• Summary
4

Start with the Core
! "#$%&' ( $#
Add in the
Distribution Layer …
! "#$%&' ( $#
) *+,#*- . / "0%
&' ( $#
Traditional Multi-Layer
Distribution …
! "#$%&' ( $#
) *+,#*- . / "0%
&' ( $#
VSS-based
Distribution …
! "#$%&' ( $#
) *+,#*- . / "0%
&' ( $#
122
Add in the
Access Layer …
! "#$%&' ( $#
) *+,#*- . / "0%
&' ( $#
122
344$++
&' ( $#
Multi-Layer Access …
L3 terminated at Dist.
! "#$%&' ( $#
) *+,#*- . / "0%
&' ( $#
122
344$++
&' ( $#
Routed Access …
L3 terminated at Access
! "#$%&' ( $#
) *+,#*- . / "0%
&' ( $#
122
344$++
&' ( $#
Converged Access …
Wired / Wireless
! "#$%&' ( $#
) *+,#*- . / "0%
&' ( $#
122
344$++
&' ( $#
Instant Access …
! "#$%&' ( $#
) *+,#*- . / "0%
&' ( $#
122
344$++
&' ( $#
Add in
Wired clients ...
! "#$%&' ( $#
) *+,#*- . / "0%
&' ( $#
122
344$++
&' ( $#
Add in
Access Points …
! "#$%&' ( $#
) *+,#*- . / "0%
&' ( $#
122
344$++
&' ( $#
… and some
Wireless clients …
! "#$%&' ( $#
) *+,#*- . / "0%
&' ( $#
122
344$++
&' ( $#
Add in a Campus
Services Layer …
! "#$%&' ( $#
) *' #$+
! ' , - . /
) $#012$/
31/4#15. 6"7%
&' ( $#
8) )
922$//
&' ( $#
… with some Wireless
LAN Controllers (WLCs)
! "#$%&' ($#
) *' #$+
! ' , - . /
) $#012$/
31/4#15. 6"7%
&' ($#
8) )
922$//
&' ($#
! "# ! "#
… and some Firewalls
! "#$%&' ($#
) *' #$+
! ' , - . /
) $#012$/
31/4#15. 6"7%
&' ($#
8) )
922$//
&' ($#
! "#
$%&' ( ) **
! "#
$%&' ( ) **
Form the WLCs into
a Mobility Group …
! "#$%&' ($#
) *' #$+
! ' , - . /
) $#012$/
31/4#15. 6"7%
&' ($#
8) )
922$//
&' ($#
! "#
$%&' ( ) **
! "#
$%&' ( ) **
Create the CUWN
CAPWAP overlay …
! "#$%&' ($#
) *' #$+
! ' , - . /
) $#012$/
31/4#15. 6"7%
&' ($#
8) )
922$//
&' ($#
! "#
$%&' ( ) **
! "#
$%&' ( ) **
Add in Converged
Access to the mix …
… and add in the
Data Center for the site
Internet access, dual-
homed, with RA VPN
Guest wireless access,
terminated in DMZ
Now, let’s move out
to the WAN …
First, we may have
MAN connectivity …
We may also have a
traditional WAN (T1, etc)
We may have an SP-
provided MPLS service
We may be using
DMVPN over Internet
We may be using GET
VPN over WAN/MPLS …
… or we may be using
DMVPN over 3G/4G/Sat
Branches may be single-
attached to the WAN …
Or branches may be
dual-WAN-attached
Add in remote
teleworkers …
We may have an second,
backup Data Center …
… using a variety of DCI
options for connectivity
Finally, all of this may be
virtualized “N” times …
Complexity
in Today’s Solution

Access
Dist.
Core
VLAN 22 WLAN
10.1.22.0/24
VLAN 11 Voice
10.1.11.0/24
Trunk
HSRP
VLAN 10 Data
10.1.10.0/24
VLAN 21 Voice
10.1.21.0/24
Layer 2
VLAN 20 Data
10.1.20.0/24
Multilayer
SOME VLANS Span
GLBP
VLAN 31 Voice
10.1.31.0/24
VLAN 30 Data
10.1.30.0/24
VLAN 41 Voice
10.1.41.0/24
VLAN 40 Data
10.1.40.0/24
Layer
3
Multilayer
NO VLANS Span
VLAN 51 Voice
10.1.51.0/24
P-to-P
Link
No FHRP
Needed
Layer 3
VLAN 50 Data
10.1.50.0/24
VLAN 61 Voice
10.1.61.0/24
VLAN 60 Data
10.1.60.0/24
Routed
Access
VLAN 70 Data
10.1.70.0/24
VLAN 71 Data
10.1.71.0/24
VLAN 72 Voice
10.1.72.0/24
No
FHRP
Needed
VSS
OSPF
EIGRP
BGP
Summarization
Route redistribution
Route filtering …
Custom
Topologies
VLAN 80 Data
10.1.80.0/24
VLAN 81 Data
10.1.81.0/24
VLAN 82 Voice
10.1.82.0/24
OSPF
EIGRP
OSPF
EIGRP
Many Options – All with some benefits and challenges

Enterprise Campus
Collaboration and Video Evolution
• IP Telephony (IPT) is now a mainstream technology
• Ongoing evolution to the full spectrum of Unified Communications
• High Definition Video Communications requires stringent
Service-Level Agreement (SLA)
– Reliable Service – High Availability Infrastructure
– Application Service Management – End-to-End QoS
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential

One Time Zone—Real Time
Enterprise Campus
21st Century Business Realities
Rapid Collaborative Decisions
Strict Governance for Compliance and Risk Reduction
Workers, Customers, and Partners Operate Anywhere
Resources Must be Leveraged to Their Maximum

• Introduction
Practices
Technologies
• Summary
9

Building BlockWAN Internet
SiSi SiSi SiSi SiSi SiSi SiSi
SiSi SiSi
SiSi SiSi
SiSi SiSi
SiSi
SiSi
Access
Distribution
Core
Distribution
Access
• Offers hierarchy—each layer has specific
role
• Modular topology—building blocks
• Easy to grow, understand, and
troubleshoot
• Creates small fault domains—clear
demarcations and isolation
• Promotes load balancing and redundancy
• Promotes deterministic traffic patterns
• Incorporates balance of both Layer 2 and
Layer 3 technology, leveraging the
strength of both
• Can be applied to both the multilayer
and routed campus designs
Hierarchical Network Design
Without a Rock Solid Foundation the Rest Doesn’t Matter

L2
Multilayer Campus Network Design
Layer 2 Access with Layer 3 Distribution
• Each access switch has
unique VLAN’s
• No layer 2 loops
• Layer 3 link between distribution
• No blocked links
• At least some VLAN’s span
multiple access switches
• Layer 2 loops
• Layer 2 and 3 running over link
between distribution
• Blocked links
SiSi SiSi SiSi SiSi
Vlan 10 Vlan 20 Vlan 30 Vlan 30 Vlan 30 Vlan 30
L3

Well Understood Best Practices
• Mature, 10+ year old design
• Evolved due to historical pressures
– Cost of routing vs. switching
– Speed of routing vs. switching
– Non-routable protocols
• Well understood optimization of
interaction between the various
control protocols and the topology
– STP Root and HSRP primary tuning to load
balance on uplinks
– Spanning Tree Toolkit (RootGuard,
LoopGuard, …)
– etc, …
SiSi SiSi
SiSi SiSi
BRKCRS-2031 – Multilayer Campus Architectures and Design Principals
Root
Bridge &
HSRP
Active
HSRP
Standby
CISF, BPDU Guard
LoopGuard
RootGuard

0
2
4
6
8
10
250 msec 3 secs
Good Solid Design Option
• Utilizes multiple Control
Protocols
– Spanning Tree (802.1w, …)
– FHRP (HSRP, VRRP, GLBP…)
– Routing Protocol (EIGRP, …)
• Convergence is dependent on
multiple factors
– FHRP - 900msec to 9 seconds
– Spanning Tree - 400msec to
50 seconds
• FHRP Load Balancing
– HSRP/VRRP – Per Subnet
– GLBP – Per Host
TimetorestoreVoIPdata
flows(seconds)
HSRP Hello Timers
FHRP Convergence

3/2 3/2
3/1 3/1
Switch 1 Switch 2
DST MAC 0000.0000.4444
DST MAC 0000.0000.4444
Layer 2 Loops and Spanning Tree
• Campus Layer 2 topology has sometimes proven a operational or
design challenge
• Spanning tree protocol itself is not usually the problem, it’s the
external events that triggers the loop or flooding
• L2 has no native mechanism to dampen down a problem:
– L2 fails Open, as opposed to L3 which fails closed
• Implement physical L2 loops only when you have to

• Introduction
Practices
Technologies
• Summary
15

Best Practices—Campus Routing
Leverage Equal Cost Multiple Paths
• Use routed pt2pt links and do not peer
over client VLANs, SVIs.
• ECMP used to quickly re-route around
failed node/links while providing load
balancing over redundant paths
• Tune CEF L3/L4 load balancing hash to
achieve maximum utilization of equal
cost paths (CEF polarization)
• Build triangles not squares for
deterministic convergence
• Insure redundant L3 paths to avoid
black holes
• Summarize distribution to core to limit
event propagation
• Utilized on both Multi-Layer and
Routed Access designs
Data CenterWAN Internet
Layer 3 Equal
Cost Link’s
Layer 3 Equal
Cost Link’s SiSiSiSi
SiSiSiSi
SiSi SiSiSiSiSiSi

Routed Interfaces Offer Best Convergence
Properties
• Configuring L3 routed interfaces provides for faster convergence
than a L2 switchport with an associated L3 SVI
21:32:47.813 UTC: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet2/1, changed state to down
21:32:47.821 UTC: %LINK-3-UPDOWN: Interface GigabitEthernet2/1, changed state to down
21:32:48.069 UTC: %LINK-3-UPDOWN: Interface Vlan301, changed state to down
21:32:48.069 UTC: IP-EIGRP(Default-IP-Routing-Table:100): Callback: route, adjust Vlan301
1. Link Down
2. Interface Down
3. Autostate
4. SVI Down
5. Routing Update
21:38:37.042 UTC: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet3/1, changed state to down
21:38:37.050 UTC: %LINK-3-UPDOWN: Interface GigabitEthernet3/1, changed state to down
21:38:37.050 UTC: IP-EIGRP(Default-IP-Routing-Table:100): Callback: route_adjust GigabitEthernet3/1
SiSiSiSi
L2
SiSiSiSi
L31. Link Down
2. Interface Down
3. Routing Update
~ 8 msec loss
~ 150-200 msec
loss

Best Practice—Build Triangles Not Squares
Deterministic vs. Non-Deterministic
• Layer 3 redundant equal cost links provide fast convergence
• Hardware based—fast recovery to remaining path
• Convergence is extremely fast (dual equal-cost paths: no need for OSPF or EIGRP to
recalculate a new path)
Triangles: Link/Box Failure Does Not
Require Routing Protocol Convergence
Model A
Squares: Link/Box Failure Requires
Routing Protocol Convergence
Model B
SiSi
SiSiSiSi
SiSi
SiSi
SiSiSiSi
SiSi

0
0.5
1
1.5
2
2.5
3
3.5
500 1000 5000 10000 15000 20000 25000
Convergence(sec)
ECMP ECMP (SXI2) MEC
CEF ECMP—Optimize Convergence
ECMP Convergence Is Dependent on Number of Routes
• Until recently, time to update switch HW FIB
was linearly dependent on the number of
entries (routes) to be updated
• Summarization and Filtering will decrease
RP load as well as speed up convergence
Number or Routes in Area – Sup720
SiSi
SiSi
SiSi
Time for ECMP
Recovery
Time for ECMP/MEC Unicast Recovery

CEF Load Balancing
Underutilized Redundant Layer 3 Paths
• The default CEF hash ‘input’ is L3
source and destination IP addresses
• Imbalance/overload could occur
• CEF polarization: in a multihop
design, CEF could select the same
left/left or right/right path
• Redundant paths are
ignored/underutilized
• Two solutions:
1. CEF Hash Tuning
2. CEF Universal ID
Redundant
Paths
Ignored
SiSiSiSi
SiSi SiSi
SiSi SiSi
L
L
R
R
Distribution
Default L3 Hash
Core
Default L3 Hash
Distribution
Default L3 Hash
Access
Default L3 Hash
Access
Default L3 Hash
70%
load
30%
load

SiSiSiSi
SiSi SiSi
SiSi SiSi
CEF Load Balancing
1. Avoid Polarization with CEF Hash Tuning
• With defaults, CEF could select the
same left/left or right/right paths and
ignore some redundant paths
• Alternating L3/L4 hash and default
L3 hash will give us the better load
balancing results
• The default is L3 hash—no
modification required in core
or access
• In the distribution switches use:
– mls ip cef load-sharing full
to achieve better redundant path
utilization
RL
RDistribution
L3/L4 Hash
Core
Default L3 Hash
Distribution
L3/L4 Hash
L
RL
Left Side
Shown
Access
Default L3 Hash
Access
Default L3 Hash
All Paths
Used
L

CEF Load Balancing
2. Avoid Polarization with Universal ID
• Cisco IOS uses “Universal ID” concept (also
called Unique ID) to prevent CEF polarization
– Universal ID generated at bootup (32-bit pseudo-random
value seeded by router’s base IP address)
• Universal ID used as input to ECMP hash,
introduces variability of hash result at each
network layer
• Universal ID supported on Catalyst 6500
Sup-32, Sup-720, Sup-2T
• Universal ID supported on Catalyst 4500
SupII+10GE, SupV-10GE and Sup6E
Hash using Source IP
(SIP), Destination IP (DIP)
& Universal ID
Original Src IP + Dst IP
Universal* Src IP + Dst IP + Unique ID
Include Port Src IP + Dst IP + (Src or Dst Port) + Unique ID
Default* Src IP + Dst IP + Unique ID
Full Src IP + Dst IP + Src Port + Dst Port
Full Exclude Port Src IP + Dst IP + (Src or Dst Port)
Simple Src IP + Dst IP
Full Simple Src IP + Dst IP + Src Port + Dst Port
Catalyst 4500 Load-Sharing Options Catalyst 6500 PFC3** Load-Sharing Options
SiSi SiSi
SiSi SiSi
SiSi
* = Default Load-Sharing Mode

• Introduction
Practices
Technologies
• Summary
23

Routed Access Design
Layer 3 Distribution with Layer 3 Access: no L2 Loop
• Move the Layer 2/3 demarcation to the network edge
• Leverages L2 only on the access ports, but builds a L2 loop-free network
• Design Motivations: simplified control plane, ease of
troubleshooting, highest availability
Data 10.1.20.0/24 2001:DB8:CAFE:20::/64
Voice 10.1.120.0/24 2001:DB8:CAFE:120::/64
EIGRP/OSPF EIGRP/OSPF
GLBP Model
SiSiSiSi
Layer 3
Layer 2
Layer 3
Layer 2
EIGRP/OSPF EIGRP/OSPF
SiSi SiSi
Data 10.1.40.0/24 2001:DB8:CAFE:40::/64
Voice 10.1.140.0/24 2001:DB8:CAFE:140::/64

Routed Access Advantages
Simplified Control Plane
• Simplified Control Plane
– No STP feature placement (root bridge, loopguard, …)
– No default gateway redundancy setup/tuning (HSRP,
VRRP, GLBP ...)
– No matching of STP/HSRP priority
– No asymmetric routing and unicast flooding
– No L2/L3 multicast topology inconsistencies
– No Trunking Configuration Required
• L2 Port Edge features still apply:
– Spanning Tree Portfast
– Spanning Tree BPDU Guard
– Port Security, DHCP Snooping, DAI, IPSG
– Storm Control
– 802.1x
– QoS Settings ...
SiSi
SiSiSiSi
SiSi
L3 L3 L3 L3
L3
SiSi SiSi

Simplified Network Recovery
• Routed Access network recovery is
dependent on L3 re-route
• Time to restore downstream flows is
based on a routing protocol re-route
– Time to detect link failure
– Time to determine new route
– Process the update for the SW RIB
– Update the HW FIB
• Time to restore upstream traffic flows is
based on ECMP re-route
– Time to detect link failure
– Process the removal of the lost routes from the
SW RIB
– Update the HW FIB
Upstream Recovery: ECMP
Downstream Recovery: Routing Protocol
SiSi
SiSiSiSi
SiSi
SiSi SiSi

0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
RPVST+
FHRP
OSPF EIGRP
Upstream
Downstream
Faster Convergence Times
• RPVST+ convergence times
dependent on FHRP tuning
– Proper design and tuning can
achieve sub-second times
• EIGRP converges <200 msec
• OSPF converges <200 msec
with LSA and SPF tuning
Both L2 and L3 Can Provide
Sub-Second Convergence
SiSiSiSi
SiSi SiSi

SiSi
Designated
Router
(High IP Address)
IGMP Querier
(Low IP address)
Designated
Router & IGMP
Querier
Non-DR has to
drop all non-RPF
Traffic
SiSi
SiSi SiSi
SiSi
A Single Router per Subnet: Simplified Multicast
• Layer 2 access has two multicast routers per access subnet,
RPF checks and split roles between routers
• Routed Access has a single multicast router which simplifies
multicast topology and avoids RPF check altogether

Ease of Troubleshooting
• Routing troubleshooting tools
– Consistent troubleshooting:
access, dist, core
– show ip route / show ip cef
– Traceroute
– Ping and extended pings
– Extensive protocol debugs
– IP SLA from the Access Layer
• Failure differences
– Routed topologies fail closed—i.e.
neighbor loss
– Layer 2 topologies fail open—i.e.
broadcast and unknowns flooded
SiSi
SiSiSiSi
SiSi
L3 L3 L3 L3
L3

Routed Access Design Considerations
Design Constrains
• Can’t span VLANs across multiple
wiring closet switches
+ Contained Broadcast Domains
+ But can have the same VLAN ID on all closets
• RSPAN no longer possible
– Can use ER-SPAN on Catalyst 6500
• IP addressing—do you have enough
address space and the allocation plan
to support a routed access design?
–
SiSi
SiSiSiSi
SiSi
L3 L3 L3 L3
L3
SiSi SiSi

Routed Access Design Considerations
Platform Requirements
• Catalyst Requirements
– Cisco Catalyst 3850 & 3650
– Cisco Catalyst 4500
– Cisco Catalyst 6500
• Catalyst IOS IP Base minimum feature set
– EIGRP-Stub – Edge Router
– PIM Stub – Edge Router
– OSPF for Routed Access
– 200 Dynamically Learned Routes
– Catalyst 3x00 Series IOS 12.2(55)SE
– Catalyst 4500 Series IOS 12.2(53)SG
– Catalyst 6500 Series IOS 12.2(33)SXI4
–
SiSi
SiSiSiSi
SiSi
L3 L3 L3 L3
L3
SiSi SiSi

Migrating from a L2 Access Model
• Typical deployment uses Vlan/Subnet for different user groups
• To facilitate user mobility, vlans extend to multiple closets
DHCP
DNS
10.1.20.0/24
10.1.30.0/24
...
10.1.120.0/24
VLAN 20
VLAN 30
...
VLAN 120
EIGRP/OSPF
GLBP Model
VLAN 20
VLAN 30
...
VLAN 120
VLAN 20
VLAN 30
...
VLAN 120
20,30 ... 120
User
Groups
User
Groups
interface Vlan20
ip address 10.1.20.3 255.255.255.0
ip helper-address 10.5.10.20
standby 1 ip 10.1.20.1
standby 1 timers msec 200 msec 750
standby 1 priority 150
standby 1 preempt
standby 1 preempt delay minimum 180
interface GigabitEthernet1/1
switchport
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 20-120
switchport mode trunk
switchport nonegotiate
10.5.10.20
SiSiSiSi
SiSi
SiSi

DHCP
DNS
• As the routing is moved to the access layer, trunking is no longer required
• /31 addressing can be used on p2p links to optimize ip space utilization
10.1.20.0/24
10.1.30.0/24
...
10.1.120.0/24
VLAN 20
VLAN 30
...
VLAN 120
EIGRP/OSPF
GLBP Model
VLAN 20
VLAN 30
...
VLAN 120
VLAN 20
VLAN 30
...
VLAN 120
20,30 ... 120
User
Groups
User
Groups
interface Vlan20
ip address 10.1.20.3 255.255.255.0
standby 1 preempt
switchport
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 20-120
switchport mode trunk
switchport nonegotiate
10.5.10.20
SiSiSiSi
L3
L3L3
L3 L3
SiSi
SiSi
description Distribution Downlink
ip address 10.120.0.196 255.255.255.254

DHCP
DNS
• SVI configuration at the access layer is simplified
• Larger subnets used before can simply be split into smaller ones and assigned to new DHCP scopes
10.1.20.0/24
10.1.30.0/24
...
10.1.120.0/24
VLAN 20
VLAN 30
...
VLAN 120
EIGRP/OSPF
GLBP Model
VLAN 20
VLAN 30
...
VLAN 120
User
Groups
User
Groups
interface Vlan20
ip address 10.1.20.3 255.255.255.0
standby 1 preempt
10.5.10.20
SiSiSiSi
L3
L3L3
L3 L3
interface Vlan20
ip address 10.1.20.3 255.255.255.128
10.1.20.0/25
10.1.30.0/25
...
10.1.120.0/25
10.1.20.128/25
10.1.30.128/25
...
10.1.120.128/25
SiSi
SiSi

• Introduction
• Campus Routing Foundation and Best Practices
– EIGRP Design to Route to the Access Layer
– OSPF Design to Route to the Access Layer
– Other Design Considerations
Technologies
• Summary
35

Deploying a Stable and Fast Converging EIGRP
Campus Network
•The key aspects to consider are:
1. Using EIGRP Stub at the access layer
2. Route Summarization at the distribution layer
3. Leverage Route filters
4. Consider Hello and Hold Timer tuning

EIGRP Neighbors
Event Detection
• EIGRP neighbor relationships are created when a link
comes up and routing adjacency is established
• When physical interface changes state, the routing
process is notified
– Carrier-delay should be set as a rule because
it varies based upon the platform
• Some events are detected by the
routing protocol
– Neighbor is lost, but interface is UP/UP
• To improve failure detection
– Use routed interfaces and not SVIs
– Decrease interface carrier-delay to 0
– Decrease EIGRP hello and hold-down timers*
• Hello = 1
Hold-down = 3
– * Not recommended with NSF/SSO
SiSiSiSi
ip address 10.120.0.50 255.255.255.252
ip hello-interval eigrp 100 1
ip hold-time eigrp 100 3
carrier-delay msec 0
Hellos
Routed
Interface
SiSi
SiSi
SiSi
L2 Switch
or VLAN Interface

EIGRP in the Campus
Conversion to an EIGRP Routed Edge
• The greatest advantages of EIGRP
are gained when the network has an
ip addressing plan that allows for use
of summarization and stub routers
• EIGRP allows for multiple tiers of
hierarchy, summarization and route
filtering
• Relatively painless to migrate to a L3
access with EIGRP
• Deterministic convergence time in
very large L3 topology
• EIGRP maps easily to campus
topology
10.10.0.0/1710.10.128.0/17
10.10.0.0/16
SiSi SiSi SiSi SiSi
SiSi SiSi

EIGRP Design Rules for HA Campus
Limit Query Range to Maximize Performance
• EIGRP convergence is largely dependent on query
response times
• Minimize the number of queries to speed up
convergence
• Summarize distribution block routes to limit how far
queries propagate across the campus
– Upstream queries are returned immediately with infinite cost
• Configure access switches as EIGRP
stub routers
– No downstream queries are ever sent
SiSiSiSi
SiSiSiSi
router eigrp 100
network 10.0.0.0
eigrp stub connected
interface TenGigabitEthernet 4/1
ip summary-address eigrp 100 10.120.0.0 255.255.0.0 5
router eigrp 100
network 10.0.0.0
distribute-list Default out <mod/port>
ip access-list standard Default
permit 0.0.0.0

EIGRP Query Process
Queries Propagate the Event
• EIGRP is an advanced distant vector; it
relies on its neighbor to provide routing
information
• If a route is lost and no feasible
successor is available, EIGRP
actively queries its neighbors for the lost
route(s)
• The router waits for replies from all
queried neighbors before the
calculating a new path
• If any neighbor fails to reply,
the queried route is stuck in
active and the router resets
neighbor adjacency
• The fewer routers and routes
queried, the faster EIGRP converges;
solution is to limit query propagation
SiSiSiSi
Query
SiSiSiSi
SiSiSiSi
Query
Query
Query
Query
Query
Query
Query
Query
Reply
Reply
Reply
Reply
Reply
Reply
Reply
Reply
Reply
Access
Distribution
Core
Distribution
Access

No Queries
to Rest of Network
from Core
Limiting the EIGRP Query Range
With Summarization
• When we summarize from distribution
to core for the
subnets in the access we can
limit the upstream query/
reply process
• In a large network this could be
significant because queries will now
stop at the core; no additional
distribution blocks will be involved in
the convergence event
• The access layer is still queried
SiSiSiSi
SiSiSiSi
Query Query
Query ReplyReply
Reply
Reply∞Reply∞
interface gigabitethernet 3/1
ip address 10.120.10.1 255.255.255.252
ip summary-address eigrp 1 10.130.0.0 255.255.0.0
Summary
Route
Summary
Route

Limiting the EIGRP Query Range
With Stub Routers
• A stub router signals (through hellos) that
it is a stub and not a transit path
• Queries are not sent towards the stub
routers but marked as if a “No path this
direction” reply had been received
• D1 knows that stubs cannot be transit
paths, so they will not have any path to
10.130.1.0/24
• D1 will not query the stubs, reducing the
total number of queries in this example to
one
• Stubs will not pass D1’s advertisement
of 10.130.1.0/24 to D2
• D2 will only have one path to
10.130.1.0/24
D2D1 Query
Distribution
Access
SiSi SiSi
STUB
10.130.1.0/24
Hello, I’m a
Stub—
I’m Not Going to
Send You Any
Queries Since
You Said That
Stub Stub Stub
Reply

No Queries
to Rest of Network
from Core
EIGRP Query Process
With Summarization and Stub Routers
• When we summarize from distribution
into core we can limit the upstream
query/reply process
• Queries will now stop at the core; no
additional routers will be involved in
the convergence event
• With EIGRP stubs we can further
reduce the query diameter
• Non-stub routers do not query stub
routers—so no queries will be sent to
the access nodes
• Only three nodes involved in
convergence event—No secondary
queries
SiSiSiSi
SiSiSiSi
Query Reply
Reply∞Reply∞
Stub Stub
Summary
Route
Summary
Route

SiSiSiSi
SiSiSiSi
EIGRP Route Filtering in the Campus
Control Route Advertisements
• Bandwidth is not a constraining
factor in the campus but it is still
advisable to control number of
routing updates advertised
• Remove/filter routes from the core to
the access and inject a default route
with distribute-lists
• Smaller routing table in access is
simpler to troubleshoot
• Deterministic topology
ip access-list standard Default
permit 0.0.0.0
router eigrp 100
network 10.0.0.0
distribute-list Default out <mod/port>
Default
0.0.0.0
Default
& other
Routes

SiSiSiSi
SiSiSiSi
EIGRP Routed Access Campus Design
Summary
• Detect the event:
– Set hello-interval = 1 second and hold-time =
3 seconds to detect soft neighbor failures *
– Set carrier-delay = 0
• Propagate the event:
– Configure all access layer switches as stub
routers to limit queries from the distribution
layer
– Summarize the routes from the distribution to
the core to limit queries across the campus
• Process the event:
– Summarize and filter routes to minimize
calculating new successors for the RIB and
FIB
– * Not recommended with NSF/SSO
Summary
Route
Stub
Default
0.0.0.0
Stub Stub
Default
& other
Routes

• Introduction
Technologies
• Summary
46

Deploying a Stable and Fast Converging OSPF
Campus Network
• Key Objectives of the OSPF Campus Design:
1. Map area boundaries to the hierarchical design
2. Enforce hierarchical traffic patterns
3. Minimize convergence times
4. Maximize stability of the network

OSPF Design Rules for HA Campus
Where Are the Areas?
• Area size/border is bounded by the
same concerns in the campus as
the WAN
• In campus the lower number of
nodes and stability of local links
could allow you to build larger
areas however-
• Area design also based on address
summarization
• Area boundaries should define
buffers between fault domains
• Keep area 0 for core infrastructure
do not extend to the access routers
SiSiSiSi
SiSiSiSi
SiSi
SiSiSiSiSiSi
Area 100 Area 110 Area 120
Area 0

Hierarchical Campus Design
OSPF Areas with Router Types
BGP
SiSi SiSi
SiSi SiSi
SiSi SiSi
Area 0
Area 200
Area 20 Area 30Area 10
BackboneBackbone
ABR ABR
InternalInternal
Area 0
ABR
Area 100
ASBR
ABR
ABR
Area 300
Access
Distribution
Core
Distribution
Access
SiSi
SiSi

OSPF in the Campus
Conversion to an OSPF Routed Edge
• OSPF designs that utilize an area
for each campus distribution
building block allow for straight
forward migration to Layer 3
access
• Converting L2 switches to L3
within a contiguous area is
reasonable to consider as long as
new area size is reasonable
• How big can the area be?
– It depends
– Switch type(s)
– Number of links
– Stability of fiber plant
Area 200
Branches
Area 0
Core
Area 10
Dist 1
Area 20
Dist 2
SiSi SiSi SiSi SiSi
SiSiSiSi

When a Link Changes State
• Every router in area
hears a specific
link LSA
• Each router computes
shortest path
routing table
Router 2, Area 1
Old Routing Table New Routing Table
Link State Table
LSA
Dijkstra Algorithm
ACKSiSi
Router 1, Area 1

OSPF LSA Process
LSAs Propagate the Event
• OSPF is a Link State protocol; it relies
on all routers within an area having the
same topology view of the network.
• If a route is lost, OSPF sends out an
LSA to inform it’s peers within the area
of the lost route.
• All routers with knowledge of this route
in the OSPF network will receive an
LSA and run SPF to remove the lost
route.
• The fewer the number of
routers with knowledge of the
route, the faster OSPF converges;
• Solution is to limit LSA
propagation range
SiSiSiSi
LSA 2
SiSiSiSi
SiSiSiSi
LSA 2
LSA 2
LSA 2
LSA 2
LSA 2
LSA 2
LSA 2
LSA 2
Area 0
Area 0
SPF
SPF SPF
SPF
SPFSPF
SPF SPF
SPF SPF
Access
Distribution
Core
Distribution
Access

SiSiSiSi
Backbone
Area 0
Area 120
OSPF Regular Area
ABRs Forward All LSAs from Backbone
ABR Forwards the
Following into an Area
Summary LSAs (Type 3)
ASBR Summary (Type 4)
Specific Externals (Type 5)
Access Config:
router ospf 100
network 10.120.0.0 0.0.255.255 area 120
Distribution Config
router ospf 100
area 120 range 10.120.0.0 255.255.0.0 cost 10
network 10.120.0.0 0.0.255.255 area 120
network 10.122.0.0 0.0.255.255 area 0
SiSiSiSi
External Routes/LSA Present in Area 120

SiSiSiSi
Backbone
Area 0
Area 120
OSPF Stub Area
Consolidates Specific External Links—Default 0.0.0.0
Stub Area ABR Forwards
Summary LSAs
Summary 0.0.0.0 Default
Distribution Config
router ospf 100
area 120 stub
area 120 range 10.120.0.0 255.255.0.0 cost 10
network 10.120.0.0 0.0.255.255 area 120
network 10.122.0.0 0.0.255.255 area 0
SiSiSiSi
Access Config:
router ospf 100
network 10.120.0.0 0.0.255.255 area 120
Eliminates External Routes/LSA Present in Area (Type 5)

SiSi
Backbone
Area 0
Area 120
A Totally Stubby Area
ABR Forwards
Summary Default
OSPF Totally Stubby Area
Use This for Stable—Scalable Internetworks
Distribution Config
router ospf 100
area 120 stub no-summary
area 120 range 10.120.0.0 255.255.0.0 cost 10
network 10.120.0.0 0.0.255.255 area 120
network 10.122.0.0 0.0.255.255 area 0
Access Config:
router ospf 100
network 10.120.0.0 0.0.255.255 area 120
SiSi
SiSi
SiSi
Minimize the Number of LSAs and the Need for Any
External Area SPF Calculations

SiSi
Backbone
Area 0
Area 120
Area Border Router
ABRs Forward
Summary 10.120.0.0/16
Summarization Distribution to Core
Reduce SPF and LSA Load in Area 0
Access Config:
router ospf 100
network 10.120.0.0 0.0.255.255 area 120
Distribution Config
router ospf 100
area 120 stub no-summary
area 120 range 10.120.0.0 255.255.0.0 cost 10
network 10.120.0.0 0.0.255.255 area 120
network 10.122.0.0 0.0.255.255 area 0
SiSi
SiSiSiSi
Minimize the Number of LSAs and the Need for Any SPF
Recalculations at the Core

SiSiSiSi
SiSiSiSi
OSPF Design Considerations
What Area Should the Distribution Link Be In?
• Two aspects of OSPF behavior can
impact convergence
– OSPF ABRs ignore LSAs generated by other
ABRs learned through non-backbone areas
when calculating least-cost paths
– In a stub area environment the ABR will
generate a default route when any type
of connectivity to the backbone exists
• Ensure loopbacks are ‘not’ in area 0
• Configure dist to dist link as a trunk
using 2 subnets one in area 0 and one
in stub area when possible

SiSi
SiSi
OSPF Timer Tuning
High-Speed Campus Convergence
• OSPF by design has a number of
throttling mechanisms to prevent the
network from thrashing during periods of
instability
• Campus environments are candidates to
utilize OSPF timer enhancements
– Sub-second hellos*
– Generic IP (interface) dampening
mechanism
– Back-off algorithm for LSA generation
– Exponential SPF backoff
– Configurable packet pacing Reduce
LSA and SPF
Interval
SiSi
SiSi
Reduce Hello
Interval
* Not recommended with NSF/SSO

Access Config:
dampening
ip ospf dead-interval minimal hello-multiplier 4
ip ospf network point-to-point
router ospf 100
timers throttle spf 10 100 5000
timers throttle lsa all 10 100 5000
timers lsa arrival 80
Subsecond Hellos
Neighbor Loss Detection—Physical Link Up
• OSPF hello/dead timers detect neighbor loss
in the absence of physical link loss
• Useful in environments where an
L2 device separates L3 devices
(Layer 2 core designs)
• Aggressive timers quickly detect
neighbor failure
• Not recommended with NSF/SSO
• Interface dampening is recommended with
sub-second hello timers
• OSPF point-to-point network type to avoid
designated router (DR) negotiation.
OSPF
Processing
Failure
(Link Up)
A B
SiSi
SiSi
SiSi
SiSi

5.68
0.72
0.24
0
1
2
3
4
5
6
Default
Convergence
10 msec. SPF 10 msec. SPF
and LSA
OSPF Requires Sub-Second Throttling of LSA
Timers to Speed Convergence
• OSPF has an SPF throttling timer designed
to dampen route recalculation
• After a failure, the router waits for the SPF
timer to expire before recalculating
a new route
• By default, there is a 500ms delay before
generating router and network LSAs; the
wait is used to collect changes during a
convergence event and minimize the
number of LSAs sent
• Propagation of a new instance
of the LSA is limited at the originator
• Acceptance of a new LSAs is limited by the
receiver
• Make sure lsa-arrival < lsa-hold
TimetoRestoreVoiceFlows(sec)

OSPF Design Rules for HA Campus
LSA/SPF Exponential Back-off Throttle Mechanism
• Sub-second timers without risk
1. spf-start or initial hold timer controls how long to wait prior to starting the SPF
calculation
2. If a new topology change event is received during the hold interval, the SPF
calculation is delayed until the hold interval expires and the hold interval is
temporarily doubled
3. The hold interval can grow until the maximum period configured is reached
4. After the expiration of any hold interval, the timer is reset
timers throttle spf <spf-start> <spf-hold> <spf-max-wait>
timers throttle lsa all <lsa-start> <lsa-hold> <lsa-max-wait>
Time [ms]
Topology Change Events
SPF Calculations
200 1600 msec100 400 800 msec

• Introduction
Technologies
• Summary
62

Routing Protocol Churn Can Be Reduced with IP
Event Dampening
• Prevents routing protocol churn caused by
constant interface state changes
• Dampening is applied on a system: nothing
is exchanged between routing protocols
• Supports all IP routing protocols
– Static routing, RIP, EIGRP, OSPF, IS-IS, BGP
– In addition, it supports HSRP and CLNS routing
– Applies on physical interfaces and can’t be applied on
subinterfaces individually
Up
Up
Interface State Perceived by EIGRP or OSPF
Interface State
description Uplink to Distribution 1
dampening
ip address 10.120.0.205 255.255.255.254
Down
Up
Down
SiSi
SiSiSiSi
Up
Down
Up
Up
Down
Down

Using Redundant Supervisors at the Access Layer
with SSO
1. Supervisor switchover event occurs
2. SSO maintains SSO-aware applications,
including L2 tables, L2/L3 forwarding is
maintained
3. Routing protocols will restart on the newly
active Supervisor
– L3 routes are purged stopping L3 forwarding
4. Routing neighbors lose adjacency with the
restarting router
– Routes to the lost neighbor are purged
5. Routing neighbors reestablish
adjacencies, forwarding to and from non-
directly connected L3 networks resumes
SiSiSiSi
SiSi SiSi
SSO alone is not enough with a Routed Access
do not run SSO w/o NSF in the RA design

NSF—Configuration and Monitoring
Switch(config)#router eigrp 100
Switch(config-router)#nsf
Router#sh ip ospf
Routing Process "ospf 100" with ID 10.120.250.4
Start time: 00:01:37.484, Time elapsed: 3w2d
Supports Link-local Signaling (LLS)
<snip>
Non-Stop Forwarding enabled, last NSF restart
3w2d ago (took 31 secs)
Router#sh ip protocol
*** IP Routing is NSF aware ***
Routing Protocol is "eigrp 100 100"
<snip
EIGRP NSF-aware route hold timer is 240s
EIGRP NSF enabled
EIGRP
Switch(config)#router ospf 100
Switch(config-router)#nsf
NSF-Capable
NSF-Aware
OSPF
Recommendation Is to Not Tune IGP Hello Timers. Use Default Hello and Dead
Timers for EIGRP/OSPF When Peering to a Device Configured for NSF/SSO

Using Redundant Supervisors at the Access Layer,
Now with NSF/SSO
1. Supervisor switchover event occurs
2. SSO maintains SSO-aware applications,
including L2 tables, L2/L3 forwarding is
maintained
3. NSF-capable router signals NSF-aware
routing peers of a routing protocol restart
4. NSF-aware routers detect the restarting
router
– Assist in re-establishing full adjacency
– Maintain forwarding to and from the
restarting router
5. NSF restart complete, traditional L3
convergence event is avoided
2
SiSiSiSi
SiSi SiSi
1
4
3

SiSiSiSi
SiSiMaster
Access
S1 S2 S3
Single logical Switch
SiSiSiSi
Design Consideration with StackWise at the Access
Layer
• Recommended Design:
– Configure priority for master and its backup for
deterministic failures
– Avoid using master as uplink to reduce uplink
related losses
– Use “stack-mac persistent timer 0” to avoid the
gratuitous ARP changes for
• Best convergence
• Where GARP processing is disabled in the
network, e.g. Security
• Where network devices/host do not support
GARP, e.g. Phones
• Upstream traffic is not interrupted by
master failure
• Downstream traffic is interrupted due to
routing protocol restart and adjacency
reset
– Run 12.2(37)SE or higher for NSF support

Routed Access Does Not Require Switch
Management Vlan
• In the L2 design it was considered a best practice
to define a unique Vlan for network management
• In the routed access model, the best way is to
configure a loopback interface
• The /32 address should belong to the summarized
routed advertised from the distribution block
• The loopback interface should be configured as
passive for the IGP
• ACLs should be used as required to ensure secure
network management
SiSi
SiSiSiSi
SiSi
SiSi SiSi
SNMP
Server
interface Loopback0
description Dedicated Switch Management
ip address 10.120.254.1 255.255.255.255

• Introduction
Practices
Technologies
• Summary
69

Virtual Switch
Catalyst 6500 Virtual Switching System (VSS)
• Virtual Switching System consists of two Catalyst 6500’s defined as members
of the same virtual switch domain running a VSL (Virtual Switch Link) between
them
• Single Control Plane with Dual Active Forwarding Planes
• Extends NSF/SSO infrastructure to Two Switches
VSS
SiSiSiSi
Switch 1 + Switch 2 =
Virtual Switch Domain
Virtual Switch Link (VSL)

Virtual Switch System
Impact to the Campus Topology
 Physical network topology does not
change
Still have redundant chassis
Still have redundant links
 Logical topology is simplified as we now
have a single control plane
 Allows the design to replace traditional
topology control plane with Multi-chassis
Etherchannel (MEC)
No reliance on IGP Protocol to provide link
redundancy
Convergence and load balancing are based
on Etherchannel
SiSiSiSi SiSiSiSi SiSiSiSi SiSiSiSi
BRKCRS-3035 – Advance Enterprise Campus Design: Virtual Switching System (VSS)
SiSiSiSi

VSS and Routed Access Design
Link Down Convergence Without VSS
• Downstream traffic recovery is
dependent upon the Interior
Gateway Protocol reroute to the
peer distribution switch
– Use Stub on the access devices, and
proper summarization from distribution
– Tune IGP ... etc.
• Upstream traffic recovery is
dependent upon updates to the
Access Switch’s Forwarding
Information Base removing the
adjacency for the lost link (ECMP)
Downstream IGP reroute
Upstream CEF ECMP
SiSi
SiSi
SiSi
SiSi
L3 ECMP

• Access layer switch has one neighbor
• Distribution switch has neighbor count
reduced by half
• Upstream and Downstream traffic
convergence now is an Etherchannel
link event
– No IGP reconvergence event
– No Impact of number of routes/vlans
• Fast IGP Timers not needed nor
recommended (only 1 IGP peer)
• Summarization rules still recommended
• Achieves sub-second failure and no L2
loop on the topology
VSS and Routed Access Design
Link Down Convergence with VSS MEC
Downstream IGP reroute
Upstream CEF ECMP
SiSi
SiSi
SiSi
SiSi
L3 ECMPMEC

• Introduction
Practices
• Routed Access Design for IPv6
Technologies
• Summary
74

Analyzing the Impact on Advanced Technologies
• Unified Communications Deployments work the same way. You still
need to provision a voice vlan/subnet per wiring closet switch
• TrustSec (802.1x) solutions work the same: user vlan assigment still
possible, as well as per user dACL (checkout BRKSEC-2005)
• Wireless LAN works seamlessly as well, since LWAPP works with
UDP hence at L3.
• We will take a closer look at;
– Network Virtualization

• Access control techniques remain the same with a Routed
Access Model
• Path Isolation techniques remain the same, but there are provisioning
implications by running routing at the access layer
Network Virtualization
Functional Architecture
Access Control Path Isolation Services Edge
WAN – MAN – CampusBranch – Campus Data Center – Internet Edge –
Campus
Ethernet
VRFs
GRE
VRFs
MPLS
VPNs
BRKCRS-2033 – Deploying a Virtualized Campus Network Infrastructure

VRF
VRF
Global
Path Isolation
Functional Components
• Device virtualization
–Control plane virtualization
–Data plane virtualization
–Services virtualization
• Data path virtualization
–Hop-by-Hop
–(VRF-Lite End-to-End)
–Multi-Hop
–(VRF-Lite+GRE, MPLS-VPN)
VRF: Virtual Routing and Forwarding
Per VRF
Virtual Routing Table
Virtual Forwarding Table
IP
802.1q

Network Virtualization and Routed Access
Path Isolation Issues—VRFs to the Edge
• Define VRFs on the access layer
switches
• One VRF dedicated to each virtual
network (Red, Green, etc.)
• Map device VLANs to the
corresponding VRF
• Provisioning is more challenging,
because multiple routing processes
and logical interfaces are required.
• The chosen path isolation
technique must be deployed
from the access layer devices
VRF-lite Ethernet
– VRF-Lite GRE
– MPLS L3 VPNs
Campus
Core
Layer 3
Links
SiSiSiSi
VLAN 21 Red
VLAN 22 Green
VLAN 23 Blue
VLAN 21 Red
VLAN 22 Green
VLAN 23 Blue
VRF Blue
VRF Green
VRF Red

Network Virtualization and Routed Access
Path Isolation Issues—VRFs to the Edge (Cont.)
• Catalyst 6500 supports all three
path isolation techniques:
– 802.1Q Ethernet VRF-Lite
– GRE with VRF-Lite
– MPLS VPN
• Catalyst 3000s and 4500s only
support 802.1Q Ethernet VRF-Lite
• Convergence times increase
– ~800ms for 9 VRFs + Global
– Increased load from multiple routing
processes and logical interfaces
• Operational impact of managing
multiple logical networks
Campus
Core
Layer 3
Links
SiSiSiSi
VLAN 21 Red
VLAN 22 Green
VLAN 23 Blue
VLAN 21 Red
VLAN 22 Green
VLAN 23 Blue
VRF Blue
VRF Green
VRF Red
Network Virtualization--Path Isolation Design Guide
http://www.cisco.com/en/US/docs/solutions/Enterprise/Network_Virtualization/PathIsol.html#wp277205

• Introduction
Practices
• Routed Access Design for IPv6
Technologies
• Summary
82

= STP Blocked Link
STP-Based
Redundant Topology
B
Routed Access
Redundant Topology
SiSi SiSi
SiSi SiSi
SiSi SiSi
SiSi SiSi SiSi SiSi
SiSi SiSi
SiSi SiSi
SiSi SiSi
Routed Access Campus Design
End to End Routing: Fast Convergence and Maximum Reliability
B
B
B
B

Summary
• Traditional Layer 2 designs
remain valid
• Routed Access Design:
– Simplified Control Plane (no
dependence on STP, HSRP, etc.)
– Increased Capacity: Provide flow-
based load balancing
– High Availability: 200 msec or better
recovery
– Simplified Multicast
– No L2 Loops
– Easy Troubleshooting
• Flexibility to provide for the right
implementation for
each network requirement
SiSi SiSi SiSi SiSi
SiSi SiSi

Campus Design Guidance
Where To Go for More Information
http://www.cisco.com/go/srnd

Call to Action
• Visit the World of Solutions for
– Cisco
– Walk in Labs
– Technical Solution Clinics
• Meet the Engineer
• Lunch time Table Topics
• DevNet zone related labs and sessions
• Recommended Reading: for reading material and further resources for this
session, please visit www.pearson-books.com/CLMilan2015
87

Complete Your Online Session Evaluation
• Please complete your online session
evaluations after each session.
Complete 4 session evaluations
& the Overall Conference Evaluation
(available from Thursday)
to receive your Cisco Live T-shirt.
• All surveys can be completed via
the Cisco Live Mobile App or the
Communication Stations
88

Interested in Learning about Next Gen Solutions?
• Have your account team setup a
meeting @ Enterprise Segment
Innovation Forum
• Requirements
– Cisco Account Team Presence
– Cisco NDA in Place
• Please use the address if you have
any queries…
ciscolive-ebc-eng-sw@external.cisco.com
• We are at
MiCo - Milano Congressi, Piazzale Carlo
Magno 1, 20149 Milano Italy, Meeting
Village, North Building, Level 1
89

Continue Your Education
• Demos in the Cisco Campus
• Walk-in Self-Paced Labs
• Table Topics
• Meet the Engineer 1:1 meetings
90

Advanced enterprise campus design. routed access (2015 milan)

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Advanced enterprise campus design. routed access (2015 milan) (20)

Recently uploaded (20)

Advanced enterprise campus design. routed access (2015 milan)