© 2009 cisco systems, inc. all rights reserved.cisco confidential 1 roberto mari technical...

56
09 Cisco Systems, Inc. All rights reserved. Cisco Confidential 1 Roberto Mari Technical Marketing Engineer Data Center Business Unit November 2009 version 1.1 Nexus 7000 virtual Port-Channel Best Practices & Design Guidelines

Post on 20-Dec-2015

214 views

Category:

Documents


0 download

TRANSCRIPT

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 1

Roberto MariTechnical Marketing Engineer

Data Center Business UnitNovember 2009

version 1.1

Nexus 7000 virtual Port-Channel

Best Practices & Design Guidelines

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 2

Agenda Feature Overview & Terminology

vPC Design Guidance & Best PracticesBuilding a vPC domain

Attaching to a vPC domain

Layer 3 and vPC

Spanning Tree Recommendations

Data Center Interconnect (& Encryption)

HSRP with vPC

vPC and Services

vPC latest enhancements

ISSU

Convergence and Scalability

vPC Hands-on Lab Information

Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 4

Allow a single device to use a port channel across two upstream switches

Eliminate STP blocked ports

Uses all available uplink bandwidth

Dual-homed server operate in active-active mode

Provide fast convergence upon link/device failure

Reduce CAPEX and OPEX

Available on current and future hardware for M1 and D1 generation cards.

Logical Topology without vPC

Logical Topology with vPC

Feature Overview & TerminologyvPC Definition

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 5

vPC peer – a vPC switch, one of a pair

vPC member port – one of a set of ports (port channels) that form a vPC

vPC – the combined port channel between the vPC peers and the downstream device

vPC peer-link – Link used to synchronize state between vPC peer devices, must be 10GbE

vPC peer-keepalive link – the keepalive link between vPC peer devices, i.e., backup to the vPC peer-link

vPC VLAN – one of the VLANs carried over the peer-link and used to communicate via vPC with a peer device.

non-vPC VLAN – One of the STP VLANs not carried over the peer-link

CFS – Cisco Fabric Services protocol, used for state synchronization and configuration validation between vPC peer devices

vPC

vPC peer

non-vPC device

vPC peer-keepalive link

vPC member

port

vPCvPC

member port

CFS protocol

vPC peer-link

Feature Overview & Terminology vPC Terminology

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 6

Building a vPC DomainConfiguration Steps

Following steps are needed to build a vPC (Order does Matter!)

1. Configure globally a vPC domain on both vPC devices

2. Configure a Peer-keepalive link on both vPC peer switches (make sure is operational)NOTE: When a vPC domain is configured the keepalive must be operational to allow a vPC domain to successfully form.

3. Configure (or reuse) an interconnecting port-channel between the vPC peer switches

4. Configure the inter-switch channel as Peer-link on both vPC devices (make sure is operational)

5. Configure (or reuse) Port-channels to dual-attached devices

6. Configure a unique logical vPC and join port-channels across different vPC peers

vPC vPC member port

vPC peer-keepalive link

vPC peer-link

Standalone Port-channel

vPC peer

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 7

Building a vPC DomainPeer Link

Definition:

Standard 802.1Q Trunk

Can Carry vPC and non vPC VLANs*Carries Cisco Fabric Services messages (tagged as CoS=4

for reliable communication)

Carries flooded traffic from a vPC peer

Carries STP BPDUs, HSRP Hellos, IGMP updates, etc.

Requirements:

Member ports must be 10GE interfaces one of the N7K-M132XP-12 modules

Peer-link are point-to-point. No other device should be inserted between the vPC peers.

Recommendations (strong ones!)

Minimum 2x 10GbE ports on separate cards for best resiliency.

Dedicated 10GbE ports (not shared mode ports)*It is Best Practice to split vPC and non-vPC VLANs on different Inter-switch Port-Channels.

vPC peer-link

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 8

Common Nexus 7000 configuration:1x 10G, 7x 1G cards

vPC recommendation is 2 10G cards

Potential problem occurs if Nexus 7000 is L3 boundary with single 10G card

Use Object Tracking Feature available in 4.2

More information from CCO:

http://www.cisco.com/en/US/docs/switches/datacenter/sw/4_2/nx-os/interfaces/configuration/guide/if_vPC.html#wp1529488

Building a vPC DomainPeer Link with Single 10G Module

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 9

Scenario:

vPC deployments with a single N7K-M132XP-12 card, where core and peer-link interfaces are localized on the same card.

This scenario is vulnerable to access-layer isolation if the 10GE card fails on the primary vPC.

vPC Object Tracking Solution:

Leverages object tracking capability in vPC (new CLI commands are added).

Peer-link and Core interfaces are tracked as a list of boolean objects.

vPC object tracking suspends vPCs on the impaired device, so traffic can get diverted over the remaining vPC peer.

e1/…

e1/…

e1/…

e1/…

e1/… e1/… e1/… e1/…

vPCPrimary

e2/… e2/… vPCSecondary

vPC PL

vPC PKL

L3

L2

rhs-7k-1(config-vpc-domain)# track <object>

Building a vPC DomainPeer Link with Single 10G Module – Object Tracking

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 11

Building a vPC DomainPeer-Keepalive (1 of 2)

Definition:

Heartbeat between vPC peers

Active/Active (no Peer-Link) detection

Messages sent on 2 second interval

3 second hold timeout on peer-link loss

Fault Tolerant terminology is specific to VSS and deprecated in vPC.

Packet Structure:

UDP message on port 3200, 96 bytes long (32 byte payload), includes version, time stamp, local and remote IPs, and domain ID.

Keepalive messages can be captured and displayed using the onboard Wireshark Toolkit.

Recommendations:

Should be a dedicated link (1Gb is adequate)

Should NOT be routed over the Peer-Link

Can optionally use the mgmt0 interface (along with management traffic)

As last resort, can be routed over L3 infrastructure

vPC peer-keepalive link

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 12

Building a vPC DomainPeer-Keepalive (2 of 2)

Cautions/Additional Recommendations:

When using supervisor management interfaces to carry the vPC peer-keepalive, do not connect them back to back between the two switches.

Only one management port will be active a given point in time and a supervisor switchover may break keep-alive connectivity

Use the management interface only if you have an out-of-band management network (management switch in between).

vPC1 vPC2

vPC_PL

Management Network

Standby Management InterfaceActive Management Interface

Management Switch

vPC_PKvPC_PK

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 13

Definition:

Port-channel member of a vPC peer.

Requirements:

Configuration needs to match other vPC peer’s member port config.

In case of inconsistency a VLAN or the entire port-channel may suspend (i.e. MTU mismatch).

Number of member ports on both vPC peers is not required to match.

Up to 8 active ports between both vPC peers (16-way port-channel can be build with multi-layer vPC)

vPC member

port

vPC member

port

Building a vPC DomainvPC Member Port

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 14

vPC works seamlessly in any VDC based environment. 

One vPC domain per VDC is supported, up to the maximum number of VDCs supported in the system.

It is still necessary to have a separate vPC peer-link and vPC Peer-Keepalive Link infrastructure for each VDC deployed.

Can vPC run between VDCs on the same switch?

This scenario should technically work, but it is NOT officially supported and has not been extensively tested by our QA team.

Could be useful for Demo or hands on, but It is NOT recommended for production environments. Will consolidate redundant points on the same box with VDCs (e.g. whole aggregation layer on a box) and introduce a single point of failure.

ISSU will NOT work in this configuration, because the vPC devices can NOT be independently upgraded.

Building a vPC DomainVDC Interaction

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 15

Agenda Feature Overview & Terminology

vPC Design Guidance & Best PracticesBuilding a vPC domain

Attaching to a vPC domain

Layer 3 and vPC

Spanning Tree Recommendations

Data Center Interconnect (& Encryption)

HSRP with vPC

vPC and Services

vPC latest enhancements

ISSU

Convergence and Scalability

vPC Hands-on Lab Information

Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 16

Attaching to a vPC domainThe One and Only Rule…

ALWAYSdual attach

devices to a vPC Domain!!!

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 17

Definition:Port-channel for devices for devices dual-attached to

the vPC pair.

Provides local load balancing for port-channel members

STANDARD 802.3ad port channel

Access Device RequirementsSTANDARD 802.3ad capability

LACP Optional

Recommendations:

Use LACP when available for better failover and mis-configuration protection

vPC member

port

vPC

RegularPort-

channel port

Attaching to a vPC DomainIEEE 802.3ad and LACP

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 18

* VLAN that is NOT part of any vPC and not present on vPC peer-link

Attaching to a vPC Domain”My device can’t be dual attached!”

Recommendations (in order of preference):1. ALWAYS try to dual attach devices using vPC (not applicable for routed links).

PROS: Ensures minimal disruption in case of peer-link failover and consistent behavior with vPC dual-active scenarios. Ensures full redundant active/active paths through vPC.

CONS: None

2. If (1) is not an option – connect the device via a vPC attached access switch (could use VDC to create a “virtual access switch”).

PROS: Ensures minimal disruption in case of peer-link failover and consistent behavior with vPC dual-active scenarios. Availability limited by the access switch failure.

CONS: Need for an additional access switch or need to use one of the available VDCs. Additional administrative burden to configure/manage the physical/Virtual Device

3. If (2) is not an option – connect device directly to (primary) vPC peer in a non-vPC VLAN* and provide for a separate interconnecting port-channel between the two vPC peers.

PROS: Traffic diverted on a secondary path in case of peer-link failover

CONS: Need to configure and manage additional ports (i.e. port-channel) between the Nexus 7000 devices.

4. If (3) is not an option – connect device directly to (primary) vPC peer in a vPC VLAN

PROS: Easy deployment

CONS: VERY BAD. Bound to vPC roles (no role preemption in vPC) , Full Isolation on peer-link failure when attached vPC toggles to a secondary vPC role.

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 19

Attaching to a vPC DomainvPC and non-vPC VLANs (i.e. single attached .. )

Orphan Ports

Orphan Ports

SS

S

S

P P

P

P

1. Dual Attached 2. Attached via VDC/Secondary Switch

3. Secondary ISL Port-Channel 4. Single Attached to vPC Device

Primary vPC

Secondary vPCS

P

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 20

* Run the same STP mode as the vPC domain. Enable portfast/port type edge on host facing ports

Attaching to a vPC Domain”My device only does STP!”

Recommendations (in order of preference):

1. ALWAYS try dual attach devices using vPC

PROS: Ensures minimal disruption in case of peer-link failover and consistent behavior with vPC dual-active scenarios. Ensures full redundant active/active paths through vPC.

CONS: None

2. If (1) is not an option – connect the device via two independent links using STP. Use non-vPC VLANs ONLY on the STP switch.*

PROS: Ensures minimal disruption in case of peer-link failover and consistent behavior with vPC dual-active scenarios. Ensures full redundant Active/Active paths on vPC VLANs.

CONS: Requires an additional STP port-channel between the vPC devices. Operational burden in provisioning and configuring separate STP and vPC VLAN domains. Only Active/Standby paths on STP VLANs.

3. If (2) is not an option – connect the device via two independent links using STP. (Use vPC VLANs on this switch)

PROS: Simplify VLAN provisioning and does not require allocation of an additional 10GE port-channel.

CONS: STP and vPC devices may not be able to communicate each other in certain failure scenarios (i.e. when STP Root and vPC primary device do not overlap). All VLANs carried over the peer-link may suspend until the two adjacency forms and vPC is fully synchronized".

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 21

SS

S

P

P

P

1. All devices Dual Attached via vPC 2. Separate vPC and STP VLANs

3. Overlapping vPC and STP VLANs

Primary vPC

Secondary vPC

Primary STP Root

Secondary STP Root

S

P

SR

PR

PRSR

SR PR

vPC Design principlesAttaching to a vPC Domain - vPC and non-vPC VLANs (STP/vPC Hybrid)

Non vPC port-channel

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 22

Multi-Layer vPC can join 8 active ports port-channels in a unique 16-way port-channel*

vPC peer side load-balancing is LOCAL to the peer

Each vPC peer has only 8 active links, but the pair has 16 active load balanced links

Nexus 7000

Nexus 5000

* Possible with any device supporting vPC/MCEC and 8-way active port-channels

16-way port channel

Attaching to a vPC Domain16-way Port-Channel (1 of 2)

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 23

16 active ports between 8 active port-channel devices and 16 active port-channel devices?

vPC peer side load-balancing is LOCAL to the peer

Each vPC peer has only 8 active links, but the pair has 16 active load balanced links to the downstream device supporting 16 active ports

D-series N7000 line cards will also support 16 way active port-channel load balancing, providing for a potential 32 way vPC port channel!

Nexus 7000

Nexus 5000

Nexus 5000 16-port port-channel support introduced in 4.1(3)N1(1a) release

16-port port-channel

Attaching to a vPC Domain16-way Port-Channel (2 of 2)

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 24

Agenda Feature Overview & Terminology

vPC Design Guidance & Best PracticesBuilding a vPC domain

Attaching to a vPC domain

Layer 3 and vPC

Spanning Tree Recommendations

Data Center Interconnect (& Encryption)

HSRP with vPC

vPC and Services

vPC latest enhancements

ISSU

Convergence and Scalability

vPC Hands-on Lab Information

Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 25

Router

7k1 7k2

Switch

Po1

Po2

Use separate L3 links to hook up routers to a vPC domain is still standing.

Don’t use L2 port channel to attach routers to a vPC domain unless you can statically route to HSRP address

If both, routed and bridged traffic is required, use individual L3 links for routed traffic and L2 port-channel for bridged traffic

Router

Switch

L3 ECMP

Po2

Layer 3 and vPCRecommendations

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 26

vPC view Layer 2 topology Layer 3 topology

Port-channel looks like a single L2 pipe. Hashing will decide which link to

chose

Layer 3 will use ECMP for northbound traffic

7k1 7k2

R

7k1 7k2

R

7k vPC

R

R could be any router, L3 switch or VSS

building a port-channel

Layer 3 and vPCWhat can happen… (1 of 3)

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 27

1) Packet arrives at R

2) R does lookup in routing table and sees 2 equal paths going north (to 7k1 & 7k2)

3) Assume it chooses 7k1 (ECMP decision)

4) R now has rewrite information to which router it needs to go (router MAC 7k1 or 7k2)

5) L2 lookup happens and outgoing interface is port-channel 1

6) Hashing determines which port-channel member is chosen (say to 7k2)

7) Packet is sent to 7k2

8) 7k2 sees that it needs to send it over the peer-link to 7k1 based on MAC address

R

7k1 7k2

S

Po1

Po2

Layer 3 and vPCWhat can happen… (2 of 3)

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 28

9) 7k1 performs lookup and sees that it needs to send to S

10) 7k1 performs check if the frame came over peer link & is going out on a vPC.

11) Frame will only be forwarded if outgoing interface is NOT a vPC or if outgoing vPC doesn’t have active interface on other vPC peer (in our example 7k2)

R

7k1 7k2

S

Po1

Po2

Layer 3 and vPCWhat can happen… (3 of 3)

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 29

Agenda Feature Overview & Terminology

vPC Design Guidance & Best PracticesBuilding a vPC domain

Attaching to a vPC domain

Layer 3 and vPC

Spanning Tree Recommendations

Data Center Interconnect (& Encryption)

HSRP with vPC

vPC and Services

vPC latest enhancements

ISSU

Convergence and Scalability

vPC Hands-on Lab Information

Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 30

Spanning Tree RecommendationsOverview – STP Interoperability

STP Uses:• Loop detection (failsafe to vPC)

• Non-vPC attached device

• Loop management on vPC addition/removal

Requirements:• Needs to remain enabled, but doesn’t dictate vPC member

port state

• Logical ports still count, need to be aware of number of VLANs/port-channels deployed!

Best Practices:• Not recommended to enable Bridge Assurance feature on

vPC channels (i.e. no STP “network” port type). Tracked by CSCsz76892.

• Make sure all switches in you layer 2 domain are running with Rapid-PVST or MST (IOS default is non-rapid PVST+), to avoid slow STP convergence (30+ secs)

• Remember to configure portfast (edge port-type) on host facing interfaces to avoid slow STP convergence (30+ secs)

vPCvPCSTP is running to manage loops outside of vPC’s

direct domain, or before initial vPC configuration

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 31

Spanning Tree RecommendationsPort Configuration Overview

Aggregation

Access

Data Center Core

B

L

R

N

E

BPDUguard

Loopguard

Rootguard

Network port

Edge or portfast port type

- Normal port type

B

RR

N N

- - -

-

-

- - -RRRRRR

--

B

E

BB

E

B

E

Layer 3

Layer 2 (STP + Rootguard)

Layer 2 (STP + BPDUguard)

L

E

SecondaryRoot

HSRPSTANDBY

PrimaryRoot

HSRPACTIVE

E

-

PrimaryvPC

SecondaryvPC

vPCDomain

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 32

Agenda Feature Overview & Terminology

vPC Design Guidance & Best PracticesBuilding a vPC domain

Attaching to a vPC domain

Layer 3 and vPC

Spanning Tree Recommendations

Data Center Interconnect (& Encryption)

HSRP with vPC

vPC and Services

vPC latest enhancements

ISSU

Convergence and Scalability

vPC Hands-on Lab Information

Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 33

Long DistanceDC 1 DC 2

CO

RE

AG

GR

AC

CE

SS

Server Cluster

CO

RE

AG

GR

AC

CE

SS

Server Cluster

Key Recommendations

vPC Domain id for facing vPC layers should be different No Bridge Assurance on interconnecting vPCs BPDU Filter on the edge devices to avoid BPDU

propagation No L3 peering between DCs (i.e. L3 over vPC)

vPC domain 10 vPC domain 20

vPC domain 21vPC domain 11

Rootguard

B

F

N

E

BPDUguard

BPDUfilter

Network port

Edge or portfast port type

- Normal port type

R

EE

- -

--

- -

-

-

-

-

F

F

F

F-

-

- -

-

--

BB

N N NN

N

NN

N

RR

-

RRRR

RR

Data Center InterconnectMulti-layer vPC for Agg and DCI

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 34

Nexus 7010 Nexus 7010

DC-1 DC-2

Nexus 7010 Nexus 7010

vPCvPC

Data Center InterconnectEncrypted Interconnect

CTS Manual Mode (802.1AE 10GE line-rate

encryption)No ACS is required

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 35

Agenda Feature Overview & Terminology

vPC Design Guidance & Best PracticesBuilding a vPC domain

Attaching to a vPC domain

Layer 3 and vPC

Spanning Tree Recommendations

Data Center Interconnect (& Encryption)

HSRP with vPC

vPC and Services

vPC latest enhancements

ISSU

Convergence and Scalability

vPC Hands-on Lab Information

Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 36

Support for all FHRP protocols in Active/Active mode with vPC

No additional configuration required

Standby device communicates with vPC manager produces to determine if vPC peer is “Active” HSRP/VRRP peer

General HSRP best practices still applies.

When running active/active aggressive timers can be relaxed (i.e. 2-router vPC case)

L3

L2

HSRP/VRRP “Standby”: Active for

shared L3 MAC

HSRP/VRRP “Active”: Active for

shared L3 MAC

HSRP with vPCFHRP Active/Active

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 37

L2/L3 Aggregation

ACTIVE HSRP STANDBY HSRP

GW GW

GW

VLAN 100 VLAN 200

VLAN 100, 200

L3 CORE

Cautions:

Not recommended using HSRP link tracking in a vPC configuration

Reason: vPC will not forward a packet back on a vPC once it has crossed the peer-link, except in the case of a remote member port failure

HSRP with vPCDo NOT use Object Tracking

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 38

Use an OSPF point-to-point adjacency (or equivalent L3 protocol) between the vPC peers to establish a L3 backup path to the Core through in case of uplinks failure

A single point-to-point VLAN/SVI will suffice to establish a L3 neighborship.

L3L2

OSPFOSPF

PrimaryvPC

SecondaryvPC

OSPFOSPF

OSPF

VLAN 99

HSRP with vPCL3 Backup Routing

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 39

Scenario:

Provide L2/L3 interconnect between L2 Pods, or between L2 attached Datacenters (i.e. sharing the same HSRP group).

A vPC domain without an active HSRP instance in a group would not able to forward traffic.

Multi-layer vPC with single HSRP:

L3 on the N7K supports Active/Active on one pair, and still allows normal HSRP behavior on other pair (all in one HSRP group)

L3 traffic will run across Intra-pod link for non Active/Active L3 pair

Active Standby Listen Listen

HSRP with vPCDual L2/L3 Pod Interconnect

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 40

Agenda Feature Overview & Terminology

vPC Design Guidance & Best PracticesBuilding a vPC domain

Attaching to a vPC domain

Layer 3 and vPC

Spanning Tree Recommendations

Data Center Interconnect (& Encryption)

HSRP with vPC

vPC and Services

vPC latest enhancements

ISSU

Convergence and Scalability

vPC Hands-on Lab Information

Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 41

vPC and ServicesCatalyst 6500 Services Chassis w. Services VDC SandwichTwo Nexus 7000 Virtual Device Contexts used to “sandwich”

services between virtual switching layers

• Layer-2 switching in Services Chassis with transparent services

• Services Chassis provides Etherchannel capabilities for interaction with vPC

• vPC running in both VDC pairs to provide Etherchannel for both inside and outside interfaces to Services Chassis

Design considerations:• Access switches requiring services are connected to sub-

aggregation VDC

• Access switches not requiring services may be connected to aggregation VDC

• May be extended to support multiple virtualized service contexts by using multiple VRF instances in the sub-aggregation VDC

Design Cautions:• Be aware of the Layer 3 over vPC design caveat. If Peering at

Layer 3 is required across the two vPC layers an alternative solution should be explored (i.e. using STP rather than vPC to attach service chassis)

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 42

Agenda Feature Overview & Terminology

vPC Design Guidance & Best PracticesBuilding a vPC domain

Attaching to a vPC domain

Layer 3 and vPC

Spanning Tree Recommendations

Data Center Interconnect (& Encryption)

HSRP with vPC

vPC and Services

vPC latest enhancements

ISSU

Convergence and Scalability

vPC Hands-on Lab Information

Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 43

Several enhancements to vPC:

vPC Object Tracking

vPC Peer-Gateway

vPC Delay Restore

Multi-layer vPC with single HSRP group

vPC unicast ARP handling

vPC Exclude Interface-VLAN

vPC single attached device Listing

vPC Convergence and Scalability

For more details:

4.2 Release Noteshttp://www.cisco.com/en/US/docs/switches/datacenter/sw/4_2/nx-os/release/notes/42_nx-os_release_note.html#wp218085

vPC Latest EnhancementsSummary

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 44

vPC PL

vPC PKL

L3

L2

Scenario:

Interoperability with non RFC compliant features of some NAS devices (i.e. NETAPP Fast-Path or EMC IP-Reflect)

NAS device may reply to traffic using the MAC address of the sender device rather than the HSRP gateway.

Packet reaching vPC for the non local Router MAC address are sent across the peer-link and can be dropped if the final destination is behind another vPC.

vPC Peer-Gateway Solution:

Allows a vPC switch to act as the active gateway for packets addressed to the peer router MAC (CLI command added in the vPC global config)

Local Routing for peer router –mac Traffic

N7k(config-vpc-domain)# peer-gateway

vPC Latest EnhancementsvPC Peer-Gateway for NAS interoperability

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 45

Agenda Feature Overview & Terminology

vPC Design Guidance & Best PracticesBuilding a vPC domain

Attaching to a vPC domain

Layer 3 and vPC

Spanning Tree Recommendations

Data Center Interconnect (& Encryption)

HSRP with vPC

vPC and Services

vPC latest enhancements

ISSU

Convergence and Scalability

vPC Hands-on Lab Information

Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 46

ISSU is still the recommended system upgrade in a multi-device vPC environment

vPC system can be independently upgraded with no disruption to traffic.

Upgrade is serialized and must be run one at the time (i.e. config lock will prevent synchronous upgrades)

Configuration is locked on “other” vPC peer during ISSU.

Begin End Caveats

4.1(x) 4.2(x) None

4.2(x) 4.1(x) None

4.1(3) 4.1(3)4.2(1)

4.1(3)

4.2(1)

4.2(1)

In-Service Software Upgrade (ISSU)vPC System Upgrade/Downgrade

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 47

Agenda Feature Overview & Terminology

vPC Design Guidance & Best PracticesBuilding a vPC domain

Attaching to a vPC domain

Layer 3 and vPC

Spanning Tree Recommendations

Data Center Interconnect (& Encryption)

HSRP with vPC

vPC latest enhancements

ISSU

Convergence and Scalability

vPC Hands-on Lab Information

Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 48

vPC Peer Link LACP Channel (2x10 GigE)

vPC Peer-Keepalive (GigE)

L2/L3 Aggregation

Nexus 7000 vPC

L3 CoreNexus 7000

L2 AccessNexus 5000

E2/14 E2/14N7K-1 N7K-2

Po10

20 flows @1000 pps

20 flows @1000 pps

20 flows @1000 pps

Po20Po160

16-way port-channel 4-way port-channel

OSPFOSPF

OSPFOSPF

4.2(1) vPC EnhancementsConvergence Topology

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 49

vPC on Nexus 7000Convergence Numbers

Failover case Failure Topology Convergence Time

Failure Restoration

Failure of secondary vPC peer*

4.1(4)

North-Bound: ~700 msSouth-Bound: ~2.5 sec

4.1(4)

North-Bound: ~3 secSouth-Bound: ~3.4 sec

4.2(1)

North-Bound: ~50 ms.South-Bound: ~100 ms

4.2(1)

North-Bound: 100 – 900 msSouth-Bound: 1.2 -2 s

Failure of a primary vPC peer*

4.1(4)

North-Bound: ~150 msSouth-Bound: ~3 sec

4.1(4)

North-Bound:~4.5 secsSouth-Bound: ~5 secs

4.2(1)

North-Bound: ~50 msSouth-Bound: ~100 ms

4.2(1)

North-Bound: ~400 ms-1.5 sSouth-Bound: ~1.5 s

Failover of the vPC Peer Link

4.1(4)

North-Bound: ~1.3 sSouth-Bound: ~1.8 s

4.1(4)

North-Bound: ~900 ms South-Bound: up to 10+ s (CSCsz88998)

4.2(1)

North-Bound: 100-300 msSouth-Bound: 50-500 ms

4.2(1)

North-Bound: 150 - 900 msSouth-Bound: ~ 900 ms–1.5 s

NOTE: Convergence numbers may vary depending on the specific configuration (i.e. scaled number of VLANs/SVIs or HSRP groups) and traffic patterns (i.e. L2 vs L3 flows).

P S

P S

P S

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 50

vPC on Nexus 7000Scalability Number Improvements

Release Supported Scalability

4.1(5) 192 vPC’s (2-port) with the following,200 VLANs200 HSRP Groups40K MACs & 40K ARPs10K (S,G) w. 66 OIFs (L3 sources)3K (S,G) w. 34 OIFs (L2 sources)

LatestAnkara 4.2(1)

256 vPC’s (4-port) with the following,260 VLANs 200 SVI/HSRP Groups40k MACs & 40K ARPs

10K (S,G) w. 66 OIFs (L3 sources) 3K (S,G) w. 64 OIFs (L2 sources)

NOTE: Supported numbers of VLANs/vPCs are NOT related to an hardware or software limit but reflect what has been currently validated by our QA. The N7k BU is planning to continuously increase these numbers as

soon as new data-points become available.

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 51

Agenda Feature Overview & Terminology

vPC Design Guidance & Best PracticesBuilding a vPC domain

Attaching to a vPC domain

Layer 3 and vPC

Spanning Tree Recommendations

Data Center Interconnect (& Encryption)

HSRP with vPC

vPC and Services

vPC latest enhancements

ISSU

Convergence and Scalability

vPC Hands-on Lab Information

Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 52

vPC Hands-on Lab InformationOn Demand vPC Lab Overview

Pod 1 Pod 2

Pod 3 Pod 4

Pod 5 Pod 6

N7K-1POD 1-2 VPC

N7K-2POD 1-2 VPC

N7K-4POD 3-4 VPC

N7K-3POD 3-4 VPC

N7K-8POD 5-6 VPC

N7K-7POD 5-6 VPC

Pod 1 Pod 2

N7K-Aggr N7K-Aggr

Instructor-led hands-on lab introducing the vPC (virtual Port-channel) feature for the Nexus 7000.

Participants exposed to the configuration of vPC with NX-OS.

Lab needs to be manually booked through Nexus 7000 TMEs.

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 53

vPC Hands-on Lab Information vPC Lab Logistics and Timing

The vPC Laboratory consists of 6 independent PODs.

A group of 2 students is assigned to each Pod.

Each student will configure a vPC peer device.

PODs are logically independent. Two adjacent PODs are physically bound to the same Nexus. Virtual Device Contexts (VDCs) are used to define logically independent devices on the same Nexus 7010 box.

The vPC Lab session is expected to be completed in around two hours.

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 54

Agenda Feature Overview & Terminology

vPC Design Guidance & Best PracticesBuilding a vPC domain

Attaching to a vPC domain

Layer 3 and vPC

Spanning Tree Recommendations

Data Center Interconnect (& Encryption)

HSRP with vPC

vPC and Services

vPC latest enhancements

ISSU

Convergence and Scalability

vPC Hands-on Lab Information

Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 55

L2/L3 Aggregation

Nexus 7000 vPC

L3 Core

L2 Access

6500 VSS

E1/26

E1/25

Te2/2/1

E1/25

E1/26

Te1/2/1 Te1/2/2 Te2/2/2

Po10

vPC Peer Link LACP Channel (2x10 GigE)

vPC Peer-Keepalive (GigE)

E2/14 E2/14

Po100

VSS VSL Channel (2x10 GigE)

N7K-1 N7K-2

6K-26K-1

Po100

Physical Logical

Reference MaterialvPC/VSS Interop Test Details

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 56

The following scenarios were tested:• VSS and vPC member failover and convergence

• Dual active scenarios and behavior

• Best practice guidelines for STP, L3 (NSF), Multicast

Catalyst 6500/Nexus 7000 interoperability:• Multiple ports per chassis act as one larger ether-channel

Reference MaterialvPC/VSS Interop Test Details

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 57

Enterprise Solutions Engineering:http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/DC_3_0/DC-3_0_IPInfra.html

Implementing Nexus 7000 in the Data Center Aggregation Layer with Services:

https://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/nx_7000_dc.html

Configuration Guide for Object Tracking Feature: http://www.cisco.com/en/US/partner/docs/switches/datacenter/sw/4_2/nx-os/interfaces/configuration/guide/if_vPC.html#wp1530133

vPC white Paper:http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9402/white_paper_c11-516396.html

Reference MaterialOther Solution Tests and Recent vPC Documentation