Almost at the finish line!

I labbed for 4 hours this morning completing a lab from last night so latter MPLS VPN, QoS & Multicast. I hit a wall per se and it’s simply called tiredness. I have been on the routers 10 hours\day since Thursday and it takes its toll. Therefore I spent the rest of the afternoon and evening going through blog notes and crib sheets.

There are some great entries out there with both INE & IPExpert outputting top class SP tech-notes. Add these to Ivan Pepelnjak’s blog & MPLS VPN crib sheets, packetlife’s cheat-sheets [always hated that term!], Cisco Learning Network SP Notes and you have a great set of notes maybe 50-75 pages worth that cover the backbone [excuse the pun] of the SP Lab Exam.

Tomorrow is final prep day, some small labbing – going over concepts, reinforce my attack plan, pack the bag for Brussels – its an early flight – 0600 getting in at 0900 local time, a day in the hotel to chill out and get some sleep then report in for the Lab on Thursday morning.

My plan of attack? -> Fly out of the blocks and build up some real speed – I have been speed drilling for the last 2 weeks and its amazing how fast concepts such as MPLS & Multicast in particular can be typed. Obviously due care is required but I am determined to break the back of the MPLS VPN section by lunch time if at all possible. I cannot allow any IOS issues to stall this, there will be a 15 minute max limit placed on each section and I move on regardless. If the issues is core to the lab I move on anyway, perhaps to SP Management or QoS as I have proved that troubleshooting too long and you cannot see the wood for the trees – get away, do something else whilst rebooting the troublesome router should aid that process.

This is my final blog entry prior to the lab so I will be back on Friday with my results – positive or otherwise. Thanks for all the encouragement.

Steve.

Labbing – Labbing – 6 days to go

Less blogging more labbing – completed INE Lab 4 and IPExpert Lab3 from the Lab Mentoring Kit – I use both vendors rack rentals as well and nothing but good things on that – So where am I at?

• I am getting faster – finished L2, L3 and MPLS in 2hs 10 minutes today.
• I am more confident in the MPLS VPN world
• I am still getting caught in the MPLS TE World – I honestly thought I had this subject nailed but I keep missing small things that cost time such as MPLS router-id on the transit routers, etc
• QoS is now my weakest section not MPLS VPN’s
• Oh and I still dislike Systems Management Questions – thats why we buy Solarwinds, etc.

I have no work until my Lab in Brussels on Thursday so one full lab a day until Tuesday, fly out Wednesday, Lab on Thursday, Result on Friday - Also I will bring my camera to Brussels and put up a journey blog entry for the trip there so prospective Lab Candidates can see the Cisco office, etc during my lab day.

Keep studying – Steve.

PPPoE Examples by the experts.

I am not a frequent poster but these articles really are excellent in demonstrating PPPoE configurations.

Brian Dennis => http://blog.internetworkexpert.com/2008/01/20/example-configurations-for-ppp-over-ethernet-pppoe/

Ivan PepelInjak => http://wiki.nil.com/PPPoE_testbed

Cisco => http://www.cisco.com/en/US/docs/ios/12_1t/12_1t2/feature/guide/dtpppofe.html

 

 

MPLS VPN Tasklist – 14 days to go.

I have been  trying to master and speed up my approach to the MPLS VPN Section which I did not do my 1st time around – here is an abridged tasklist based on the relevant IOS Configuration Guides providing me with a roadmap to navigate through this crucial section. HTH.

Stephen Bowes – How to Configure MPLS Layer 3 VPNs:

Step 1: Configuring the Core Network (required)

            1a. Assessing the Needs of MPLS VPN Customers (not required in CCIE Lab as details provided)

• Identify the size of the network.
• Identify the routing protocols.
• Determine if you need MPLS High Availability support.
• Determine if you need BGP load sharing and redundant paths.

            1b. Configuring Routing Protocols in the Core (required – For SP Lab this will be OSPF or ISIS for scalability reasons)

            Configuring OSPF in the core:

• enable
• configure terminal
• router ospf process-id
• Router(config-router)# network ip-address wildcard-mask area area-id

            Configuring ISIS in the core:

• enable
• configure terminal
• router isis [area-tag]
• net network-entity-title
• end

            Enabling IS-IS as an IP Routing Protocol on the Interface (required)

• enable
• configure terminal
• interface type number
• ip address ip-address mask [secondary]
• ip router isis [area-tag]
• end

                        Monitoring IS-IS (optional)

                        Not listed here – various show commands

                        Shutting Down IS-IS to Make Changes to Your IS-IS Network (optional)

                        Not listed here

            1c. Configuring MPLS in the Core (required)

• enable
• configure terminal
• mpls ip
• mpls label protocol {ldp | tdp | both}
• interface type number
• mpls ip
• exit
• exit
• show mpls interfaces [interface] [detail]
• show mpls ldp discovery [all | vrf vpn-name] [detail]
• show mpls ldp neighbor [[vrf vpn-name] [address | interface] [detail] | [all]]

            LDP used as the example here – obviously could be TDP, etc.

            1d. Determining if CEF Is Enabled in the Core (required)

• sh run
• sh ip cef

            1e. Configuring Multiprotocol BGP on the PE Routers and Route Reflectors (required)

• enable
• configure terminal
• router bgp as-number
• no bgp default ipv4-unicast
• neighbor {ip-address | peer-group-name} remote-as as-number
• neighbor {ip-address | peer-group-name} activate
• address-family vpnv4 [unicast]
• neighbor {ip-address | peer-group-name} send-community extended
• neighbor {ip-address | peer-group-name} activate
• end

                                                                                                                                   

Step 2: Connecting the MPLS VPN Customers (required)

            2a. Defining VRFs on the PE Routers to Enable Customer Connectivity (required)

• enable
• configure terminal
• ip vrf vrf-name
• rd route-distinguisher
• route-target {import | export | both} route-target-ext-community
• import map route-map
• exit

            2b. Configuring VRF Interfaces on PE Routers for Each VPN Customer (required)

• enable
• configure terminal
• interface type number
• ip vrf forwarding vrf-name
• end

            2c. Configuring Routing Protocols Between the PE and CE Routers (required)

We can run BGP, RipV2, OSPF, Static Routes or EIGRP as the PE-CE Routing Protocol – here are the configuration tasks for all.

            Configuring BGP as the Routing Protocol Between the PE and CE Routers

• enable
• configure terminal
• router bgp as-number
• address-family ipv4 [multicast | unicast | vrf vrf-name]
• neighbor {ip-address | peer-group-name} remote-as as-number
• neighbor {ip-address | peer-group-name} activate
• exit-address-family
• end

            Configuring RIPv2 as the Routing Protocol Between the PE and CE Routers

• enable
• configure terminal
• router rip
• version {1 | 2}
• address-family ipv4 [multicast | unicast | vrf vrf-name]
• network ip-address
• redistribute protocol [process-id] {level-1 | level-1-2 | level-2} [as-number] [metric metric-value] [metric-type type-value] [match {internal | external 1 | external 2}] [tag tag-value] [route-map map-tag] [subnets]
• exit-address-family
• end

            Configuring Static Routes Between the PE and CE Routers

• enable
• configure terminal
• ip route vrf vrf-name
• address-family ipv4 [multicast | unicast | vrf vrf-name]
• redistribute protocol [process-id] {level-1 | level-1-2 | level-2} [as-number] [metric metric-value] [metric-type type-value] [match {internal | external 1 | external 2}] [tag tag-value] [route-map map-tag] [subnets]
• redistribute protocol [process-id] {level-1 | level-1-2 | level-2} [as-number] [metric metric-value] [metric-type type-value] [match {internal | external 1 | external 2}] [tag tag-value] [route-map map-tag] [subnets]
• exit-address-family
• end

            Configuring OSPF as the Routing Protocol Between the PE and CE Routers

• enable
• configure terminal
• router ospf process-id [vrf vpn-name]
• network ip-address wildcard-mask area area-id
• address-family ipv4 [multicast | unicast | vrf vrf-name]
• redistribute protocol [process-id] {level-1 | level-1-2 | level-2} [as-number] [metric metric-value] [metric-type type-value] [match {internal | external 1 | external 2}] [tag tag-value] [route-map map-tag] [subnets]
• exit-address-family
• end

            Configuring EIGRP as the Routing Protocol Between the PE and CE Routers

• enable
• configure terminal
• router bgp as-number
• no synchronization
• neighbor ip-address remote-as as-number
• neighbor ip-address update-source loopback interface-number
• address-family vpnv4
• neighbor ip-address activate
• neighbor ip-address send-community extended
• exit-address-family
• address-family ipv4 vrf vrf-name
• redistribute eigrp as-number [metric metric-value][route-map map-name]
• no synchronization
• exit-address-family
• end

            Configuring EIGRP Redistribution in the MPLS VPN

• enable
• configure terminal
• router eigrp as-number
• address-family ipv4 [multicast | unicast | vrf vrf-name]
• network ip-address wildcard-mask
• redistribute bgp {as-number} [metric bandwidth delay reliability load mtu] [route-map map-name]
• autonomous-system as-number
• exit-address-family
• end

                                                                                                                                   

Step 3: Verifying Connectivity between MPLS VPN Sites (optional)

            3a. Verifying the VPN Configuration

• show ip vrf

            3b. Verifying IP Connectivity from CE Router to CE Router Across the MPLS Core

• enable
• ping [protocol] {host-name | system-address}
• trace [protocol] [destination]
• show ip route [ip-address [mask] [longer-prefixes]] | [protocol [process-id]] | [list access-list-number access list number
• disable

            3c. Verifying that the Local and Remote CE Routers are in the Routing Table

• enable
• show ip route vrf vrf-name [prefix]
• show ip cef vrf vrf-name [ip-prefix]
• exit

                                                                                                                                   

References:

Cisco IOS IP Routing: ISIS Configuration Guide, Release 12.4

Cisco IOS IP Routing: OSPF Configuration Guide, Release 12.4

Cisco IOS Multiprotocol Label Switching Configuration Guide, Release 12.4

                                                                                                                                   

New CCIE SP Lab Checklist v3 Edition – Nov 2009

I have re-compiled my v2 CCIE SP Checklist which I published here before my last attempt back in February. I have added in nuances, new tips and additional information I have come across in other forums which I hope can help not only you but also me!!

Here is the copy in blog format and I have also added a rapidshare link at the bottom of this blog entry for the PDF Version – let me know what you think.

                                                                                                                                   

Title:                 CCIE SP Lab Checklist

Author:             Stephen Bowes

Version:           3.0

Date:                November 2009

                                                                                                                                   

Abstract: 

This is a compilation of notes, gotcha’s, pointers, etc from my research in preparation for my upcoming CCIE SP Lab exam which I have acquired over many years. Please feel free to notify me of more improved ways to those listed below and or errata through my CCIE blog at cciesplab.wordpress.com or by email at cciesp@rocketmail.com.

Points Scoring and Timings:

I am conscious of the number of candidates who have failed due to running out of time. There are a number of reasons for this, here they are and proposed solutions. 

Reasons for Failure:	Solutions:
Misinterpreting the questions	Read the question more slowly, read it again, do not over-engineer the solution, answer what is asked, confirm any doubts with proctor, if proctor answer unacceptable, ask the same question a different way again.
Typing in the right configuration on the wrong interface or router	Tread carefully, cross-check and reference, validate before moving on.
Tasks taking too long to configure in the time window available	Practise speed drills, type faster, use aliases, notepad for verbose configurations, and use the Doc CD less if possible. Configure technologies router by router rather than interface by interface [explained later]
Lack of Task Verification	Failing to fully verify – ensure you use the three way approach [1] Ping, [2] Trace Route & [3] Routing Table

 To this end my timing plan is as follows -> Total Time = 8 hours = 480 Minutes. Lab Points Total = 100 Points, allowing 30 minutes for opening moves [see below] and 45 minutes for checking, validation and verification at the end, gives me 400 minutes for configuration

=> 4 Minutes/Point.

Pre-Lab Actions:

1 Month:

Adjust your body to performing 8 hour labs – Stamina will be key – you will be no use to anyone if you get tired after 5 hours of labbing. With 1 month to go ensure you are not doing 4 hour mini-labs rather the longer ones.

1 Week:

Adjust your body clock to the lab time. In my case I work 11am-7pm GMT whereas the Lab Exam in Brussels starts at 0745. This is 0645 GMT so with a week to go I will be up, showered, and had breakfast and sitting at my desk at 0730 to start an 8 hour lab with lunch at 12 for 30 minutes. I need to be fully alert at 0745 on Lab Day.

Lab Exam Day:

• Get as much sleep as is feasible the night before, up, showered, breakfast complete and be at Cisco by 0730. I booked into the nearest hotel I could find 250m away so no reliance on transport, etc.
• Bring a number of layers of clothes in case the room is cool, bring ear plugs so that the 11 guys/girls typing next to you and also so that the CCIE Voice candidates testing faxes will not interfere with your concentration levels.
• Documentation Location is http://www.cisco.com/web/psa/products/index.html

15 Minute Immediate Action: Anyone who has served in the military knows what an Immediate Action is – when something goes wrong a backup plan – in this case I’m going to move on if I cannot get any 3 pointer completed within 15 minutes ensuring I finish the lab!

Lab Action Plan: [Note: All times below are estimates and dependent on points values as per timing plan noted above]

Opening Moves: [30 Minutes: 0800->0830]

• After the proctor instructions, take a minute, calm yourself, open the booklet, read the exam end to end, visualise the Bridging/Switching, IGP, EGP, MPLS, etc.
• Draw a personalised diagram of the topology – Note: This is a talking point, some do, some don’t, and I think it’s advantageous especially from an IP/Interface perspective.
• Ignore the rush of the other candidates typing or the urge to get started.
• Create a point checklist on the rough paper provided. Here is my example.

Example Point Checklist:

Task:	Section:	Points:	Time: [Mins]	Completed:	Total Points:	Comments:
Switching	1.1	3	15	Yes	3	Watch security requirement section 7.2
Switching	1.2	2	10	Yes	5	All ok
Switching	1.3	2	10	No, moved on	5	Look up DocCD to confirm solution.

Troubleshooting: [15 Minutes: 0830->0845]

A number of faults may have been entered into the pre-configured devices. Check your SecureCRT software – can you see each of the devices? Reload each device, look for any hardware errors on boot-up, now is the time to spot this, not 11am.

As any issues could have been introduced check everything, IP Addresses matching Interfaces, subnet masks, FR DLCI’s, FR Inverse-Arp, pre-defined VLAN’s, VTP Modes on 3550’s, watch any pre-defined configurations configured on correct interfaces, ATM configurations, NSAP, IP, IP CEF, etc. 

I am not an Alias guy but now would be the time to do this, type these into notepad and cut & paste onto the routers ‘show run | b Se’ – Remember for large or repetitive configurations such as BGP, use notepad and then copy and paste but be aware of changing values such as IP’s, subnets, etc as you copy and paste.

 Bridging & Switching:

Frame-Relay: [15 Minutes: 0845->0900]

• Use your diagram to draw out the FR Topology
• A lot of this may be pre-configured so verification doubly important
• Use [1] shut [2] enc frame-relay [3] no frame inverse-arp [4] no shut.
• Decide to use either frame-relay map or use sub-interfaces
• Ping from spoke to spoke if possible to validate.
• Extra mapping required if required to ping your own interface
• If PPP over FR, then always create VT first, user/password
• Save, reload, and then verify all working.
• FRTS – Know your CIR=Bc x 1000\Tc; Be=(AR-CIR) x Tc/1000.
• DocCD Location => Main URL = http://www.cisco.com/web/psa/products/index.html

– Cisco IOS SW Release 12.4 Family – 12.4 Mainline – Configuration Guides – Cisco IOS Wide-Area Networking Configuration Guide, Release 12.4.

• Verification Tools – ping, show frame-relay map, show int virtual-template, show int virtual-access, show traffic-shape, show interfaces serial, show frame-relay lmi, show frame-relay pvc, clear frame-relay inarp, clear interface, debug serial interface, debug frame-relay lmi, debug frame-delay events, debug frame-relay packets

=> Golden Moment: Frame-Relay is the spinal cord of the inter-network, it must be 100% <=

Switching: [15 minutes: 0900->0915]

• Check VLAN’s as per instruction
• Check VTP Modes
• Check Trunking & Access Ports
• A lot of pre-configuration completed so use the verification commands below.
• Ping vlan by vlan. Select only one device and ping all other on a specific vlan.
• If naming something, type it exactly as specified – Ref: Narbik
• Specify both Duplex and Speed as Auto-Sense can be troublesome – Ref: IEMentor & Gorito
• DocCD Location => Main URL, Cisco IOS SW Release 12.4 Family, 12.4 Mainline, Configuration Guides, Cisco IOS LAN Switching Configuration Guide, Release 12.4
• Verification Tools => show interfaces, show interfaces trunk, show vlan brief, show vtp status, clear interface

 Cell-Mode MPLS: [15 Minutes: 0915->0930]

• Configure any ATM interfaces required – PVC/SVC, NSAP Addressing,
• Watch for tag-switching or label-switching.
• Security authentication may be required
• Use ping to verify
• DocCD Location => Main URL, Cisco IOS SW Release 12.4 Family, 12.4 Mainline, Configuration Guides, Cisco IOS Asynchronous Transfer Mode Configuration Guide, Release 12.4
• Verification Tools => show interfaces, show atm pvc, show atm svc, show atm map, show atm traffic,

 PPP/Ethernet: [15 Minutes: 0930->0945]

• Configure PPP/PPPoE as required, PPPoE enable, pppoe-client, interface dialer, etc.
• Know security configurations, ping and validate.
• Be aware of IOS nuances with these types of features.
• DocCD Location => Main URL, Cisco IOS SW Release 12.4 Family, 12.4 Mainline, Configuration Guides, Cisco IOS VPDN Configuration Guide, Release 12.4 & Cisco IOS Broadband Access Aggregation and DSL Configuration Guide, Release 12.4
• Verification Tools => show pppoe session

 => Golden Moment – Bridging & Switching Complete – Total Time 1 Hour 45Mins <=

IGP: [Note that probably only one of these will be the core IGP] 

OSPF: [30 Minutes: 0945->1015]

• While reading the task, use your master diagram to configure OSPF router by router not area by area. Look for the following OSPF characteristics.
• Authentication, stub or nssa, virtual link
• Refer again to your master diagram, colour in the OSPF areas.
• Make a note on redistribution, summary, area-range, DR/BDR, OPSF network type.
• Get Area 0 working 100% first.
• Ensure Area 0 Contiguous, test, create GRE/Virtual-links, and test again.
• Configure other areas.
• Leave OSPF Security until last.
• From a time perspective, router by router saves you revisiting router and typing in additional commands after the fact.
• First Interface and then router ospf

Preferred sequence for configuring interface

1) OPSF network type based,

2) priority,

3) Authentication,

Preferred sequence for configuring OSPF process

1) router-id

2) area authentication,

3) neighbor,

4) Network (copy paste from interface address)

• Validate everything is working (show ip os ne, show ip os vir, show ip os interface, show ip route)
• Do redistribute summary, area range, filtering [Be Careful!]
• Validate and verify prior to moving on.
• Save Configurations,
• Reload routers and final verification.        

Note: Some candidates do not reload, some do – I will.

• DocCD Location => Main URL, Cisco IOS SW Release 12.4 Family, 12.4 Mainline, Configuration Guides, Cisco IOS IP Routing Protocols Configuration Guide, Release 12.4
• Verification Tools => show ip ospf, show ip ospf interfaces, show ip ospf neighbor, show ip ospf database, show ip ospf virtual-links, debug ip ospf events, debug ip ospf hello, debug ip ospf packet

IS-IS: [30 Minutes: 1015->1030] – Same as OSPF – Allowing additional 15 minutes in case both are present.

• This has been noted by previous candidates and having quite a bit to do on the SP Exam! Refer again to your master diagram, colour in the ISIS areas.
• Configure ISIS on relevant routers
• Note what ISIS Levels are required – 1 or 2,

• Assign appropriate NET addresses
• Remember unlike other IGP’s, ISIS configured at Interface level and is essentially a L2 protocol.
• Verify adjacencies
• Due to ISIS only knowing two forms of media – LAN or point-to-point -> use the frame-relay map clns command to create maps for protocol to run.
• Configure any ISIS filtering/redistribution
• Configure Authentication if required.
• Configure any additional ISIS nuances/parameters such as metrics/timers, etc we encounter.
• DocCD Location => Main URL, Cisco IOS SW Release 12.4 Family, 12.4 Mainline, Configuration Guides, Cisco IOS IP Routing Protocols Configuration Guide, Release 12.4
• Verification Tools => show isis database, show isis topology, show clns protocol, show clns interface, show clns neighbors.

=> Golden Moment – IGP Complete – IGP Time 1 hour – Total Time 3 Hours <=

BGP: [60 Minutes: 1030-1130 – dependent on points] 

• While reading task, draw BGP topology on master diagram, this is important.
• Determine Route Reflector or confederation or both to do full-mesh iBGP.
• See if neighbor peer-group is required,
• Configure router by router not BGP session-by-session
• Configure one AS then another – be AS focussed.
• Ascertain required address families & configure – ipv4, vpnv4, ipv4 vrf, etc
• Ensure reachability, one AS at a time.
• Spend enough time to be absolutely correct on route-filtering (ACL, prefix-list, as-path filer), route-aggregate(w/ as-set, summary-only, supress-map, attribute-map, advertise-map), route-manipulation( w/as-prepending, med, local-pref, weight, next-hop, advertise-map/non/existing-map, origin, community, etc ) route-dampening, etc.
• Resolve any next-hop-self issues which are easier to troubleshoot working one AS at a time.
• Validate config. Use “clear ip bgp * soft “not”, clear ip bgp *.
• Leave BGP Authentication until last.
• Save, reload and test.
• DocCD Location => Main URL, Cisco IOS SW Release 12.4 Family, 12.4 Mainline, Configuration Guides, Cisco IOS IP Routing Protocols Configuration Guide, Release 12.4
• Verification Tools => show ip bgp, show ip bgp summary, show ip route bgp, show ip bgp neighbors, show ip bgp neighbors neighbor-ip-address, debug ip bgp

=> Golden Moment – EGP Complete – Ensure full Reachability Maintained, Save Configs <=

Reachability Test: [Before lunch if possible followed by reloading routers]

Test full reachability with TCL Script. Check you get an ICMP response from every router to every router. If ping has no response, write down IP address and troubleshoot.

The master diagram will help here. Method involves – show ip alias, Copy to Notepad, Search and Replace to “Massage the Data and toss in the PING Command), Wrap what’s left in a TCL or Macro, Copy and Paste into a Router.

  Run tclsh script

  “foreach addr {

  1.1.1.1 <http://1.1.1.1

  …

  } { ping $ addr}” Just copy past after tclsh – To quit, just type ” tclq”. Also to quote Scott Morris -> I’d leave “debug ip routing” turned on through the rest of the day. It can be a quick indicator to things getting messed up (like when you add ACL’s or play with NAT!)

MPLS: [30 Minutes: 1130->1200] 

• Tag Switching v Label Switching, when to use which ones – Watch for IOS Bugs here!
• Watch any integration with EGP
• MPLS might be the final piece of the jigsaw for full lab reachability.
• Cell Mode v Frame Mode
• MPLS Traffic Engineering – Levels, metric-style wide, ip explicit config, RSVP? etc.
• DocCD Location => Main URL, Cisco IOS SW Release 12.4 Family, 12.4 Mainline, Configuration Guides, Cisco IOS Multiprotocol Label Switching Configuration Guide, Release 12.4
• Verification Tools => show mpls forwarding-table, show mpls interfaces, show mpls ldp neighbor, show mpls ldp parameters, show mpls traffic-eng autoroute

Golden Moment – Lunch – Reachability, Save Configurations & Reload.

Afternoon Session:

SP Management: [15 Minutes: 1230->1245] 

• Know SNMP, setting up community strings, traps, RMON, pointing at various devices, etc
• Netflow, destination address, port no, version, etc
• NTP, master, server, source, etc.
• Know about various IP Services available in the IOS
• DocCD Location => Main URL, Cisco IOS SW Release 12.4 Family, 12.4 Mainline, Configuration Guides, Cisco IOS NetFlow Configuration Guide, Release 12.4 & Cisco IOS Network Management Configuration Guide, Release 12.4 & Cisco IOS Configuration Fundamentals Configuration Guide, Release 12.4
• Verification Tools => Multiple Commands.

SP Security: [30 Minutes: 1245->1315]

Be careful not to block or drop any IGP updates; Draw a flow on paper if required

• Consider all options for classification – std/ext/reflexive/dynamic ACL, IP Prefix List, IP inspect, tcp intercept, Unicast RFP, ip accounting output packet /access-violation/precedence.
• Be aware of various ways to configure MD5 for IGP, some of this may be completed via the IGP\EGP sections, ensure you have read ahead at the start of the lab.
• When configuring Switchport port-security mac-address, be careful to include virtual and physical mac if HSRP is running
• Know response planning to common security attacks such as DOS, Smurf, etc.
• DocCD Location => Main URL, Cisco IOS SW Release 12.4 Family, 12.4 Mainline, Configuration Guides, Cisco IOS Security Configuration Guide, Release 12.4
• Verification Tools => Multiple Commands.

MPLS VPN: [75 Minutes: 1315->1445]

So much here: VRF, VRF-Lite, MP-iBGP, MP-eBGP, Important to map out on your master diagram, the flow/direction of the VPN Traffic so that the correct configuration can be applied to the correct interface on the correct router in the correct direction!

• MP-BGP filtering, specifying route-targets, etc
• PE-CE Routing, RIP – Watch Split-Horizon is off on physical FR and ATM, authentication, version, auto-summary, etc; Other IGP/EGP considerations configure router-by-router, Advanced Options-CSC, Internet Access, Central Services, etc.
• Be aware of various backup routes for the VPN traffic in the event of line/router failure, redistribution of PE-CE to Core and vice versa.
• Be aware of VPN and Frame Relay specific limitations
• GRE/mGRE tunnels, when to use, how to configure.
• Be able to provide Internet Access from one portion of the inter-network to another.
• Be able to exchange EGP traffic across AS’s, watch next-hop, watch multi-hop, etc
• QinQ/PPoE – benefits = reduce no of VLANs, scalability, encap dot1q, pppoe enabled, etc.
• DocCD Location => Main URL, Cisco IOS SW Release 12.4 Family, 12.4 Mainline, Configuration Guides, Cisco IOS Multiprotocol Label Switching Configuration Guide, Release 12.4

Verification Tools => show ip vrf, show ip route, show ip route vrf vrf-name [prefix], show ip cef vrf vrf-name [ip-prefix], ping vrf, show ip bgp vpn all summary, show ip vrf detail, ping vrf <vrf> <ip address> source <source ip>, sh ip bgp vpn all summary, sh ip bgp vpn all, sh ip bgp vpn vrf <vrf> summary, sh ip bgp vpn vrf <vrf>, sh ip bgp vpn vrf <vrf> labels, sh mpls forwarding, sh mpls forwarding | inc <prefix>, sh mpls forwarding vrf <vrf> <prefix>, sh mpls forwarding label <label>.

SP Multicast: [30 Minutes: 1445->1515]

• Setup PIM Mode as required – Sparse/Sparse-Dense – Use address-family ipv4 multicast were required
• Identify PIM RP or Bootstrap requirements
• Don’t forget ip multicast-routing and/or ip multicast-routing vrf <VRF>
• Be aware of route filtering
• Join any IGMP Groups if required, check with pings,
• Check Unicast and multicast traffic work across different AS.
• Multicast VPN, default MDT, data MDT, MDT Group Addresses, MSDP, etc
• DocCD Location => Main URL, Cisco IOS SW Release 12.4 Family, 12.4 Mainline, Configuration Guides, Cisco IOS IP Multicast Configuration Guide, Release 12.4
• Verification Tools =>   show ip igmp groups, show ip pim rp mapping, show ip mroute, show ip interfaces.

SP QoS: [30 Minutes: 1515->1545] 

• Be careful not to block or drop any IGP updates
• Draw a flow on paper
• Interpretation of what is required & which QoS Method to use is Key!!
• Determine classification method (ACL, NBAR) and direction.
• Determine Shaping v Policing
• Consider all options for queuing (legacy custom/priority, bandwidth/priority, shape average/peak, FRTS/GTS) – Always Outbound.
• Consider all options for policing ( police, rate-limit, ip multicast rate-limit, aggregate police( 3550))
• If frame-relay, don’t forget adaptive-shaping.( becn, fecn, foresight)
• Consider all dropping mode (random detect, ecn, tail drop, marking, etc)
• DocCD Location => Main URL, Cisco IOS SW Release 12.4 Family, 12.4 Mainline, Configuration Guides, Cisco IOS Quality of Service Solutions Configuration Guide, Release 12.4
• Verification Tools => show ip rsvp, show class-map, show ip rsvp reservation, show mls qos, show policy-map, show queueing, show traffic-shape, etc.

Timings & Tips:

• According to this schedule this allows me 45 minutes for checking, saving, reloading, troubleshooting, going back to skipped sections, etc.
• Remember the pass mark is 80% not 100% – we can allow for 6 sections worth 3 points each not to work out and still pass!!!!
• Route Filtering – Know this cold, affects several areas, pass or fail the lab on this alone IMO!

• Skipping Difficult Sections – This is a dangerous but potentially rewarding path up the mountain but slippery and easy to fall down on – Risky Approach.
• Redistribution – Say no more, need to pass routes, this is it – potential failure point.
• Strategy has to be flexible depending on the progress through the day.
• Ensure the “gimme” questions are answered 100% – These are key to success.
• Ongoing Validation, via show commands and TCL Script, saving and reloading at least twice I believe is essential.
• Speed accessing resources on the DOCCD is essential – should be less than 90 seconds lookup per topic.

Authors Note: Please feel free to contact me if you can add value to this 3^rd Edition as I would like to think this can help other SP candidates with a lab structure going forward.

PDF Upload Location => http://rapidshare.com/files/304716095/CCIE_SP_Lab_Checklist_v3.pdf

                                                                                                                                   

Another SP Mock Lab Completed & Lessons Learned

I went at a full lab and kept accurate timings as I went.

Morning

• Initial Configurations – Full Lab Reading – My own diagram took 45 minutes.
• Testing of L2 and FR pre-configurations and validation took up 15 minutes.
• ISIS was fine a mixture of L2 and L1, OSPF was okay add in some advanced IGP features and complete in 20 minutes.
• BGP was going well, two AS’s to be built, iBGP, eBGP – 40 minutes
• But I got caught cold with a simple omission that I just could not see – asked to use the [not so new now] format AA:NN and advertise as such – configured up the ip community list, created the route map, matched on the community and used the set command to specify the value required but here’s where I got thrown – I played with both advertising the network using the network statement under the ipv4 address family and specifying the route map and then also with specifying the route map under the neighbor command with the route map filtering on an inbound direction. The latter was kinda cheating but strangely I saw the community value under the “sh ip bgp x.x.x.x” command but then not another time – I had forgotten the “send-community” command for the relevant neighbors and just could not see that! Some BGP advanced features were fine but I had used up over 1 hour.
• MPLS – I really like this section – LDP, Neighbors, Traffic Engineering [really well documented in DocCD], no issues – 35 minutes and that brought me up the halfway mark. It did raise a question in my head – if you are asked for a specific path in a traffic engineering tunnel should we also place a second dynamic entry in addition to the 1st explicit entry? i am leaning towards yes – Why? Because if there is an issue with one of your routers in the tunnel path then you can lose connectivity and thus points – whereas the dynamic option ensure connectivity remains and points lost are minimised – finally from reading forums and feedback over configuration will not go against you once its within the lab guidelines.
• Save configurations and reload.

Now I am a little behind on my plan which is L2, L3 IGP, EGP , MPLS, and some QoS\Security\features before lunch for my actual lab attempt as I know I’ll need time to gain as much as possible from the MPLS VPN Section.

Afternoon

• Quick check of morning work – ’sh isis nei’, ’sh ip ospf nei’, ’sh ip bgp summ’, pings, etc – 10 minutes.
• QoS – a whole myriad of items, CAR, NBAR, FRTS, etc – some items found in the 12.4 docs, some easier to find in the 12.2 doc’s – watch for anomalies here in the IOS e.g. FRTS with 7200’s – Time =36 minutes.
• Services\Management – based on the blueprint – I went for questions on RMON, Netflow, SNMP, etc – Some queries – If we get asked for Netflow do we go for V5 or V9? I have worked with V5 for years and only recently set about configuring V9 in work – one for the proctor maybe? Time = 45 mins.

MPLS VPN 

• VRF – I copied and pasted a string of VRF details in as initial configs as per the lab and this really took time for me to resolve. MP-iBGP – no issues – up and running across several routers – well documented again in the Doccd if you’re not familiar. 2nd site configured with some redistribution between OSPF & BGP – Note for redistributing OSPF into BGP I use the IEE acronym [as in the shorter version of IEEE - the engineering institute] as in matching internal ext 1 ext 2. Just helps for redistributing all OSPF routes. 1st and 2nd sites talking – redistributing and setting some advanced features – verification can be difficult having to know exactly what to look for and more importantly where – I used the INE SP Vol2 Lab Workbook in this regard as the newer solutions guide for the Dynamips version has verification and validation commands in addition to the solutions. The key to this section is two-fold – ascertaining what is being asked for – knowing where to configure what and avoiding troubleshooting if at all possible. My total time for 9 sections was 3 hours – Ouch!!!
• Security – A lot of this was done as part of the IGP\EGP and MPLS sections as it asked for authentications and filtering there – additional filtering and total time was 20 mins.
• Finally – Multicast – PIM SM across the AS’s  – RP\BSR and multicast VPN – total time of 25 minutes – no major issues – what’s nice about multicast is that if there are errors it tells you on screen and the messages are fairly accurate.

Conclusion:- Total Time = 9h 15 mins – Hmmm – have to speed up both to finish and allow additional time for verification otherwise happy enough with 20 days to go.

Keeping track & typo’s – 31 days to go.

What is it with typo’s – is it that we get used to using F7 with Microsoft Word\Outlook to auto-correct our natural spelling mistakes? There is of course no such luxury in IOS and I am constantly having to backtrack on small errors not just typos that have huge implications.

Examples include …..

• NET addressing with ISIS
• Advertising networks into BGP
• Enabling mpls traffic-eng tunnels on the wrong physical interfaces
• Specifying the wrong ip addresses for various interfaces
• Adding neighbors into the wrong address-families
• Wrong mask details for OSPF networks or when filtering through access-lists\prefix-lists, etc
• Missing configurations – e.g. not placing all route-target import\export entries across all transit routers in the inter-network
• Redstribution Filtering – trying to trap all posssibilities – I favour the tagging method.

You’ll have noticed that my blogging has been less technical and more observational – this is of course deliberate – more hours labbing! I have some rack rental time over the coming weekends with both INE & IPExpert – I have put together 4 “Master Labs” basically a culmination of both the aforementioned workbooks plus my recalled tasks I received in my 1st lab attempt [No, not available] – broken these into the 9 sections – put a big chart on my whiteboard and ticking the boxes off as I go. I am fairly confident I can sort out the foundations of the lab reasonably quickly i.e. L2\IGP\EGP & MPLS. The key for me will be obtaining as many of the MPLS VPN’s 27 points available. I am confident with Basic\Multi-VRF, MP-iBGP, MP-eBGP but advanced configurations are killing me time wise and even though I can configure up L2VPN it just seems so strange to type in ‘interworking ip’ commands on one end of the network and the neighbors appear by magic on the other – Good Stuff!

SP Verification Notes

One of the keystones to success in the SP Lab Exam is verification. Now this is obviously key to all tracks but is particularly complex in SP.

With the RS exam there are 4 ways to verify connectivity – ping, traceroute, routing table lookup and TCL Scripting [extended version of ping] where we put together a TCL Script using the foreach functionality – specify the various IP ranges used during the exam and execute the perl script on the routers. It has been covered in various other posts so here’s a link to NMC’s[or Cisco's] TCL document -> http://www.netmasterclass.net/site/articles/CISCO%20IOS%20TCL%20and%20RCMD%20testing%20and%20troubleshooting%20scripting.pdf

Note that for the for the 3550’s, they do not support TCL rather use macro’s so in config mode type…

macro name ICMP
do ping 18.1.1.1
do ping 18.2.2.2
do ping 18.3.3.3
do ping 18.4.4.4
@

However we use the address-family concept in SP and as a result this technique is only valid for L2 & L3 addresses not associated with address-families. As a result we need to place various appendages to our verification commands. Concentrating on the MPLS VPN world we use…

“show ip bgp vpn all summary” command to check the MP-BGP establishment status

“show ip vrf detail” command to verify the import and export route targets.

“ping vrf <vrf> <ip address> source <source ip>” to ping ip addresses not in standard routing table

“sh ip bgp vpn all summary”
Lists all of the MP-BGP and CE peers.

“sh ip bgp vpn all”
Lists all of the VPN prefixes advertised and received by the router.

“sh ip bgp vpn vrf <vrf> summary”
Similar to the first command, but for a specific VRF.

“sh ip bgp vpn vrf <vrf>”
Lists all of the VPN prefixes received in a specific VRF.

“sh ip bgp vpn vrf <vrf> labels”
Lists labels for the VPN prefixes in a VRF.

“sh mpls forwarding”
Shows all LFIB entries (VPN, non-VPN, TE, and so on).

“sh mpls forwarding | inc <prefix>”
Shows whether the prefix is present in the LFIB or not.

“sh mpls forwarding vrf <vrf> <prefix>”
Shows LFIB lookup based on a VPN prefix.

“sh mpls forwarding label <label>”
Shows LFIB lookup based on an incoming label.

This also extends to Multicast wher you need to amend the commands such as sh ip pim vrf <vrf> int, etc.

As you can see the verification in the MPLS VPN portion of the lab will require a good deal of time and hence perhaps why Cisco perform so much pre-config on this exam.

Details, details & details

The more mock labs I do the more I realise the SP Lab is all about details. Small details or nuances if you wish. Having completed INE Vol2 Lab 7 tonight I have begun collecting them and the trick is to be aware of their existance during the pressure moments of the lab.

• Typos – These happen to me all the way through my practise labs – NET addresses in ISIS, ATM PVC addressing, the wrong configuration on the wrong router or the right configuration on the wrong interface, wrong masks on loopbacks, etc
• Gotchas – ensure clns statements added to both ATM & FR connections if passing ISIS traffic across. Know the PPPoE configuration is dependent on IOS versions with different hardware,
• Diagram\IP Analysis – Match up your pre-configurations with the diagrams handed out. You literally have to walk each router interface by interface and match it up with the workbook you have been given. It has been known for the wrong workbooks to be handed to candidates.
• Pro-Active Management – Enable debug ip routing on relevant core devices – you need to know when routes are being deleted on one router as you make changes on another.
• Always complete IGP, EGP, MPLS & MPLS VPN prior to multicast to prevent issues such as RPF, etc -> Look what happens when you do not!

Rack1R4(config-if)#ip vrf forwarding 65001
% Interface Ethernet0/1 IP address 10.3.48.4 removed due to enabling VRF 65001
Rack1R4(config-if)#
%PIM-5-NBRCHG: neighbor 10.1.48.8 DOWN on interface Ethernet0/1 non DR
Rack1R4(config-if)#ip address 10.3.48.4 255.255.255.0

• Be aware that strange events will occur – do not let them phase you – here is an example – enabling VRF on an interface removes the ip address right?

Rack1R2(config-if)#ip vrf forwarding 65001   
% Interface FastEthernet1/0 IP address 10.3.27.2 removed due to enabling VRF 65001
Rack1R2(config-if)#
RT: del 10.3.27.0/24 via 0.0.0.0, connected metric [0/0]
RT: delete subnet route to 10.3.27.0/24
RT: delete network route to 10.0.0.0
Rack1R2(config-if)#
%DEC21140-5-REMOVE_HWADDR_FAIL: Interface FastEthernet1/0 failed to remove Addr:=0100.5e00.000d from HWAF

• Reloads – 1. Know when to reload - 2. why you should reload – 3. how reload can help you and 4. when not to reload.

My opinion? 1. At the start to ensure no gremlins, if strange events\issues strike you, just before lunch. 2. To remove issues, to ensure stable configurations & for piece of mind. 3. again to assist in resolving unknown issues and 4. at the end of the lab.

• Time Management – we stress this over and over again but we engineers do not know when to let an issue go – however be aware that the SP exam is more hierarchical then the RS exam and there is less scope for skipping sections due to the nature of the SP Core and the reasoning behind end-to-end connectivity.

These are just some notes as I have come across them – there are a few more and I will incorporate them in the Version 3 of the CCIE SP Lab Checklist which I published before my last attempt in February.
Happy labbing, Steve

SP Lab Digression – Passed VMware VCP.

Hi,

I have been dual studying this past month both CCIE Service Provider and VMware VCP. I completed the VMware 3.5 Course earlier in the year and have been part of the management team for a set of 16 ESX servers hosting 100 servers for the past year. Given the VCP4 vSphere exam takes over the VCP 3 exam in December and that I move on shortly from this role I decided to maximise both my time on this project and my training.

However I failed the VCP exam in July but managed to get through on my 2nd attempt yesterday. Cisco and VMware are tied together ownership wise Ref: http://www.vmware.com/company/news/releases/cisco.html and it made sense especially considering announcements such as these -> http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/net_brochure0900aecd806abf2f.htmlhttp://www.vmware.com/company/news/releases/cisco_vmworld08.html

http://www.vmware.com/company/news/releases/cisco_vmworld08.html

so I completed the pass and onwards to my 2nd SP attempt – 60 days to go!!