Retrospective and Goals

The year 2011 is almost over.  It's been a very interesting year for me.  This year was my first foray into blogging about my professional life.  I have found the connections made through this blog to be very rewarding.  Thank you for the encouragement, especially @etherealmind and @ecbanks who have allowed me to broaden my audience through some posts on @packetpushers.  Hopefully some of you have even found something I have written this year useful.

I also made my first appearance behind the podium as a speaker at a technology conference.  After years being on the receiving end, it was good to give some knowledge back.  Thank you to Josh Hall and company at Tech Feast for giving me the opportunity.  One lesson I learned is that it's not as hard to fill up a 90 minute presentation slot as I thought it would be.  The other thing I learned is not to volunteer to do four 90 minute presentations back to back.  My wife laughed quite hard at me when I came home and complained about my voice being gone and my feet being tired (she's a teacher).

So now it's time for the goals for next year in no particular order:

• Take and pass the ROUTE exam
• Take and pass the TSHOOT exam
• Take and pass the ARCH exam
• Earn the CCNP R&S and CCDP certifications
• Keep my office more organized
• Learn Italian (someday I will go back and this time I'll be able to talk to my relatives)
• Take the time to learn more and understand this OpenFlow stuff that the PacketPushers keep jabbering about
• Figure out how to grow and maintain grass in my back yard without making my dogs use hover boards

As you can tell, there might be a few blog posts that will double as me organizing my study notes.  My current job is mostly L2 so the routing stuff is a bit rusty.

CCDA 640-864 Official Certification Guide 4th Edition

A couple of months ago I had the privilege to read an advance e-book version of this new certification guide for the Cisco Certified Design Associate written by Anthony Bruno and Steve Jordan.  Cisco Press is bundling this e-book with a new test engine.  Unlike previous certification guides, this new test engine is not from Boson.  The test engine was very user friendly and the test material was well designed.


The book itself is very well laid out.  Although it is thick with theory about network design to meet the needs of those studying for the CCDA exam, it also has many real world examples.  These examples make the theory easier to understand and internalize.


As with most of the Cisco Press certification guides, there are plenty of questions within the chapters to help the reader self assess their progress.  My favorite feature is the memory table appendix.  These are a great way to prepare for the test.  My goal when I prepare for a test is to use the memory tables until I can accurately fill them in without the book and even produce them without the empty tables.  It's a great feeling to start an exam with a "cheat sheet" of sorts from the memory tables you can write down on your white board.

Network Egress Filtering

Today I volunteered to give a quick presentation on the importance of egress filtering to a wonderful group of education IT folks. Below is the slide deck that I used for the presentation for anyone that might find it interesting.

NMIS Configuration Part 2

In the previous post I presented a basic how to on configuring the basic notification system for NMIS.  Now I will show how to add network nodes to be monitored by NMIS.  First go to Configuration>System>Nodes.

Configuration>System>Nodes

This will bring up the Nodes listing showing currently configured nodes.  To add a node, click on add to the right of Action on the top right part of the screen.

Once you click Add, you will see the add node screen as shown below.  There are a lot of variables to configure for each node.

• Name: This is the name of the node and should have no spaces in it.  This must be unique.
• Name/IP Address:  This is the IP or the FQDN of the node.
• Group: This is the group in which the node will be displayed.
• Select Model:  This is the data model that NMIS will use to gather SNMP data from the device.  In most cases automatic works best.
• Active:  This defines whether the node is currently actively polled.  You can use this set to false to take a group of nodes out of active monitoring without deleting them.
• Ping: Should NMIS ping the node for availability?
• Collect:  Should NMIS collect SNMP data from the node?
• CBQoS:  Should NMIS collect data about Class Based QoS on inbound, outbound, both or none?
• Modem Calls:  Should NMIS collect data on modem calls (not used often)?
• Threshold:  Should NMIS run threshold calculations on this node?
• Rancid: Should the Rancid add in add this node to a rancid configuration file? (I'm not sure if this tool is still supported.)
• Web Server:  Does this device run a webserver?
• Net Type:  Is this device WAN or LAN?
• Role Type:  Is the device core, distribution or access?
• Depend:  What devices must be up for this device to be up?  This provides NMIS with a way of knowing when not to alert on a node if an upstream node is down.
• Services: Should NMIS check to make sure certain services are running on the node?
• Time Zone: What is the time zone offset for the device from GMT?
• SNMP Settubgs:  What version of SNMP should be used to poll the device and what is the configuration for that version?

When the configuration is complete, click Add and Update node to add the node to the NMIS configuration and tell NMIS to run an initial scan of the host. 

Once the nodes are added, it may take 10-15 minutes for data to start to show up depending on how often the NMIS cron jobs are set to run.  This concludes the basic configuration of NMIS.  There are a lot more nerd nobs in the software, but unfortunately good documentation is not available for the product, especially the newest 8.x versions.  Opmantek has said that documentation is a focus for their team so hopefully it will be coming soon.  In my next post I will discuss how to use NMIS once it is collecting data.

NMIS Configuration Part 1

Since I posted my first article about NMIS, I have had several requests to do a series on how to configure NMIS.  Although I took a break from blogging in September, here is the first in the requested series.


NMIS' configuration is stored in text files in /usr/local/nmis8/conf/ and can be edited by hand.  However, it is highly recommended to use the built in tools to edit the configuration.  The main configuration can be found under Configuration > System > NMIS Configuration.

Cisco Nexus 1000v Roundup

This post is merely an index for the 6 post series I did on the Cisco Nexus 1000v.  I hope that beyond being a good learning experience for myself that it will benefit others. 

Part 1

Part 2

Part 3

Part 4

Part 5

Part 6

Cisco Nexus 1000v - Adding Physical Ports (Part 6)

The previous posts have established a fully configured, but unused Nexus 1000v.  At this point it's like having a physical switch in the rack, powered up and configured, but with no network cables attached.  In VMWare, the "cables" are attached using the vSphere Client.


Attaching Physical Ports to the Nexus 1000v

1. Connect to vCenter using the vSphere Client
2. Go to Networking Inventory and select the Nexus distributed virtual switch (dVS).
3. Right click on the Nexus and choose add host.
4. Select the host and vmnic(s) to use and change their DVUplink port group to system-uplink (or what you named the system uplinks in your port group on the Nexus) for the system uplink ports and vm-uplink for the VM networking ports.
5. Click next and choose not to migrate the vmk0 or VMs. (I prefer to verify the Nexus 1000v's operation before migrating anything.)
6. Click Finish.
7. Repeat for all hosts in the cluster.

Migrating vmk0 Interfaces to Nexus

Once you have added a few test VMs to the Nexus and are certain that the Nexus 1000v is working properly, it's time to migrate the last physical NIC from the vSwitch to the Nexus 1000v and with it the vmk0 interface used for vMotion and VMWare host management.  Keep in mind that if you don't need this NIC for bandwidth reasons, it is not mandatory to move these services to the Nexus 1000v.

1. Connect to vCenter using the vSphere Client.
2. Go to Networking Inventory and select the Nexus dVS.
3. Right click on the Nexus and choose manage host.
4. Select the hosts and click next twice.
5. Click on the destination port group for the vmnic used by vmk0 and choose the Nexus port group.
6. Click next and then finish without migrating VMs.

You will need to repeat this for each host in the cluster.  Leave the host with the active VSM for last and make sure to migrate it's NICs to the Nexus before disconnecting the vSwitch from the vmnic.  

Cisco Nexus 1000v Software Installation (Part 5)

In this article I will run through installing the Virtual Ethernet Module (VEM) and creating the initial port groups on the Nexus 1000v.

Installing the Virtual Ethernet Module (VEM)

1. Open the vSphere client and connect to vCenter.
2. Right click on the host that you are going to install the VEM on and choose Maintenance Mode. (NOTE:  This will vMotion all guests from that host to other hosts if you have vMotion enabled, otherwise those guests will be shutdown.)
3. Copy the VEM bundle from the Nexus 1000v install zip file to the vMA or to the computer that you are running vCLI on.
4. Use the vCLI to install the VEM with the following command: vihostupdate -install -bundle <path to VEM Bundle> --server <host IP>

As you can see, installing the VEM software is fairly simple.

Creating the Port Groups on the Nexus 1000v

The Nexus 1000v uses port-profile configurations to define the configuration for each type of interface.  In this part of the install we need to setup profiles for the physical NICs that will uplink to the hardware switch infrastructure for both the system VLANs like VMK0 and the Nexus Control traffic as well as the VM uplinks for normal guest VLAN traffic.  On the Nexus 1000v, physical NICs are all of type Ethernet and virtual NICs are vEthernet.

1. Connect to the switch management IP address using SSH.
2. Type config t and enter to enter configuration mode.
3. Configure a port profile to use for your system uplink ports (VMK0, Nexus Control, Nexus Packet, Nexus Management).  Below is an example:
   

   
   

   port-profile type ethernet vm-uplink
     vmware port-group
     switchport mode trunk
   ! In my lab, 255 is MGMT, 256 is Nexus Packet and Control and 101 is for VMK0
     switchport trunk allowed vlan 101, 255-256
     switchport trunk native vlan 255
   ! This command has Nexus create port-channels automatically
     channel-group auto mode on
     no shutdown
   ! System VLANs come up before the VSM is fully initialized
     system vlan 101,255-256
     description SYSTEM-UPLINK
     state enabled
   

4. Configure a port profile to use for the VM Guest networks.
   

   
   

   port-profile type ethernet vm-uplink
     vmware port-group
     switchport mode trunk
     switchport trunk allowed vlan 2,102,104-105,259
     switchport trunk native vlan 102
   ! This command has Nexus create port-channels automatically
     channel-group auto mode on
     no shutdown
   ! System VLANs come up before the VSM is fully initialized.
     system vlan 102
     description VM-UPLINK
     state enabled 

5. Configure port profiles for the guest networking to match the old vSwitch port-groups.

   
   

   port-profile type vethernet example-vlan
     vmware port-group example-vlan
     switchport access vlan 
     switchport mode access
     no shutdown
     state enabled
    

6. Save the new configuration by doing copy running-config startup-config

Now that we have everything configured, the next post will be how to plug the network into the Nexus 1000v.

Cisco Nexus 1000v Software Installation (Part 4)

In the last post, I configured the VMWare stock vSwitch to allow us to start the Nexus 1000v install.  I forgot to mention one thing that I usually do (it's not required) to help with mapping physical ports.  The VMWare vSwitch supports CDP, but is usually set to listen only.  To change it so that it will send out CDP packets you need to do the following:

1. Using the VMWare vCLI either from a desktop of the vMA appliance run the following command: vicfg-vswitch -server <servername> -B both vSwitch0
2. Repeate for all of the ESX Hosts in the cluser.

The next step in the Nexus 1000v installation is to install the Virtual Supervisor Module (VSM)  VM appliance(s).

Deploying the Virtual Supervisor Module (VSM)

1. Download and decompress the Nexus 1000v software from Cisco.
2. Open up the vSphere Client and connect to vCenter.
3. Go to File and then Deploy OVF Template.
4. Select the OVF file under the Nexus 1000v VSM install folder.
5. Accept the details and the EULA.
6. Give the VSM a name and choose next.
7. Choose the ESX data store for the VSM and choose next.
8. Choose thick provisioned and click next. (NOTE: Thin provisioning is not supported by Cisco for the VSM.)
9. Map the virtual NICs to the appropriate port groups on the vSwitch.
10. Power up the VSM and connect to its console using vSphere client.
11. Once booted, go through the text based administrative setup to configure the following:
• Administrative user password
• Role (Standalone/primary/standby)  The first VSM will be primary unless it is to be the only VSM and then it would be standalone.
• Domain ID.  This number serves to tie VSMs to a VEM and to vCenter.  Each Nexus 1000v instance must have a unique domain ID.
12. Continue with the basic system configuration including:
• SNMP Read-Only community string
• Naming the switch
• Assigning the management IP address settings
• Enabling or disabling services.  NOTE: Make sure http and ssh are enabled as they are used later in the process.
13. Open up a web browser to http://<nexus1000vIPaddress/ and click on "Launch Installer Application" (Requires Java Web Start and doesn't seem to currently work with Chrome).

14. Give the wizard the credentials for the VSM that you created earlier.
15. Give the wizard the credentials for accessing vCenter and the vCenter IP Address.

16. Select the cluster where the VSM is installed.
17. Assign the proper VLANs to the VSM's virtual NICs by choosing advanced L2 and then chose the proper port groups.
18. Configure settings for the VSM.
19. Review the configuration.
20. Tell the wizard not to migrate any hosts or VMs and let it finish.  It will reboot the VSM multiple times before reporting that it is done.

Cisco Nexus 1000v Software Installation (Part 3)

In this article I will examine the process of configuring the default VMWare vSwitch with the VLANs needed to start the Nexus 1000v Install.

Assumptions: 

·         ESX is already installed

·         vCenter VM is already installed and configured

·         vSphere Client is installed on workstation

·         VLANs for Nexus Packet and Capture interfaces (can be the same VLAN) are created on the network.

·         VLAN for Nexus Management interface is created on the network.

·         The ESXi hosts have their ports configured as trunks with the Nexus Packet, Capture and Management VLANs allowed as well as the VLAN used for the ESX hosts’ IP addresses (VMK0).

The first steps to getting a Nexus 1000v installed are actually to get the basic VMWare vSwitch configured and operating.

Configuring ESXi vSwitch

1. Open vSphere Client and connect to vCenter.
2. Click on the host to configure.
3. Click on the configuration tab.
4. Click on networking. 

5. Click on properties.
6. Click on add.
7. Click on Virtual Machine and then Next 
8. Give the network a label and a VLAN ID (0 indicates the native vlan).  NOTE:  This label must be consistent on all hosts for vMotion.

Click finish to complete the process and repeat this on all of the hosts in the cluster being sure to make the network labels identical (case sensitive).  You will need to repeat this for every VLAN that you will need for the Nexus install including the Nexus Packet, Nexus Control and Management VLANs.

Cisco Nexus 1000v Part 2

As promised in my last post, this post will be an explanation of how the Nexus 1000v's architecture is laid out as well as how that fits into vSphere.  Cisco uses the familiar imagery from the physical world of a chassis (think Nexus 7000 or Catalyst 6500) with line cards.  In the physical world you would have one or two supervisor engines that provide the brains and then several line cards that provide ports.  In the Nexus 1000v paradigm, the chassis is a virtual container in which you place one or two Virtual Supervisor Modules (VSM) that are actually guest VMs that run NX-OS.  The VSMs can be hosted on the ESX cluster, another ESX host, or the Nexus 1010 appliance.  These VSM modules communicate through the VMWare infrastructure to a Virtual Ethernet Module (VEM) that resides on each ESX host.  Think of the connection through the infrastructure as the back plane or switch fabric of the virtual switch.  The VEMs show up in the Nexus 1000v as line cards in the virtual chassis.

The VSMs communicate with vCenter to coordinate the physical and virtual NICs on the servers and how they are connected to the VEMs.  You use vCenter to manage which physical NICs are associated with a VEM.  Physical NICs show up as eth<id> interfaces on the Nexus 1000v.  The virtual NICs associated with the hosts show up as veth<id> interfaces.  For those not familiar with NX-OS, ethernet interfaces can be 10Mbps, 100Mbps, 1Gbps or 10Gbps.  Unlike IOS the name doesn't designate the speed.


Because of the way that the VEMs communicate with the VSM, it is crucial to maintain the networking links between the VSMs and the VEM or the VEM will disappear from the Nexus 1000v.  If it is disconnected, the VEM continues to forward traffic in the last known configuration but it is not configurable.


In my next posts on the Nexus 1000v, I will run through the basics of getting a Nexus 1000v installed into a vSphere 4.1 environment.

Catalyst 6509-E VSS Software Upgrade Gone Bad

My work network has a pair of Cisco Catalyst 6509-E chassis that are configured in a Virtual Switching System (VSS) to serve as the network core.  Last week we had a supervisor engine crash and were having some residual craziness with our CAM table.  TAC suggested a reboot and software upgrade so we scheduled one for Sunday afternoon.  

Usually a software upgrade on the 6509 is relatively painless, but this time it proved to be very painful.  The previous software load on the VSS pair was 12.2(33)SXI, but it was the modular version (keep this in mind it's important).  The new software load suggested by TAC was 12.2(33)SXJ1 which as of SXJ is only offered in monolithic versions.

Assuming that all was well with these two versions, I started down the path of doing an enhanced Fast Software Upgrade (eFSU) of my VSS pair using the ISSU commands as listed in the Catalyst 6500 Release 12.2SX Software Configuration Guide - Virtual Switching Systems (VSS) on Cisco's website.  After issuing issue loadversion disk0:s72033-ipservicesk9_wan-mz.122-33.SXJ1.bin on the active console, I waited for the standby chassis to reload.  Unfortunately it entered a reboot loop because the new software was not compatible for ISSU.  Here is where it got hairy.  At this point I could neither abort nor complete the upgrade on the active supervisor.  It wouldn't let me change the boot system variable because it had a state somewhere that said it was in an ISSU upgrade even after power cycling the chassis.  

After 5 hours on the phone with TAC, we were able to clear this persistence and finish the upgrade, but it was a very long downtime.  The moral of the story... modular and monolithic IOS don't mix well.

Installing NMIS (Network Management Information System)

Recently I was a presenter at the annual Tech Feast conference held at Heartland Community College in Normal, IL.  The conference is a gathering of technology educators in the K-12 schools of Illinois.  In my presentations on network troubleshooting I mentioned the open source tool NMIS and had several requests for more information on the installation and use of the tool.  This post will serve as an installation guide from a bare bones Ubuntu 10.04 LTS server install.  A subsequent post will cover configuration of NMIS.

Cisco Nexus 1000v Virtual Switch for VMWare ESX

Recently the server group came to me and let me know that they had purchased Nexus 1000v licensing for the new ESX cluster.  As a Cisco geek, I was pretty stoked to get to work with the Nexus 1000v as it was virtual and the first Nexus platform I would get to work on.

In this article, I am going to try to lay out some of the basic nomenclature surrounding the Nexus 1000v.  If you are like me and have been living in your network world with little exposure to the guts of VMWare, this article will hopefully help bring you up to speed.

The first concept that can get a bit confusing in the VMWare world is the way they refer to their NICs.  The physical NICs on the server are referred to as vmnic<x> starting with vmnic0.  When I was first getting started, I kept wanting to think that these were virtual NICs since it started with VM, but that is not the case.  Usually the onboard NICs are the first vmnics and then any expansion cards are after that.


Next there is the matter of the different types of virtual switches that can be configured.  The most basic type is the standard vSwitch.  To the average networking guy this is basically configuring the ESX host's interfaces to be used as tagged (trunked) VLAN interfaces.  Each VLAN that you want to support is added as a standard vSwitch.  Once added the vSwitch can be associated with any VM.  The drawback to vSwitch from a VMWare perspective is that they need to be identically configured on every ESX host in the cluster to allow for vMotion of VMs since their network has to be present for them to move.  From a network guy's point of view the drawbacks are numerous including:

• No ACL capabilities.
• No SNMP monitoring of traffic counters.
• No Netflow monitoring.

Realizing the problems with vSwitch design for vMotion, VMWare came up with the distributed vSwitch or DVS.  This switch is configured for the entire cluster and ensures that all members of the cluster have the same network configuration available for VMs.  From a VMWare perspective this is the cat's meow, but it still lacks SNMP, ACL and Netflow capabilities that us networking nerds crave.  

That's where the Nexus 1000v comes in.  It is a cisco developed distributed vSwitch that uses the VMWare APIs to give a full NX-OS controlled DVS.  If you can do it in NX-OS you can do it with the Nexus 1000v.  In my next post I will discuss how the Nexus 1000v architecture interacts with VMWare.

Configuration Rollback Update

Jeff over at www.fryguy.net has just blogged about the Cisco Archive and Configure tools that I discussed back in March.  He does a good job of explaining how to use it to avoid long downtimes for a mistyped command.

Review of Enterprise Network Testing: Testing Throughout the Network Lifecycle to Maximize Availability and Performance

I have recently finished reading the new Cisco Press book Enterprise Network Testing: Testing Throughout the Network Lifecycle to Maximize Availability and Performance by Andy Sholomon and Tom Kunath.  Prior to reading this book I will admit that my only experience with network testing has been much more adhoc than the structure they present.


The first section of the book is devoted to learning about the methodologies, tools and types of network testing.  This section discusses how to test networks and where in the lifecycle testing should take place.  It also builds a case for why network testing is important and how to get funding and management support for testing.  If you don't get anything else from this book, that chapter is probably the most important.  Getting budgets for testing capital expenses (lab gear, space, testing tools) and operational expenses (labor, power, cooling) require showing the return on investment that can be had from preventing downtimes later.


The second section of the book looks at how testing of various types has helped real world businesses.  These case studies give the reader a good idea of tangible benefits of network testing.  Most of this section reads more like a business textbook than a technical guide which should help network engineers bridge the gap to the holders of the purse strings.  As network engineers, we have to come out of our geek world and explain to non-geeks why what we do matters to them and to them business matters.


Finally in the last section the authors explore various network projects and the test plans used to verify their operation.  For someone like me with no formal network testing training or experience, this section is an invaluable resource.  The test plan examples are very thorough and will be a great reference for building your on test plans.


I highly recommend this book, although it may not be as sexy as say a book on the Nexus 7000 or FCoTR it provides useful skills for a network engineer to improve their network.

Fun with Port-Channels

This week I did what I've done hundreds of times before.  I configured a new port-channel on a Cisco switch for a server.  In doing so I ended up with two new scenarios that I had never seen before.


The first issue was more of a head scratcher than a production problem, but nonetheless it caused me to stop and wonder why.

PAHA-1A-6509E-SW1#show etherchannel 403 summary
Flags:  D - down        P - bundled in port-channel
        I - stand-alone s - suspended
        H - Hot-standby (LACP only)
        R - Layer3      S - Layer2
        U - in use      N - not in use, no aggregation
        f - failed to allocate aggregator

        M - not in use, no aggregation due to minimum links not met
        m - not in use, port not aggregated due to minimum links not met
        u - unsuitable for bundling
        d - default port

        w - waiting to be aggregated
Number of channel-groups in use: 34
Number of aggregators:           35

Group  Port-channel  Protocol    Ports
------+-------------+-----------+-----------------------------------------------
403    Po403(SD)       LACP
403    Po403A(SU)      LACP      Gi1/3/45(P)    Gi1/4/14(P)    Gi2/3/45(P)

Last applied Hash Distribution Algorithm: Fixed

Obviously it's not exactly normal to see a port-channel identifier listed twice with one of them having an alphabetic character on the end.  Digging around on the Internet I finally found a document on Cisco's support community that explained the issue. What I had done was create the port-channel with Gi1/3/45 and Gi2/3/45.  Later when the server guys had asked me to add Gi1/4/14 I added it to the port-channel before making the configuration identical.  The document's resolution of shutting down the ports and defaulting the configurations and then recreating them did remove the crazy A from my show commands.

My second issue was that a server connected to a port-channel was disabling one of the NICs saying that it was faulted.  Again defaulting the configurations and recreating the port-channel cleared up that issue.  

The moral of my story is that even though your configuration may be 100% correct, you may have confused IOS so try starting over.

Cisco Learning Labs

Well I have now used the new IOS on Unix (IOU) based virtual labs from Cisco for the past three days for my CCNP ROUTE course.  So far I give the lab environment itself a B+.  Being that it is real IOS, the routers are full featured.  Unfortunately the web interface surrounding the IOS instances is still a bit clunky.  I would like to see tools to save and load configurations via text files.  I would also like to see a better L2 switch implementation.  The switches in the labs were unable to be setup for more than one vlan which meant the topology wasn't really setup correctly.


The lab exercises were where things really broke down.  Because you can't change the wiring configuration or the number of devices, you have to rely on Cisco to build the appropriate topologies and initial configurations to match the lab guides.  Unfortunately they apparently never reviewed the lab guides for the OSPF labs in the CCNP course.  None of those labs was configured correctly which caused a lot of extra work before being able to even start the lab you were assigned.  I guess that Cisco was trying to mix TSHOOT with ROUTE.  Because of this I have to give the lab setup a D (the EIGRP labs were done correctly).


Being that this lab environment was just released recently, I'm hoping that the quality improves quickly as people report the problems.

Cisco IOS on Unix for Class

Cisco has recently started releasing IOS on Unix through the Cisco Learning Network store as a hosted Lab solution.  For those not familiar with IOU, it is an image of IOS that is designed to run as a virtual instance on Solaris.  Generally it has only been available internally at Cisco.

This week I am taking an online ROUTE course through New Horizons and they are using the Cisco Learning Network hosted labs for the course.  So far I have found it to be a very good tool.  The IOS instances are very responsive and full featured.  No errors about a feature not being implemented in the simulator (because it's not a simulator).  The web interface to get to the labs is still a bit rough, but I figure that Cisco is probably working on that.  As others have noted, the timer starts as soon as you enter the lab so reading the lab materials does count against your block of time.  For the class that isn't too important as we have a printed lab materials, but if you buy your own block of time you will want to enter the lab, print the lab, and then exit while you read to save your time.  

One of the biggest advantages to me so far has been that I have my own "pod" to do the class labs on.  I find that I have "finger memory" so even if I see something done, I don't learn it until I've typed the commands myself a few times.  Often in past Cisco courses I have had to share a pod of equipment which meant not always getting true hands on time.

All in all, this is a great step forward for Cisco.  Now if they would please release IOU to their customers to use for adhoc labs and testing purposes they would really have something.

Review of IPv6 for Enterprise Networks

IPv6 for Enterprise Networks by Shannon McFarland, Muninder Sambi, Nikhil Sharma, and Sanjay Hooda is a timely new book that helps network engineers plan for the upcoming switch from IPv4 to IPv6.  The authors start with a quick review of CCDA/CCDP topics relating to networking models that they then use throughout the book to compare IPv4 deployments to IPv6 deployments. The one thing that I found lacking in the review was a quick overview of IPv6 itself.  Especially since Cisco Self-Study: Implementing IPv6 is nearly 8 years old and a lot has changed in that time.


The next part of the book explores the process of migrating from IPv4 to IPv6 in an enterprise.  It looks at justifications, education and planning the migration.  There is a very useful section that discusses the challenges to expect from internal and external applications.  


Finally the last part of the book is dedicated to examining the differences in implementation between IPv4 and IPv6.  The authors do a wonderful job of using real world examples from various routing protocols to pinpoint the similarities and differences with lots of screen shots.


This book will definitely be a great resource for any network engineer that will be going through the IPv4 to IPv6 transition.  That would be any network engineer not retiring in the next year.  I hope that Cisco Press will be forthcoming soon of an updated version of Cisco Self-Study: Implementing Cisco IPv6 Networks (IPV6).


Do you have a favorite learning resource for IPv6?  Please share it in the comments section.  It's going to take a lot of effort for network engineers to stay up to date during the shift since we'll be basically relearning everything we know in a new way.  Sort of like someone that knows English learning Old English.

Using Cisco Wireless Lan Controllers for a Wired Guest Network

Most people are familiar with using Cisco's mobility anchors to provide a wireless guest network.  The usual topology is that you have a controller with CAPWAP access points connected to it and then a foreign "anchor" controller that is located in the DMZ.  Between the two controllers traffic is tunneled to logically separate the traffic from the rest of the regular network. In this way guest traffic can be effectively managed while keeping internal resources secure.

The same separation can be achieved for a wired guest LAN by using the same WLC architecture.  Setting it up requires the following:

• The WLC inside the network must have it's interface setup as a dot1q trunk to handle multiple VLANs
• UDP Port 16666 or IP Protocol 97 must be able to pass through the firewall between the inside WLC and the anchor WLC.
• A dedicated L2 VLAN for the guest wired network
• Mobility Groups are already configured on the WLCs.
• DHCP configured on the Anchor Controller

Configure the Ports

First go to the switch connected to the local WLC and do the following assuming that the WLC is connected to Gi1/0/1 and the new guest wired vlan is 601

switch#configure t
switch(config)#interface Gi1/0/1
switch(config-if)# switchport trunk allowed vlan add 601

Next go to the switch connected to the access port(s) that you want on the guest wireless and do the following:

switch#configure t
switch(config)#interface range gi2/0/1 - 5
switch(config-if-range)# switchport access vlan 601

 Configure the Local WLC


Login to the local WLC web GUI and go to Controller and then Interfaces and click New.  This will bring you to the following screen where you give the Interface a name and tell it what VLAN it should be on.  

Once the information is entered, click apply.  This will bring you to another settings page for the new interface.  In this screen you need to click the box marked Guest LAN and then click Apply.

Now go to WLAN and then click Go next to where it says Create New.  This will bring you to the following screen where you will choose type Guest LAN and give the profile a name.

Clicking Apply will bring you to the WLANs Edit screen for the new profile.  Here there are a few choices to be made about security.  The default is that users will need to login using web authentication if they want onto this guest network.  This requires that you have either setup local usernames and passwords, perhaps through the lobby ambassador setup, or have guest accounts through AAA.  For this example I will use the Web Passthrough option with makes the user accept an acceptable use policy before gaining access.  On the general tab I clicked enable and chose the guest-wired-lan interface that we created earlier as the Ingress Interface.

Next I clicked on the Security Tab and Layer 3 Tab to change the Layer 3 Security to Web Passthrough.  Note that you also have the ability to require web passthrough users to give you their e-mail address, but it doesn't validate it is a real address.

Once you click Apply you will be back to the WLANs list page.  On the right side of the page there is a blue box with a white down arrow next to the WLAN profile we created.  Hover over it and choose Mobility Anchors.  Once you're in that page you will choose the IP of the foreign anchor controller from the drop down and then click Mobility Anchor Create.

This completes the configuration of the local WLC.


Configure the Anchor Controller


Login to the anchor controller and click on WLANs and then Create a new WLAN profile just like the one on the Local Controller above except that the ingress interface is None and don't click enabled.  Once you click apply, go to the mobility anchors for the new profile and make sure the Switch IP Address is set to (local) and click mobility anchor create.  Once you have done this, you can go back to the profile screen and check enabled.  


Wrap Up


This article is a rather whirlwind how to of setting up a guest wired network using the Cisco Wireless LAN Controllers.  There are many intricacies not discussed here, but there are plenty of good documents on Cisco's website about how to configure the WLCs.  I do have a question for anyone from Cisco or a wireless guru.  If you have multiple WLCs on the local LAN, can you have the same wired guest vlan piped to all of them or do you have to have a different vlan for each controller?  If you can have the same vlan, how does the traffic know which controller to associate with?

P@$$W0rD$

As a young naive network admin I thought I could remember everything about everything.  Writing things down, especially passwords, was a waste of time and a security problem.  Well, as things often do, my view on this has come full circle.

Password management is an important part of network documentation.  Without a plan, you end up with two things happening.  First you end up forgetting that rarely used password for the oddball piece of equipment in the back closet of the back room of the building into which no one ever goes.  Unfortunately the day after you forget totally about it, it acts up and you have to get access to it's command line quickly to fix the problem.  Second you end up with a lot of systems having the exact same username and password combination.  This means that when you have to share the password with a technician that doesn't need access normally you will have to reset the password everywhere.  Oh and you do remember everywhere it needs reset right?  

Both of those scenarios were about keeping yourself in the loop, but password management is also important in the event of a loss of personnel.  I often refer to this as the "What if Ben is hit by a bus tomorrow?" information that someone will need to run things if I die or otherwise am no longer at the company.  I really don't want the company sending Jack Bauer into my ICCU room to coerce the passwords out of me while doped up for pain or something.

My solution to managing passwords, both professionally and personally, is to use an application that creates an encrypted file containing all of the passwords.  There are two such software packages (both opensource) that I have used.  The first is PasswordSafe and the other is KeePass.  I personally have settled on KeePass because the application is available for Linux (using Mono), Windows and iOS (Apple, not Cisco).  I have two files, one for work and one for home.  The iPhone app is synchronized manually using a builtin webserver over the phone's wifi connection.  I use Dropbox to synchronize the files between computers.

Cisco Config Rollback and Replacement

In the previous two posts I have outlined how to create a configuration archive and how to compare configurations.  In this post I will look at how to take an archived configuration and use it to replace the current configuration for rolling back changes.  This article assumes that you already have a configuration archive in place.


Before making any changes to the configuration you need to run the command archive command to force IOS to make a backup copy of the current running-config.  Once this is done, make any changes to the configuration.  At this point you've either got a new running-config that you like and that works or you have created a problem and it's time to rollback through configuration replacement.


If you need to rollback the changes, you will need to do the following:

Router# configure replace scp://scpuser:scppasswd@scpserver/configuration1.cfg revert error  time 5
Router#configure revert now
Router#configure confirm

The configure revert now and configure confirm are both optional commands depending on how you configure the replace command and the outcome.  For example if you are sure that you don't want to revert to the previous running-config typing configure confirm will make the change permanent and resets the timer set in the replace command.  In the above example because we set the time 5 part of the command we must issue configure confirm within 5 minutes or IOS will automatically revert to the previous running config.  If you find that you need to revert  immediately to the previous config you run the configure revert now command. 

As you can see, this can be good for making and rolling back changes as necessary.  Theoretically you can also take a configuration file and edit it offline and then copy it back to the device with the changes.  I say theoretically because you have to be very careful to meet the requirements of the IOS Configuration file in terms of indention.

Cisco Config Diff Tools

In the last post I examined the archive tools provided in IOS to backup device configurations to a FTP, TFTP or SCP server.  Now I would like to look at some of the tools IOS provides to work with these archived configurations.


Diff


Contextual Configuration Diff Utility is the official Cisco name for what Linux people call diff.  Diff in it's most basic form takes two text files and tells you what is different between the two files line by line.  Before we look at IOS let's take a basic example of diff from a Linux box.


Listing of File1.txt

apples
beer
coffee
donuts
eclairs

 Listing of File2.txt

apples
beer
cottage cheese
donuts
fudge

 Example of diff

storyb@nettools:~$ diff file1.txt file2.txt
3c3
< coffee
---
> cottage cheese
5c5
< eclairs
---
> fudge

In this example, you can see that it shows that coffee was in the first file, but cottage cheese was in the second and likewise that on line 5 eclairs was in the first file, but fudge was in the second.


Cisco's Diff


Cisco's diff works very similarly to Linux's.  In the following example of using IOS' diff we will use the following two abbreviated IOS configuration files.


Listing of iosconfig1.cfg

no ip subnet-zero
ip routing
router ospf 1
  network 192.168.1.0 0.0.0.255 area 0
interface Ethernet 0/0
  ip address 192.168.1.1 255.255.255.0
  no ip route-cache
  no ip mroute-cache
interface Ethernet 0/1
  shutdown

Listing of iosconfig2.cfg

ip subnet-zero
ip routing
router ospf 1
  network 192.168.1.0 0.0.0.255 area 0
interface Ethernet 0/0
  shutdown
interface Ethernet 0/1
  ip address 192.168.1.1 255.255.255.0
  no ip route-cache
  no ip mroute-cache

 Example 1: show archive config differences

show archive config differences scp://scpuser@server/iosconfig1.cfg scp://scpuser@server/iosconfig2.cfg

+ip subnet-zero
interface Ethernet 0/0
  +shutdown
+interface Ethernet 0/1
  +ip address 192.168.1.1 255.255.255.0
  +no ip route-cache
  +no ip mroute-cache
interface Ethernet 0/0
  -ip address 192.168.1.1 255.255.255.0
  -no ip route-cache
  -no ip mroute-cache
interface Ethernet 0/1
  -shutdown

There is also a second version of the diff utility for IOS that compares a configuration file with the running configuration and tells you what lines are in the config file that are not in the running configuration.  This command is show archive config incremental-diffs <filename>.  It is useful to compare a previous archive to the running configuration to see what had been deleted from a configuration.


My next post will look at the tools that IOS provides to do configuration replacement and rollback using archived configurations.

Cisco Config Archiving

Not everyone can afford to install and maintain a massive change management database application for their IT infrastructure.  Thankfully at least on Cisco IOS devices it is quite simple to setup a reliable way to archive the devices' configurations.  For those of you that maybe aren't sure why this is important, let me give a few examples of when it could be very useful.

1. Device Failure:  Imagine that your core switch or router fails hard and you have to put in a replacement.  Think about how long it would take to configure the new device from scratch and how error prone the process would be.  Instead you could just slap the configuration that was archived from the old device on the new device and be back in business in a few minutes (not including the time to get the spare in place physically).
2. Change:  So what happens when the network goes down or has a problem?  The first question is usually "What changed?".  Your configuration archive along with some tools like diff or the built in IOS archive compare can tell you what has changed on a device.  
3. Auditing:  Many companies are subject to legislation like Sarbanes-Oxley or HIPAA.  Tools are available to perform audits on IOS configs to ensure compliance.  With an archive you can run these tools against the directory of text files quickly without having to allow the scripts access to the live device.

Ok so now that you know WHY you want configuration archiving, let's look at how to configure it.  The first step is that the device must be running an IOS version that supports the command.  In router IOS it was introduced in 12.3, in switches it has been back ported to 12.2.  

Assuming that the device now supports the archive commands, setting up archiving is fairly straight forward.

router# config t
router(config)# archive
router(config-archive)# path tftp://tftpserver.example.com/$h.cfg
router(config-archive)# write-memory
router(config-archive)# time-period 360
router(config-archive)# end
router# copy run start

The first command is simply archive which puts you into config-archive mode.  Next we set the path to where we want the archived configurations to be stored.  In this case we're putting it onto a TFTP server.  Because I like to have template configurations that I can easily paste into a device I used $h to have the router put the host name automatically into the path statement.  The next statement write-memory tells the device to archive the configuration every time someone does a copy run start or wr mem.  Finally time-period tells the router the amount of time in minutes between archives.  In this case the router would archive the configuration every 6 hours.


As configured above, the router would put a new file into the TFTP server every 6 hours forever or until the TFTP server ran out of space.  Thankfully for some URL types like SCP and FTP, IOS will manage the number of archived configurations for you.  With those types of URLs you can specify maximum <number> in the config-archive mode to tell IOS how many backup copies to keep.  When it hits the maximum it will overwrite the oldest file the next time it archives the configuration.


In my next post I will look at the tools within IOS for using archived configurations to find differences and to rollback to old configurations.