Automatic Recovery for Err-disabled Interfaces

There are four primary states for interfaces on Cisco switches: up, down, administratively disabled and err-disabled.  Up and down are fairly self explanatory.  Administratively disabled means that the port is configured to be shutdown by the administrator using the CLI.  Err-disabled though can be a bit baffling to a new network engineer.

The err-disabled interface state can be caused by many situations including:

• Bad cabling
• Duplex mismatch
• BPDU guard violation
• Port-Security violation
• Link-flap detection

The complete list is on Cisco's site.

An engineer can recover an interface by entering configuration mode for the interface and issuing the shutdown and then no shutdown commands.  By default the interface will remain err-disabled until a human intervenes because auto recovery is disabled as is shown by the following show command.

SWITCH#show errdisable recovery
ErrDisable Reason            Timer Status
-----------------            --------------
arp-inspection               Disabled
bpduguard                    Disabled
channel-misconfig (STP)      Disabled
dhcp-rate-limit              Disabled
dtp-flap                     Disabled
gbic-invalid                 Disabled
inline-power                 Disabled
l2ptguard                    Disabled
link-flap                    Disabled
mac-limit                    Disabled
loopback                     Disabled
pagp-flap                    Disabled
port-mode-failure            Disabled
pppoe-ia-rate-limit          Disabled
psecure-violation            Disabled
security-violation           Disabled
sfp-config-mismatch          Disabled
small-frame                  Disabled
storm-control                Disabled
udld                         Disabled
vmps                         Disabled

Timer interval: 300 seconds

Interfaces that will be enabled at the next timeout:

 In some cases, it would be safe to allow the switch to auto recover the interface to up if the condition that caused the err-disabled state has cleared.  For this example, let's assume that a port-security violation caused the error (psecure-violation).  This is a relatively benign error to auto recover because if the violation still exists, port security will rapidly trip again putting the interface back into err-disabled.  The default is that the switch will clear the state after 5 minutes.  So to have the switch auto recover the interface the following configuration would need to be added.

SWITCH# configure terminal
SWITCH(conf)#errdisable recovery interval 300 ! Default setting shown for completeness.
SWITCH(conf)#errdisable recovery cause psecure-violation
SWITCH(conf)#end
SWITCH#copy running-config startup-config

 Similar commands can be entered for the other reasons listed above in the show command or you can set all reasons to recover by using the keyword all.  Be careful where you enable the auto recovery, it might not be your friend on all switches.  For example, you wouldn't want a link on a core switch having a problem to start flapping because of auto recovery causing a network convergence every 5 minutes (or whatever you set the timer to).

Neuron: Cisco Switch Firmware Archive Command

First I should introduce this type of post.  For this blog, a neuron will be a short tidbit of information.

Anyone that has upgraded a Cisco switch in the last few years knows that they are usually distributed as a tar archive now.  To install the upgrade you do the following:

#archive download-sw tftp://tftpserver/upgradefile.tar

When you execute the command IOS downloads the file and extracts it onto the flash file system.  All you have to do after that is reboot.

Thanks to Cisco's latest grab for more money, if you don't have SmartNet on a piece of equipment, you can't download IOS code for it.  While this has long been their policy, it is now being enforced.  This isn't too much of a problem unless you have a device that dies and you want to replace it with a replacement that is also not under SmartNet.  The likelihood of the replacement switch coming in with the exact same IOS load is close to nil.  Most admins like to maintain certain revision levels on a certain model which poses the problem of how to get the IOS you want on the replacement.  Well the easiest way that I have found is to use Cisco's archive command again.  Keep in mind that it's best to do this BEFORE you have a switch crash.

#archive upload-sw tftp://tftpserver/firmwarefile.tar

When you execute this command, IOS will combine all of the files on the flash file system related to the IOS code into an tar archive and upload it to your TFTP server.  The resulting tar file can then be used like the stock Cisco firmware tar file.