Posts tagged ‘cisco’

4 May, 2012


by gorthx

When I first started diving into QoS, I really hated it. There are a lot of moving parts, and whenever I poked at it I felt like I was sticking my hand in a moving engine. I had the opportunity to take some Cisco training this year, but found out that Cisco got rid of their 5-day QOS class. (At the time, I was relieved, because 5 days solid of QoS sounded like something only a masochist would do.) The only QOS training Cisco offers now comes as part of the CVOICE. I signed up for that, figuring I’d take what I could get, with the added bonus that I’d be able to communicate better with my voice team [1].

read more »

13 April, 2012

Monitoring Cisco IP SLAs: syslog messages

by gorthx

Now that I have an SLA in place, and have some baseline data, I want some notifications in case my SLAs drop. I’ll cover basic syslog messages here, and SNMP traps in another post.

Cisco’s docs have a neat graph that shows how the reaction-config interprets the thresholds. There’s also a handy-dandy chart that tells us which reactions are available for each type of SLA.

I configured a udp-jitter SLA, so everything in the ‘UDP Jitter’ column that’s marked with a Y is something I can configure a reaction for. I’ll check RTT [1], jitter, MOS, and timeout for starters. Initially, I tried this out on just a few routers, with some numbers very close to my collected stats (see last week’s graphs), so I could make sure it was working. Here, I have adjusted them to some more realistic numbers; YMMV.

read more »

30 March, 2012

Monitoring Cisco IP SLAs: rrdtool

by gorthx

Last week I set up an SLA to monitor jitter & looked at the stats available on the command line. This week, I’ll graph those stats.

Like I mentioned, this is quick-and-dirty monitoring, just meant to give me an idea of what I’m working with. This isn’t necessarily something I’d deploy to production, especially since there are products available that already do this. (Which I hope to have time to review in the near future.)

These OIDs from CISCO-RTTMON-MIB look promising:

read more »

23 March, 2012

Configuring Cisco IP SLA for jitter

by gorthx

We’ve been playing around with IP SLAs at work lately (here’s a quick overview of SLAs and what they can do for you). Right now we’re mainly interested in monitoring VOIP services, so we set up an SLA to monitor jitter.

First, we need to configure a responder. This is the machine that will field all of the SLA requests. For our tests, we used a tiny lab router (a stock 2921) – we want to find out how much extra load SLAs would add, so we used something small in order to maximize opportunities for breakage.

read more »

10 February, 2012

Get Cisco Serial Numbers with SNMP

by gorthx

A friend asked me how to go about doing this, and I figured I’d post it here so he can find it again if he needs it.

You’ve already discovered that snmp-server chassis-id is a user-maintained field and therefore not reliable [1], so you can skip trying to use chassisId ( from OLD-CISCO-CHASSIS-MIB. It’s supposed to be depracated anyway.

Newer equipment supports the ENTITY-MIB. (For certain definitions of “support” … it’s not pretty.)

There are a number of ways to do this. The most straightforward, if you’re starting from scratch, is to walk entPhysicalClass and look for items of type 3, chassis.

It’ll look like this:
:::-->snmpwalk -v 2c -M ~/.snmp/mibs -m ENTITY-MIB -c public -O s myswitch entPhysicalClass | grep chassis
entPhysicalClass.1001 = INTEGER: chassis(3)

Then, use entPhysicalSerialNum and the iid you just found (the 1001 in the previous example) to find the serial number:
:::-->snmpget -v 2c -M ~/.snmp/mibs -m ENTITY-MIB -c public -O s myswitch entPhysicalSerialNum.1001
entPhysicalSerialNum.1001 = STRING: FOC14475A35

In case you don’t have those mibs installed & don’t want to bother with it, here are the numerical equivalents:
entPhysicalClass .
entPhysicalSerialNum .

That method would look like this:
:::-->snmpwalk -v 2c -c public -O s myswitch . | grep "INTEGER: 3"
mib- = INTEGER: 3
:::-->snmpget -v 2c -c public -O s myswitch .
mib- = STRING: "FOC14475A35"

Now, if you want to get fancy and maybe find out the model number as well, you can then check any of the following (in order of how useful they’ve been to me personally):
entPhysicalModelName .
entPhysicalDescr .
entPhysicalName .

…which would look like this:
:::-->snmpget -v 2c -M ~/.snmp/mibs -m ENTITY-MIB -c public -O s myswitch \
entPhysicalModelName.1001 \
entPhysicalDescr.1001 \
entPhysicalModelName.1001 = STRING: SM-ES3G-16-P
entPhysicalDescr.1001 = STRING: SM-ES3G-16-P
entPhysicalName.1001 = STRING: 1

1 – These things tend to migrate off the equipment they were originally configured on, on to other machines, via a copy & paste vector. Pretty soon you have five or six different boxes that supposedly have the same serial number.

9 December, 2011

Friday Snark – Cisco’s VTP MIB

by gorthx

OK, Cisco, with the CISCO-VTP-MIB? Let’s look at the vtpVersion OID, for starters. One would expect that it would contain, maybe, the vtp version running on the host, right?

Check this out, from the definition for that OID in the mib (available here)

"The version of VTP in use on the local system."
…so it is the VTP version…

"A device will report its version capability and not any particular version in use on the device."

…only not really. (Yes, the documentation contradicts itself. This is one of the reasons network management is so frustrating.)

Turns out you need to read further down in the MIB and use managementDomainVersionInUse (“The current version of the VTP that is in use by the designated management domain.”) to get the VTP version currently running. Right-O.

(The rest of the definition for vtpVersion states: “If the device does not support vtp, the version is none(3).” This wasn’t my experience – most of my switches (3500 and 6500 series) reported ‘none’, and were actually running v1 or v2.)

26 August, 2011

This week’s tips

by gorthx

Two things I learned on accident this week:
vim: when your cursor is over a number, ctrl-a increments it. (Members of @pdxhackathon tell me that ctrl-x decrements it.) Not sure where I’ll use it, but it’s interesting.

IOS (the Cisco kind): ‘sh run’ and then ‘/[string]’ to search is way faster than ‘sh run | include [string]’.

Also, a few weeks ago, I upgraded my production systems to PostgreSQL 9. A seamless transition, as usual. So far the only thing that’s tripping me up is the change to d [viewname] (see the release notes) – you now have to use d+ [viewname] to see the queries a view is based on. Just a matter of training my fingers to do something different.

I also discovered how to drop multiple columns from a table in a single statement:
ALTER TABLE tablename DROP COLUMN column1, DROP COLUMN column2, DROP COLUMN column3;

I don’t know if that’s new or I just figured it out (and I’m too lazy to go look up the release notes to find out right now), but I like it.

17 September, 2010

Altering [Custom] Syslogs on a Cisco ASA 5550

by gorthx

Since last time, I’ve made a couple of additions to my event list, and my custom logging filter now looks like this:

vpn# sh run | include logg
logging enable
logging timestamp
logging list gabs_test level notifications
logging list gabs_test message 713228
logging list gabs_test message 113006
logging list gabs_test message 302010
logging list gabs_test message 113005
logging trap gabs_test
logging asdm informational
logging facility 22
logging host internal [ip]

I’d also like to capture the user’s client, which is specified in this message:
%ASA-6-713184: Group = [group], Username = [user], IP = [internal_ip], Client Type: Linux Client Application Version: whatever

I need to disable the filter anyway while I add the new message, so I’ll just re-write the event list with the messages in numerical order.* I’m also going to do this from the command line, because it is so much faster than going through the ASDM gui:
vpn# config t
vpn(config)# ! disable the logging filter by setting the level
vpn(config)# ! back to notifications while we work
vpn(config)# logging trap notifications
vpn(config)# ! remove the logging filter entirely
vpn(config)# no logging list gabs_test
vpn(config)# ! re-create filter with additional messages
vpn(config)# logging list gabs_test level notifications
vpn(config)# logging list gabs_test message 113005
vpn(config)# logging list gabs_test message 113006
vpn(config)# logging list gabs_test message 302010
vpn(config)# logging list gabs_test message 713184
vpn(config)# logging list gabs_test message 713228
vpn(config)# ! re-enable the filter
vpn(config)# logging trap gabs_test
vpn(config)# exit

make sure it looks right with ‘sh run | include logg’ and save it with ‘write’ or ‘copy run start’.

*Neatness counts, people!

3 September, 2010

Configuring [Custom] Syslogs on a Cisco ASA 5550

by gorthx

Configuring logging on a Cisco ASA 5550 isn’t too tricky, but I found the docs lacking in examples, so figured I’d post mine.

Using the Cisco ASDM, the parameters you need to modify are in the Configuration -> Device Management -> Logging hierarchy. (I’m sorry there aren’t any pictures to go with this; hopefully you can follow along via the breadcrumbs in the ASDM.)

1. First, enable logging:
Configuration -> Device Management -> Logging -> Logging Setup
Make sure the “Enable Logging” box is checked.

2. Specify parameters for the destination server:
Configuration -> Device Management -> Logging -> Syslog Servers
Click “Add”.
Select the source interface (probably “internal”).
Enter the IP address of the server.
Select an appropriate protocol & port (UDP and 514 are the defaults).

3. Configure your logging facility:
Configuration -> Device Management -> Logging -> Syslog Setup
Choose your facility (0-7). Because I have one central server that handles all my logging, I usually set this to something special to allow me to split out the messages into different log files. (This is a configuration parameter in /etc/syslog.conf and is covered in another post.)

4. Configure the severity level you want to log:
Configuration -> Device Management -> Logging -> Logging Filters
Select “Syslog Servers” and click “Edit”.
Filter on Server -> Notifications (logging level 5 – you’ll get every message between Emergency and Notification levels, inclusive.)

Verify that you are receiving messages on the syslog server. You can stop here if this is all you want.

We, however, decided that we wanted to include these messages:
%ASA-6-713228: Group = [group], Username = [user], IP = [external_ip], Assigned private IP address [internal_ip] to remote user
…but these are “informational” messages (note the “6” in the second message field). So we tried bumping up our logging level, and were promptly overwhelmed with a lot of other stuff we didn’t want to read. (Informational level is chatty.) What we really want is to log at the Notification level, but also include messages with ID 713228.

Tip: don’t rely on the syslog message reference (or on this post, for that matter) to tell you which messages you might want to log – look at your own logs. The message texts in the logs won’t necessarily match those in the docs.

There are a couple of ways to configure custom logging of this type:

1. Simplest:
Configuration -> Device Management -> Logging -> Syslog Setup
Find the message ID (713228 in our case) in the “Syslog ID Setup” window.
Select the message ID, click “Edit” and change the level to “Notifications”.
Click “OK” to finish.

Apply and save your changes.

Here’s what Option #1 looks like from the cli:
vpn# sh run | include logging
logging enable
logging timestamp
logging trap notifications
logging asdm informational
logging facility 22
logging host internal [ip]
logging message 713228 level notifications

(Logging facility 22 corresponds to local6; see


2. More complex:
Configuration -> Device Management -> Logging -> Event Lists
Click “Add”.
Give it a name (no spaces), eg gabs_test.

Configure the base event class and/or severity:
Under “Event Class/Severity Filters”, click “Add”.
Leave “Event Class” set to “All”*; select “Notifications” under “Severity”; click “OK”.

*Apparently the first three numbers of the message are associated with the event class, but I was unable to find a reference doc that specified which were which. That doesn’t matter here because we want all Notifications, regardless of event class.

Then add the additional specific message ID we’re interested in:
Under “Message ID Filters”, click “Add”, then enter the message ID (again, 713228); click “OK”.

Click “OK” one more time to save your new Event List.

Now apply the Event List you just created to the appropriate destination (syslogs, in our case):
Configuration -> Device Management -> Logging -> Logging Filters
Select “Syslog Servers” and click “Edit”.
Check the “Use Event List” button, select your list, and click “OK”. Notice that the event list now appears in the “All Event Classes” field.

Apply and save your changes.

Here’s what Option #2 looks like from the cli:
vpn# sh run | include logging
logging enable
logging timestamp
logging list gabs_test level notifications
logging list gabs_test message 713228
logging trap gabs_test
logging asdm informational
logging facility 22
logging host internal [ip]

Why would you select one method over the other? Method #1 is certainly easier, but it does require you to remember that you’ve altered the default behavior. Method #2 is a bit more complicated to set up, but it’s more obvious that you’re doing something special due to the “Event list” keyword in the Logging Filters page, so that’s the method I ultimately chose.

Tip: You can’t rename or copy a filter (boo!) so choose your name carefully.

Tip: You also can’t edit a filter while it’s in use – you need to remove it from the destination under “Logging Filters”, apply/save, edit the filter, apply/save, re-enable it for the destination, and apply/save.

Obviously, there is a lot more you could do with this, by creating different custom filters and then applying them to different email destinations. I’ll save that for the next rainy weekend.


Tags: ,
14 November, 2008

Cisco Syslog Parser – slides

by gorthx

Here are the slides from my talk this week. A link to the accompanying podcast will be along soon.

Other fun things we discussed at the meeting:

Thanks for the lively discussion!

[edit] podcast!

[edit] Clarification of two items from the podcast:
– multiline messages do indeed come in multiple packets. There is a message counter that increments for each message, so you could use the host name + message counter to match up multi-line messages. For what I’m doing, the important part is in that first line, so the payoff isn’t worth the investment.
– re hypens in the mnemonic field of the system message: I went back through and wasn’t able to find any examples of this, so I retract my statement. (I do have examples of system messages with hyphens in the facility field.)

Tags: ,