Posts tagged ‘command-line’

1 August, 2016

Upgrade Ubuntu 15.04 to 16.04

by gorthx

I use Ubuntu on my home machines, and tend to rely on the GUI Software Updater for upgrades & such.  Life happens and I left one of them on 15.04 past EOL, and the Software Updater didn’t provide an upgrade path directly to 16.04.  I had to upgrade to 15.10 first, which is also EOL.

Here’s what I pieced together from various forums around the net.  (I took this after-the-fact from my command-line history;  I’ll take better notes next time.)

apt-get update
apt-get upgrade # make sure system is as up-to-date as possible
apt-get autoremove # clean up some leftover crap

apt-get dist-upgrade # recommended next step on one of the forums,
                     # but didn't actually do anything
                     # and I probably could have skipped this

do-release-upgrade -d # supposely this should have worked without the -d
                      # but it didn't
                      # -d means "latest development release"

And booyah, I’m on 15.10 and could continue on to 16.04.


2 February, 2015

Updating My Linux Command line Toolbox, episode 3

by gorthx

Part 2

This week’s tips:

1. ulimit -a will show you all settings, plus the units.

2. crontab -l -u [user] will read out another user’s crontab for you (assuming you have the right perms)

3. and what I call “diff-on-the-fly” – pass the output of shell commands to diff. I like this one because I don’t make a bunch of “temporary” files that I forget to clean up later.

diff <([shell commands]) <([other shell commands])

For example, I need to compare ids in two files, but they’re in different fields in each file, and not in the same order:

diff <(cut -d"," -f1 file1 | sort -u) <(cut -d"," -f3 file2 | sort -u)

6 October, 2014

RDS: Three weeks in

by gorthx

I’ve spent the past few weeks learning my way around Amazon’s RDS offering (specifically Postgres, and a bit of elasticache). It’s a mixed bag so far; for every feature I think “Hey, this is neat!” I find at least one or two others that are not so thrilling.

One of the things that may annoy you if you’re used to running your own Pg server is not having “real” superuser access to your own cluster. There’s an rdsadmin database which you won’t have access to, and a couple of rds users as well. This is part of Amazon’s security implementation to protect all their users from destroying each other.

You need to exclude the rdsadmin database out of any management queries you run, like so:
SELECT datname, pg_size_pretty(pg_database_size(datname))
FROM pg_database
WHERE datname NOT IN ('postgres','template0','template1', 'rdsadmin')
ORDER BY datname;

Otherwise you’ll get a permissions error.

Today, I had this bit of fun:
pg_dumpall --globals-only
ERROR: permission denied for relation pg_authid

So no dumping your roles. (No pg_dumpall, period.) I don’t have high hopes for an alternative.

I also haven’t found a place to report bugs – the “Feedback” button only allows you to contact tech support if you have a contract. So far I’m relying on the “I know someone who knows someone” method and the RDS user forums; AWS team members respond fairly quickly to posts there.

Aside from all that, I’m primarily interested in automating my instance management, so I’ve focused on the CLI tools.

My first thought was “Egads, I can’t get away from Java, can I”, but installing the toolkit turned out to be the easiest part. I set my $JAVA_HOME as outlined here to avoid that annoying “Unable to find $JAVA_HOME” error.

The CLI support is much more extensive than I expected – you can manage pretty much everything, very easily, once you find your way around and learn the appropriate incantation. Which is my biggest complaint: the docs on the web and the cli help don’t always match each other, and sometimes neither are correct. It cost me a significant amount of startup time flipping back and forth between the cli help, the web help, and just flat-out experimenting and cursing that mysterious “Malformed input” error message. (I probably have unrealistically high standards after working with the Pg docs for so many years.)

Fun stuff you can do:
RDS offers “event subscriptions” to help you keep tabs on your instance health (failover, storage, etc)1. They’re pretty easy to configure from the web console, but once you’ve done so, there’s no way to view or edit them except from the CLI. (At least, not that I can find.)

You can grab your Cloudwatch metrics, if you need to “roll your own” and integrate them into an existing monitoring system. (Yes, I briefly considered this!)

Log files are also accessible through the CLI for watching or downloading. It’s a huge improvement on the tiny green-text-on-black background on the web console. There’s no glob expression matching, so you have to request them one at a time. If you request a log file that doesn’t exist, you don’t get an error.

log_line_prefix is one of the unconfigurable GUCs on RDS, so if you are planning to use pgbadger 2, specify Amazon’s format as outlined at the bottom of this page.

1 – Be warned that restoring a snapshot will generate an “instance deleted” alert for your newly-restored instance. Your on-call person may not appreciate this.

2 – Alternatively, the pg_stat_statements extension is available on RDS, so you could get query stats this way.

26 August, 2011

This week’s tips

by gorthx

Two things I learned on accident this week:
vim: when your cursor is over a number, ctrl-a increments it. (Members of @pdxhackathon tell me that ctrl-x decrements it.) Not sure where I’ll use it, but it’s interesting.

IOS (the Cisco kind): ‘sh run’ and then ‘/[string]’ to search is way faster than ‘sh run | include [string]’.

Also, a few weeks ago, I upgraded my production systems to PostgreSQL 9. A seamless transition, as usual. So far the only thing that’s tripping me up is the change to d [viewname] (see the release notes) – you now have to use d+ [viewname] to see the queries a view is based on. Just a matter of training my fingers to do something different.

I also discovered how to drop multiple columns from a table in a single statement:
ALTER TABLE tablename DROP COLUMN column1, DROP COLUMN column2, DROP COLUMN column3;

I don’t know if that’s new or I just figured it out (and I’m too lazy to go look up the release notes to find out right now), but I like it.

17 September, 2010

Altering [Custom] Syslogs on a Cisco ASA 5550

by gorthx

Since last time, I’ve made a couple of additions to my event list, and my custom logging filter now looks like this:

vpn# sh run | include logg
logging enable
logging timestamp
logging list gabs_test level notifications
logging list gabs_test message 713228
logging list gabs_test message 113006
logging list gabs_test message 302010
logging list gabs_test message 113005
logging trap gabs_test
logging asdm informational
logging facility 22
logging host internal [ip]

I’d also like to capture the user’s client, which is specified in this message:
%ASA-6-713184: Group = [group], Username = [user], IP = [internal_ip], Client Type: Linux Client Application Version: whatever

I need to disable the filter anyway while I add the new message, so I’ll just re-write the event list with the messages in numerical order.* I’m also going to do this from the command line, because it is so much faster than going through the ASDM gui:
vpn# config t
vpn(config)# ! disable the logging filter by setting the level
vpn(config)# ! back to notifications while we work
vpn(config)# logging trap notifications
vpn(config)# ! remove the logging filter entirely
vpn(config)# no logging list gabs_test
vpn(config)# ! re-create filter with additional messages
vpn(config)# logging list gabs_test level notifications
vpn(config)# logging list gabs_test message 113005
vpn(config)# logging list gabs_test message 113006
vpn(config)# logging list gabs_test message 302010
vpn(config)# logging list gabs_test message 713184
vpn(config)# logging list gabs_test message 713228
vpn(config)# ! re-enable the filter
vpn(config)# logging trap gabs_test
vpn(config)# exit

make sure it looks right with ‘sh run | include logg’ and save it with ‘write’ or ‘copy run start’.

*Neatness counts, people!

9 December, 2008


by gorthx

(This is on solaris, no -N option to diff for me!)

This produces the easiest-to-read diff IMO:
diff -btu [oldfile] [newfile] > patchfile
-b = ignore blanks
-t = preserve source indentation
-u = 3 lines of context with the + and – in front of changed lines

But the -t option interferes with proper patch application, so just use:
diff -u [oldfile] [newfile] > patchfile
…where oldfile is the file you want to patch, and newfile is the file with the changes you want to apply.

patch -b -p0 < patchfile
-b = make a backup ;)

If you are asked for the filename to patch – you probably don’t have the depth set right with -p.  Try increasing it.

If you get the message “Reversed (or previously applied) patch detected!”, somebody messed with your stuff between creating the patch & applying it, OR you did the diff backwards (common Monday morning mistake.)