Upgrade Ubuntu 15.04 to 16.04

I use Ubuntu on my home machines, and tend to rely on the GUI Software Updater for upgrades & such.  Life happens and I left one of them on 15.04 past EOL, and the Software Updater didn’t provide an upgrade path directly to 16.04.  I had to upgrade to 15.10 first, which is also EOL.

Here’s what I pieced together from various forums around the net.  (I took this after-the-fact from my command-line history;  I’ll take better notes next time.)

apt-get update
apt-get upgrade # make sure system is as up-to-date as possible
apt-get autoremove # clean up some leftover crap

apt-get dist-upgrade # recommended next step on one of the forums,
                     # but didn't actually do anything
                     # and I probably could have skipped this

do-release-upgrade -d # supposely this should have worked without the -d
                      # but it didn't
                      # -d means "latest development release"

And booyah, I’m on 15.10 and could continue on to 16.04.

 

This month’s RDS tip, a pgloader exercise, and bonus recruiter spam

1. Several months back, AWS re-worked the web console. You can get sparkline-style graphs for CPU usage, memory, storage, and database connections at the top level, instead of having to drill down to the Cloudwatch metrics for the instance.

I find this really handy – but for one quirk with the graphs:

There’s white space to the right of the red line. Therefore, I was interpreting the red line to mean “trouble is brewing & I should do something about this”.

Turns out that red line is the limit; there’s nowhere else to go. Whoops!

2. This week I finally had a reason to try out pgloader: One of my analysts needed some help loading some ugly fixed-width data¹.

Installing was super-easy on my mac (`brew install pgloader`).  I worked through the examples before starting to work with my actual dataset².

The data to be loaded came bundled as several text files: the data, plus three or four additional files describing the layout.  I wrote a truly glorious string of cut, sed, paste, and awk to create a pgloader control file that would work. And it did!

Except:

field1 | field2 | field3 
--------+--------+----------
 [null] | D | IMITRI
 [null] | G | ABRIELLE
 [null] | M | ARK
 [null] | S | ELENA

Uh, what?

The data descriptions had character counts starting at 1, and pgloader expects them to start at 0 (as they should).  (For extra fun, the first column in all records of this dataset was nothing but spaces.)

3. This week’s hilarious recruiter spam:
“Hey there Gabrielle! I was doing my homework on sites like Meetup, GitHub, etc. and I noticed your Java skills.”

I’m pretty sure you didn’t.

—

1 – Is that redundant?
2 – 600 columns of fixed-width data?  Who does that?!
3 – Why couldn’t it all be in one file? Again: who does this?!⁴
4 – People who hate DBAs, that’s who.

Try these at home!

We had one of those truly amazing meetings at PDXPUG this week. Along with the ideas that came out of this meeting (such as, leveraging Calagator for optimal scheduling of new user groups and this), Matt Smiley schooled a bunch of us on some basic unix utilities. Recorded here so I don’t forget them; these are version-dependent, YMMV.

less:
-S prevents line wrap, then you use the arrow keys to page through your output. This is super-handy when viewing wide, tabular output.

top:
– ctrl-m sorts by mem
– s lets you choose the refresh rate

sar:
– await is the value to use for disk latency
– svctime is not :) (it’s a calculated value instead of an actual measurement). The sar man page notes that this field is not to be trusted and will be removed in the future.

iostat
– collect ongoing stats: iostat -x -t -k 1 100
-x = extended stats
-t = include timestamps
-k = measurements in kB :)
1 = one second intervals
100 = 100X

Your first (and possibly second) set of data collected from this can be thrown out, as it contains the cumulative stats since the system started. This also affects running a single timepoint.

I also learned about a couple of monitoring tools I need to check out: saidar and Data Dog.

Tuesday Tidbit – JOIN on multiple fields

I use this so rarely I have a hard time remembering the syntax; I always try to use a comma instead of AND. Which, of course, throws an error.

Looks like this:
SELECT [stuff]
FROM table1 t1
JOIN table2 t2 ON
(t1.field1 = t2.field1 AND t1.field2 = t2.field2);

Net::Ping permissions

While Net::Ping::External neatly maneuvers around Net::Ping‘s root access requirement, sometimes you want to use a feature of Net::Ping that Net::Ping::External doesn’t provide, such as getting ping response time. (Net::Ping::External is a bit slower; timing it doesn’t provide adequately accurate data for me.)

You can ask your sysadmin to allow you sudo access to a script that uses Net::Ping, but that sometimes leads to file ownership issues.

There is another option available, provided:
1. You are using Solaris10
2. You have a pretty good relationship with your sysadmin.

Request that your sysadmin add net_icmpaccess to your entry in /etc/user_attr, as described here.

Log out & log back in to activate your new perms, and voila.

vim tidbits of the day

:::–>vim

:grep [regexp] [file list]

vim will then load the files that match the regexp & position the cursor on the matching line.

use :cn and :cp to move between instances of the match.

patch

(This is on solaris, no -N option to diff for me!)

This produces the easiest-to-read diff IMO:
diff -btu [oldfile] [newfile] > patchfile
-b = ignore blanks
-t = preserve source indentation
-u = 3 lines of context with the + and – in front of changed lines

But the -t option interferes with proper patch application, so just use:
diff -u [oldfile] [newfile] > patchfile
…where oldfile is the file you want to patch, and newfile is the file with the changes you want to apply.

patch -b -p0 < patchfile
-b = make a backup ;)

If you are asked for the filename to patch – you probably don’t have the depth set right with -p.  Try increasing it.

If you get the message “Reversed (or previously applied) patch detected!”, somebody messed with your stuff between creating the patch & applying it, OR you did the diff backwards (common Monday morning mistake.)