Posts tagged ‘linux’

2 February, 2015

Updating My Linux Command line Toolbox, episode 3

by gorthx

Part 2

This week’s tips:

1. ulimit -a will show you all settings, plus the units.

2. crontab -l -u [user] will read out another user’s crontab for you (assuming you have the right perms)

3. and what I call “diff-on-the-fly” – pass the output of shell commands to diff. I like this one because I don’t make a bunch of “temporary” files that I forget to clean up later.

diff <([shell commands]) <([other shell commands])

For example, I need to compare ids in two files, but they’re in different fields in each file, and not in the same order:

diff <(cut -d"," -f1 file1 | sort -u) <(cut -d"," -f3 file2 | sort -u)

Advertisements
8 September, 2014

Updating My Linux Command line Toolbox, episode 2

by gorthx

Part 1

Five more, all from this week:

1. date -u to get your date in UTC

2. pushd and popd – create your own directory stack.  I’m still trying this one out.  (“why not just use the up arrow?”)

3. pbcopy – copy to clipboard from the command line.

4. !$ contains the last arg of the previous command, so you can do something like this:
ls -l filename.*    # check what you have in the dir
vi !$

5. This one is my favorite: !?[string] runs last command that contains that string.

Tags: ,
25 October, 2013

Manipulating .pdf files on Linux using Ghostscript

by gorthx

I have to digitally fold, spindle, and mutilate .pdf documents frequently. On Ubuntu, I tried the GiMP, pdftops, pdftk, and some truly tortuous gymnastics involving screencaps, but none of them really did what I wanted.

Then I found Ghostscript.

It’s a command line tool, which I dig, because it means that I can type instead of having to point & click, and I can write quick shell scripts to do my dirty work.

Here’s how I use it most often:

Combine multiple .pdfs into a single file:
gs -sDEVICE=pdfwrite \
-o 2012_final_report.pdf \
2012-*_receipts.pdf

Pull first page only from multiple files:
for each in `ls 2012_Account_Statement_*`
do
cp $each ${each}.backup
gs -sDEVICE=pdfwrite \
-dFirstPage=1 -dLastPage=1 \
-o ${each%.pdf}_firstpage.pdf \
${each}
done

Combine multiple .pdfs and convert them to B&W:
gs -sDEVICE=pdfwrite \
-sColorConversionStrategy=Gray \
-dProcessColorModel=/DeviceGray \
-dCompatibiltyLevel=1.4 \
-dAutoRotatePages=/None \
-o 2012_final_report.pdf \
2012-*_receipts.pdf

The Ghostscript Quick Start guide is here.

Tags: ,
15 March, 2013

Monitoring Tools: sar

by gorthx

What: sar
What it monitors: pretty much every system stat you can imagine (and some you haven’t)
Where to get it: it’s probably pre-installed on your system; if not, try the sysstats package (the same one that includes iostats)
Why you’d want to use it:

  • you need an answer fast, but maybe don’t have access to the “enterprise” monitoring (or there isn’t any…[1])
  • you’re doing system testing and want a command-line tool that’s easy to configure and run in discrete timeframes.

Why you wouldn’t want to use it:

  • you want data you can easily throw into a graphing or analysis program; the data produced by sar isn’t readily machine-readable
  • you’re looking for a near-real-time long-term monitoring solution. In that case, just go ahead and set up munin or collectd.

Because it’s lightweight and so readily available, it’s a good tool to have in your toolbox. Plus, it’ll tell you things like fan speed and temperature, and I’m just a sucker for environmental monitoring [2].

read more »

8 March, 2013

Updating My Linux Command line Toolbox

by gorthx

Over the past few months I’ve been working with some people with many more years of unix-y experience than I have. They’ve been teaching me new stuff, and showing me updated versions of commands I’ve been using in sometimes kludgy ways. Here are 10 examples.

1. join, which feels kind of like a stoneage tool; it’s not that versatile, and I’m sure there are perl one-liners that could do the same thing. But join’s readily available and it’s a fast way to join two files, if they meet the requirements: same file separator, a “key” field (which is easy to add with nl, #2 below), etc.

2. nl to number the lines in a file.

My most-used switches:
nl -v 800 -w 3 -n rn -s, oldfile > newfile
-v = start with this number
-w = number of characters (in this example, numbers > 999 will be truncated)
-n rn = ‘right justified, no 0 padding’ so you don’t have to go back through another round of text processing to strip them off
-s, = use a comma as a field sep

3. truncate as a desperate move to free up some space. Bonus: do this on a log file to confuse your coworkers.

4. watch [1]
Current favorite:
watch -n 10 “psql -d my_db -c \”SELECT datname, procpid, usename, backend_start, xact_start, query_start, waiting, current_query FROM pg_stat_activity WHERE current_query LIKE ‘autovacuum:%’\””

5. lastlog, last, and lastb
most recent login, all logins, and all bad logins

6. df -h instead of df -k
df -k was one of the first unix commands I learned. It used to be easy to read the blocks, Used, and Available columns (they had fewer digits back then). df -h makes it easy again, and shows me the units. The muscle memory on this one has been particularly hard to shed.

7. echo * | wc -w instead of ls | wc -l or ls -1 | wc -l
The first sysadmin I ever worked with taught me ls -1 | wc -l. Turns out you don’t need the -1 to list the files individually, if you are piping the output to another command.

Use echo * to quickly list the number of files in a directory, when that number is more than a few thousand; assumes no spaces in the filenames.

8. awk can be kind of intimidating due to its sheer power and uninformative error messages. I have a small cheatsheet I keep handy, for tasks I do frequently. These are the most recent additions:

To remove everything owned by me in a directory:
ls -l | awk ‘/gabrielle/{print $9}’ | xargs rm

To find all files > 0 size in a directory:
ls -l | awk ‘$5>0{print}’

9. Everything here: https://dougvitale.wordpress.com/2011/12/21/deprecated-linux-networking-commands-and-their-replacements/

10. And sar, which I’ll write a separate post about.

Thanks to Joe, Vibhor, Oscar, and of course MJM.

1 – “How did I not know about ‘watch’?” “You were using Solaris all these years, m’dear.”

Tags: ,