29 November, 2013

## OSX, VMWare, CentOS, and postfix

(Original title: “mail server arrrgh”)

I’m running VMWare Fusion on my Mac. I often run multiple VMs for testing Postgres on various OSes, and decided it would be fun if I could get system emails from the guests on the mac host. Yeah, “it would be fun”. Even though I was working with Postfix, this task gave me flashbacks to my sendmail experiences many years ago.

This is what I had to do to get it working. This isn’t in the order I did all the steps; there was a ton of trial, error, and wtfery that went on here. Also, this is on a machine that’s behind a few firewalls; probably not something you want to configure on a server that actually has a port open on the Internet. Although it seems that most of the mailserver-related footguns are no longer enabled by default. Caveat Emptor, or something.

Host: Mac OSX Lion 10.7.5
Guests: CentOS 6.4
VMWare: Fusion 4.1.3

Set up postfix on the mac host; starting with the instructions here: http://www.phase2technology.com/blog/how-to-enable-local-smtp-server-postfix-on-os-x-leopard/

This “worked” in that the simple “telnet to port 25″ test worked (see “Useful tools and commands” below), but I had to make some additional changes to receive mail from my guest OSes.

On the Mac host, I made some changes to /etc/postfix/main.cf:
myhostname = princess mydomain = localdomain mydestination = $myhostname,$myhostname.$mydomain, localhost.$mydomain, localhost, mailhost.$mydomain inet_interfaces = 192.168.247.1, 127.0.0.1 mynetworks = 192.168.247.0/24, 127.0.0.0/8 The value for inet_interfaces is the IP of my vmware interface, obtained from ifconfig. You want the vmware8 interface and it should be a 192.168 address. I just added the whole vmware subnet to mynetworks parameter. You can read more about these parameters in main.cf; it is surprisingly well-commented. I also commented out imap_submit_cred_file line as discussed here https://discussions.apple.com/thread/3247974. I’m not going to bother with this for local use. ‘postfix reload’ did not apply these changes for me; I had to explicitly stop and restart it1: postfix stop postfix start On the guest: First, I added the vmware host IP to /etc/hosts: 192.168.247.1 princess princess.localdomain mailhost mailhost.localdomain This worked via the ‘telnet to port 25′ test method, but regular email wasn’t getting delivered; in fact, it was bouncing. Looking into the mail queue, I discovered it was trying to use a completely different IP for its mailhost. I checked out the ‘hosts’ value in /etc/nsswitch.conf (just grep hosts /etc/nsswitch.conf) and determined that it was set to ‘files dns’, which ordinarily indicates that /etc/hosts should override dns from the vmware dns server. That wasn’t what was happening. A bit of investigation revealed that postfix was already running2, so I had to dink with it on the guests as well. Fixed by editing /etc/postfix/main.cf (on the guest) to include this: relayhost = mailhost.localdomain Booyah. Thanks to mjm for keeping me sane during this. Useful tools/commands: ‘postfix status’ # see what postfix thinks it’s doing and get its pid, since the processes aren’t named postfix and therefor ‘ps -ef | grep post’ won’t show them :) netstat -an | grep ‘\.25 ‘ # see if anything’s listening on port 25: gabrielle@princess-~/ (master *) :::-->netstat -an | grep '\.25 ' tcp4 0 0 192.168.247.1.25 *.* LISTEN tcp4 0 0 127.0.0.1.25 *.* LISTEN mailq # see status of queued mail messages [tail|vi|whatever] /var/log/mail.log # see connections, postfix startup/reload, etc SMTP test program, something like this http://www.port25.com/how-to-check-an-smtp-connection-with-a-manual-telnet-session-2 You may want this as well: http://topicdesk.com/faqs/os-x-server-mail-services-faq/69-how-do-i-completely-disable-postfix-and-cyrus 1 – I’ve since been told it’s better to use launchctl to control processes. 2 – This surprised me, given the other things that are locked down/not available on the CentOS default. 22 November, 2013 ## My Current “Most Useful” Android Apps I’ve only been using CamScanner for a month or so, but it’s already saved me at least an hour of dealing with a flatbed scanner. It’s very easy to photograph, crop, convert to B&W, convert to a .pdf and send them off to someone (or share via DropBox or some other method). I recommend doing the short tutorial to get a feel for what the different icons mean. The MultcoLib app was really slow when it was first released, but has improved dramatically. Search, hold, renew – all from your phone! Much-awaited by Portlanders, the TriMet Ticketer still has some quirks, not the least of which is that some bus drivers don’t seem to know what to do with electronic tickets. Still, it’s super-convenient if you’re stranded somewhere. As long as it wasn’t Tri-Met that stranded you there. I started using CardioTrainer to record my running workouts after my iPod nano died1. It really grew on me – I get a map of my route, it works for a variety of activities (running, mt biking, hiking, house cleaning2), it interfaces with a HRM, etc. Unfortunately, the only way to get access to your raw data (if you haven’t rooted your phone) is to upload it to the Noom website and use their export tool. This doesn’t work with my phone (HTC One), and the company has no plans for further development. So I’m looking for an alternative, because I really like being able to fold, spindle, and mutilate my data myself. etc Strava is completely unacceptable, for reasons I won’t go into here. The Android Ravelry app Ravulous is the only paid app I have right now; I think it’s a dollar or two, and totally worth it for those spur-of-the-moment stops into the LYS. It does have some quirks: - you can’t access your favorites - you can’t update projects with * in the notes (known issue, see the Ravelry forums) If you like to take a lot of WIP photos, I recommend getting the Ravelry Photo Uploader as a companion app. 1 – More about that in a later post. 2 – No, seriously. 8 November, 2013 ## nook Simple Touch 18-month review I’ve had my simple touch for well over a year now, and I’m still pretty happy with it, especially since I’m traveling a bunch. Updates on specific issues: - I haven’t had any further problems with mangled print; I suspect that was a feature of the particular material I was reading. - I never did get ebrary to work correctly, but my library doesn’t offer that anymore anyway, so it’s moot. - I said I would purchase one again (and they’re going for about$70 right now), but I’d rather get a kobo just for philosophical reasons. I like the “open ebook format” idea.

My library offers two e-lending options: OverDrive (the new incarnation of library2go) and 3M Cloud Library. They’re both now searchable from my library’s catalog and from the lender’s website, which makes finding stuff to read much simpler. I prefer OverDrive; downloads from the 3M Cloud Library are quirky and very slow (sometimes taking multiple hours), and the app crashes frequently. I haven’t tried running it on Windows yet; my problems may be due to using it with Wine.

Library2go used to provide a way to “return” ebooks before the lending period was up; no more. 3M doesn’t allow that either. If you are particularly voracious you will bump up against the 5 item limit in short order. Checked-out materials become unreadable on your device once the lending period is up. (Other e-lending solutions I’ve tried allow you to keep reading the book until you re-sync your device.) This left me stranded 30 pages from the end of a book I was really enjoying. Boo.

The nook itself has started getting a bit flaky: the touchscreen randomly doesn’t work, and sometimes the hard buttons cause it to skip ahead multiple pages. I cleaned it with a soft toothbrush & eyeglasses cloth, as discussed here, and that seems to have cleared it up[1]. If not, there’s always the option to take it apart and put it back together again.

One strange thing I’ve noticed is ADE shows a lot of “missing” books that don’t show up on a regular directory listing of the nook. They seem to be duplicate file listings; even if I delete them through ADE, they show back up next time I connect the nook. Still trying to figure that piece out.

1 – This is what happens when you take electronics camping.

Tags:
1 November, 2013

## Powershell.

Yes, here we are again, with me using a Windows machine. I can’t decide if Powershell makes having to use Windows tolerable, or just throws salt in the wounds. Powershell provides much more efficient methods of searching files and moving/renaming them than messing with Exploder, but every time I need it, I have to look up the syntax because it’s just not familiar.

Here are samples of the commands I use regularly, so they’re all in one place & I can easily C&P them from anywhere.

Find all .zip files:
Get-ChildItem -path c:\path\to\search -recurse -filter *zip

Order of the options is not important, and recurse can be shortened to rec.

Find a certain file somewhere on my hard drive:
Get-ChildItem -path c:\ -filter settings.xml -rec

I search file content a lot, so I made an alias for grep (also in my profile), because it’s easier for me to remember:
Set-Alias grep select-string

Find my notes about JSON, somewhere on my hard drive:
Get-ChildItem -path c:\ -inc *.txt -rec | grep -pattern "json"
…this is a case-insensitive search.

Convoluted way to move files (still looking for something easier):
Get-ChildItem -path c:\old\path -rec -filter *zip | foreach-object { copy-item -path $_.fulllname -destination c:\new\path } If your paths or filenames include spaces, you’ll have to quote them, of course. There is a way to diff files but I find the output nearly unusable. Additional tips: - You don’t have to type the commands in in camel case; powershell will transform it. - There is some tab-completion available. - I added this to my profile to save my history between sessions: https://lopsa.org/content/persistent-history-powershell. There’s no up/down arrow paging for commands from a previous session, though; you have to list the history items and then execute them from the menu. (With a command e.g. “i 2″. Yeah, that’s intuitive. Feels like the 80s in here.) And: <esc> for <ctrl>-u. Useful links: http://www.powershellatoms.com/desktop-management/creating-persistent-aliases-in-powershell/ http://blogs.technet.com/b/heyscriptingguy/archive/2012/02/27/use-powershell-to-copy-files-to-a-shared-drive.aspx Tags: 25 October, 2013 ## Manipulating .pdf files on Linux using Ghostscript I have to digitally fold, spindle, and mutilate .pdf documents frequently. On Ubuntu, I tried the GiMP, pdftops, pdftk, and some truly tortuous gymnastics involving screencaps, but none of them really did what I wanted. Then I found Ghostscript. It’s a command line tool, which I dig, because it means that I can type instead of having to point & click, and I can write quick shell scripts to do my dirty work. Here’s how I use it most often: Combine multiple .pdfs into a single file: gs -sDEVICE=pdfwrite \ -o 2012_final_report.pdf \ 2012-*_receipts.pdf  Pull first page only from multiple files: for each in ls 2012_Account_Statement_* do cp$each ${each}.backup gs -sDEVICE=pdfwrite \ -dFirstPage=1 -dLastPage=1 \ -o${each%.pdf}_firstpage.pdf \ ${each} done Combine multiple .pdfs and convert them to B&W: gs -sDEVICE=pdfwrite \ -sColorConversionStrategy=Gray \ -dProcessColorModel=/DeviceGray \ -dCompatibiltyLevel=1.4 \ -dAutoRotatePages=/None \ -o 2012_final_report.pdf \ 2012-*_receipts.pdf The Ghostscript Quick Start guide is here. Tags: , 18 October, 2013 ## Try these at home! We had one of those truly amazing meetings at PDXPUG this week. Along with the ideas that came out of this meeting (such as, leveraging Calagator for optimal scheduling of new user groups and this), Matt Smiley schooled a bunch of us on some basic unix utilities. Recorded here so I don’t forget them; these are version-dependent, YMMV. less: -S prevents line wrap, then you use the arrow keys to page through your output. This is super-handy when viewing wide, tabular output. top: - ctrl-m sorts by mem - s lets you choose the refresh rate sar: - await is the value to use for disk latency - svctime is not :) (it’s a calculated value instead of an actual measurement). The sar man page notes that this field is not to be trusted and will be removed in the future. iostat - collect ongoing stats: iostat -x -t -k 1 100 -x = extended stats -t = include timestamps -k = measurements in kB :) 1 = one second intervals 100 = 100X Your first (and possibly second) set of data collected from this can be thrown out, as it contains the cumulative stats since the system started. This also affects running a single timepoint. I also learned about a couple of monitoring tools I need to check out: saidar and Data Dog. 27 September, 2013 ## PgOpen recap Previous years: 2011 2012 PgOpen returned to Chicago for its 3rd year, and just keeps getting better and better. We moved the venue to Hotel Sax, a funky place with friendly staff and a pretty amazing catering service. (I’m just here for the food.) My talk picks: Schema migrations with Alembic (Selena Deckelmann) – inspired me to try out this tool. Using Postgres FDW (John Melesky) – I love the “here is what I did and I don’t really recommend it” talks. Inside Pg Shared Memory (Bruce Momjian) – Bruce’s talks are always thorough overviews and this one did not dissapoint. Replication Refresh (Simon Riggs) – basic discussion of the current state of Pg replication Secure your Webapp Passwords in Pg (Magnus Hagander) – sensible information delivered with humor. Links to slides for these and most of the remaining talks are on the wiki. If you presented and haven’t uploaded your slides yet, this is your cue. :) Of course, the best part of the conference is meeting new people, and people new to Postgres (hi, Catherine!). As we are a community-supported event, we like to contribute back to the local community as well. Big thanks to Steven Frost for running the auction again! This year, in addition to funds generated from the auction, other members of the Pg community chose to donate cash on the spot. The conference committee matched some contributions, and we raised over$3500 for Chicago Hopes! Thank you.

I’m really excited to be on the committee again next year! Planning starts soon for PgOpen 2014, so if you haven’t already, please fill out the survey.

5 July, 2013

## postgresql.conf: rudimentary log settings, annotated

One of the first changes I make in a new Postgres install is to enable activity logging. This way, if I encounter some problems down the the road, I’ve already got some data collected that may help me troubleshoot.

Here are my basic settings:

# enable logging, of course: log_destination = 'stderr' logging_collector = on

 # target directory, on its own mount point if you can do it log_directory = '/pg_log' # I just use the default here log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log' # log_line_prefix in pgbadger-compatible format # timestamp [pid]: line_number user application remote_host_&_port database % log_line_prefix = '%t [%p]: [%l-1] %u %a %r %d %%' # so I know when people are tinkering log_statement = 'ddl' # pre-emptive troubleshooting items log_checkpoints = on # log all statements that take longer than 10 minutes # YMMV here - adjust for your application log_min_duration_statement = 10min # keep track of longer-running autovac jobs; may indicate that I need to tweak my autovacuum settings # again, YMMV log_autovacuum_min_duration = 10min # again, I want to keep track of these; can be a harbinger of problems # you may need to adjust your deadlock_timeout log_lock_waits = on 

# translates an IP to a hostname. # Useful, but can cause problems on high-activity systems, so use with caution. log_hostname = on

What are your favorite log settings?

28 June, 2013

## Postgres 9.3 – my picks

There’s a lot to be excited about in the upcoming 9.3 release. Here are my top 3:

3. -R option for pgbasebackup creates a recovery.conf file for you! One less step to remember.

2. \watch command for psql! Hot dog.

1. The human-readable WAL file export is possibly the best thing since sliced bread.

17 May, 2013

## Fiat 500

Last year I started a job as a consultant. Of course, this means traveling a bit. I rent cars from Enterprise a lot, and am a member of their “Enterprise Plus” club. I’ve racked up a lot of points, but haven’t actually tried to redeem them yet; as of yet the only advantage to being a Plus member is getting to go to the head of the line at the rental counter. Which I always feel guilty about.

I almost always get a compact or “econobox” and occasionally get upgraded. The conversation usually goes like this: they ask if I have a preference of vehicles 1, and I say “I don’t really care, just as long as it’s not a Nissan Versa”2, and then we pick something out. This week the sales associate said “Well, what about this Fiat?” and stopped next to this cute little red thing. Uh sure, twist my arm!

Car: Fiat 500, fire-engine red.

Fuel economy: Meh. Manufacturer claims 34-40 highway, I got on the low end of that3.

Handling: The tight turning radius is a big plus; I could flip a U on a two-lane road. It got a bit squirrelly at high speeds, though.

Conversation starter: A+. People wanted to chat at stoplights, or while they were turning through an intersection (!!!), and one time, I came out of the grocery store to find someone waiting by the car because he wanted to ask me about it.

Appearance: Cute, but not as cute as a Mini Countryman. The red is a really nice deep red, though.

Comfort: Very! 3 hours straight and no back pain or tingly feet. The back seat is probably limited to children or very small people, though I imagine it’s motion-sicknessville back there. Bonus: I could drive with one window down without that annoying air thumping, up to about 75mph.

Trunk space: What?

Annoyances, allow me to list them:
1. The dashboard layout. Seriously: who designed this?! The automatic windows are controlled by two separate switches, and you have to reach over the gearshift to work the passenger side window. May as well have manual cranks if you’re going to do that. (In this car, the distance isn’t that much farther.)

2. No auto-up on the windows. See “reacharound” above.

3. Despite the additional fisheye mirror, you could hide a minivan in the driver’s side blind spot.

4. Can’t pop the hatchback from in the car. Or maybe you can, but I never found the lever (see comment about dashboard layout above.). You have to hit the button on the key, then manually open the door. Then, the door doesn’t have a handle or anything to grab to pull it down, just a little teeny strap.

Moment of hilarity: Driving along Illinois 288 and hitting the perfect frequency on the concrete seams to make the car bounce so much I thought it might take off. I laughed so hard I almost had to pull over.

If I bought this car, the first thing I would do is replace the stock tires with something that grips a bit better; it was kinda scary in some of the turns.

1 – So far my favorite car is the Toyota Yaris. (I know, I’m surprised too!)
2 – The less said about that travesty of a vehicle, the better. Bleah.
3 – For reference, my personal car is a TDI Beetle. I get 50-60mpg on the highway, if I don’t have bikes on the top; then I get 40-45. So compared to that, everything else is going to be “meh” or worse.