Archive for the ‘:: *nix ::’ Category

Tar and Gzip When Low On Disk Space

Friday, March 12th, 2010

Anyone who knows me knows how I feel about AIX :-P.  One of my peeves is that out of the box, I cannot tar/gzip a directory with the “tar -cvz” command without breaking it up into two separate tar/gzip commands.

Every once in a while, you need to pull a large directory off your server to your local machine or just to copy it to another server.  Sometimes it’s easy just to copy the entire directory uncompressed.  Other times, it just makes sense to tar and gzip the directory so that the file that you are transferring is in a single archive and is compressed as to shorten the transfer time.

The problem with the tar/gzip option is that if you tar then gzip, you need the disk space to do so.

On AIX, here is a solution that will allow you to send what you tar to stdout then gzip it on stdin with no problems with disk space unless the gzip’d tar is still bigger than the diskspace you have available.  This *should* work on other *NIX systems that you cannot throw the gzip switch in the tar command.

tar -cvf - /path/to/dir/you/want/to/tar | gzip > /destination/for/gzip'd/tar.tgz

How To: Get Your Mac Serial Number From Terminal

Sunday, January 24th, 2010
ioreg -l | grep IOPlatformSerialNumber

Run A Command Every N Seconds

Thursday, January 21st, 2010

Short story long…

This week, Apple officially release support for Windows 7 in BootCamp 3.1.  I thought to myself “Self, you have a wonderfully powerful Mac Pro with 8 cores and 10GB RAM that is being completely unused by your XP install, you really should upgrade to to Win7 x64 now that Apple officially supports it”.

So, that’s what I did, with no thought as to how much MORE disk space I would need on my BootCamp partition.

So after installing Windows 7 and a bunch of games, I received my you only have < 500MB of disk space left alert.

This brought me to a point where I needed to resize my OSX HFS partition and grow my Windows NTFS partition.  During this madness, I realized that with BootCamp 3.1, I could view my HFS partitions from Windows!  But much to my chagrin, I couldn’t see my slice that had all of my useful data because it was a part of a concatenated RAID.

After resizing my OS partitions, I decided to move data off my RAID slice so I could break it then just create a normal HFS partition that can be viewed from Windows.  The problem is, moving almost a TB of data can take a long time and turning from my laptop to my left 45 degrees to my desktop on my right every time I wanted to see how my disk space looked left me with a pain in my neck!

I was at that very moment reminded about using while & sleep. Below is a handy little command that will run any command every number of seconds until you break (ctrl-c) it.

while true
do
command
sleep 30 (seconds)
done

I personally ran:

while true
do
df -h | grep disk1s2
sleep 30
done

Which allowed me to monitor the growth of the destination disk I was copying to.

-boogybren

Grep For More Than One Word Recursively

Tuesday, January 19th, 2010

I have an application that runs on AIX that is nothing short of ugly when it comes to pointing it to a different datasource.  This application currently uses an outdated version of Savvion BPM that requires manual editing of any config file that contains the datasource and/or credentials.  To complicate things further, our application which runs in WebSphere also has some config files that need manual updating.

Today…I decided to try and document the process as to save me and anyone else who comes behind me some serious grief.

Because AIX’s version of grep doesn’t have a -R option, I will give you syntax that *should* work on any flavor of UNIX/Linux with grep.

find /path/to/parent/directory -type f -print | xargs grep -i -E 'word1|credential1|server1|server2|server3' > /path/to/output/file

Let’s dissect this for clarification:

find /path/to/parent/directory -type f -print

We are telling the find command where we want grep to search.  You can either specify the explicit path for example, I searched under /opt, or you can change directory to the parent directory then issue a “.” (sans double quotes) to tell find to “search from here”.

The -type f simply tells find to locate regular text files, not binary files.

The -print is important.  It will print the full path to the file.  The reason this is important is because when grep finds a match in one of the files, it will print the full path out in addition to the line it found within the file so you know exactly where the file is located for future reference.

xargs

xargs simply is a command that allows you to issue any command (grep in our case) against the list of standard-in (list of regular files) one at a time.  As if you did it one at a time manually.

grep -i -E 'word1|credential1|server1|server2|server3'

grep is a command that allows you to search the contents of a file for a specific pattern.  The -i tells grep that this search is case insensitive.  The -E tells grep that we want to use regular expressions in our search.

We then follow with a basic regular expression.  Our search is in single quotes and each word we want to search for is separated by the pipe symbol.  We are just asking grep to find any word that matches in the list.

> /path/to/output/file

Here  we are just redirecting the results of our search to a file so that we can review and modify as we deem necessary.  Remember that if you already have data in the destination file, a > will overwrite the contents with the new data found.  A >> will just append to the file.

Once this command has finished, you will be able to edit the file and remove any false positives that grep might have come across.

Some what dirty, but a simple solution.

HOWTO: Find the Top 50 Largest Files in Linux/Unix

Thursday, November 12th, 2009

This should be linux/unix OS agnostic.

sudo find . -type f -exec du -m {} \; | sort -nr | head -50

If you need more or less files, adjust the last digit following ‘head’.

-boogybren

grep -R Alternative

Thursday, July 2nd, 2009

Ugh…

I have worked with unix/linux now going on my twelfth year.  However, I am still pretty green with AIX as I have only dabbled in it for about two years now.

Although I have grown to enjoy their implementation of logical partitions and other value adds, I have always felt that their OS is old and stuffy.  I was reminded of this recently when I could not perform a recursive grep.  There are a lot of archaic command line switches in this stodgy old OS, not to mention all of the new and cool commands that are missing.

If you find yourself needing to do a grep -R in AIX or in any other *nix OS, here is an alternative command for you using find:

find . -type f | xargs grep -i <string>

For example, if you wanted to find any file that contains the word “voova” in your current working directory and all of the child directories, you would issue:

find . -type f | xargs grep -i voova
  • find . -type f tells the command to locate all files
  • xargs allows for you to execute whatever command follows (in this case it is grep) on each individual file that the find command locates as if you grep’d each file one at a time
  • grep searches for the string where the -i makes the search case insensitive.

Hopefully, this will save you the exercise of RTFM’s

-boogybren