jump to navigation

The power of the UNIX command line – part I January 9, 2009

Posted by haskelladdict in Linux.
Tags: , , , ,
add a comment

Quite frequently, I notice that particularly among Unix newcomers the power of the command line seems to be somewhat forgotten. This is unfortunate, because often one can accomplish amazing things in just one or two lines of shell code. Admittedly, using the command line takes a bit of getting used to, but then it’s just pure fun.

In the following few installations I’ll give a few examples of how I leverage the command line to make my life easier. If this wets your appetite, you owe it to yourself to check out “Unix Power Tools” [1] and “The Bash Cookbook” [2].

Let’s get started. Recently, I faced the following task: I had a bunch of subdirectories that each contained some simulation data files. The directories were – quite inconveniently – named iteration_50001, iteration_50002, etc. all the way up to iteration_75000.

Unfortunately, in order to feed them into the next step of some modeling pipeline they needed to be named iteration_1, iteration_2, ..

Bummer! I guess most people (including myself) wouldn’t have hesitated to throw a few manual “mv iteration_50001 iteration_1”, “mv iteration_50002 iteration_2”, etc. at their keyboard if there would have been no more than 10 files or so. But 25,000 of them! Even assuming an average of 1 s for producing each of the mv commands, this would still take about 6 hours to complete. What else could be done? Well, how about this:

#!/bin/bash
for ((new_count = 1; new_count <= 25000; new_count++)); do
   ((current_count = new_count + 50000))
   mv iteration_${current_count} iteration_${new_count}
done

This small bash script does it all and completes in just a few seconds. Let’s briefly go over it. The complete program is one big loop, incrementing the new_count variable starting at 1 all the way up to 25,000. In the second line we specify the loop condition using bash’s arithmetic expansion operator (the ((…)) expression) just like we’d do it in C or C++. The new_count variable will count our (new) target directories, i.e., using bash’s parameter substitution they will be called iteration_${new_count}. Next, we need to compute the counter for the directory to be moved. Luckily, this simply amounts to adding 50000 to new_count which is what line 3 is all about, again using bash’s arithmetic expansion operator. The name of the directory to be moved then is iteration_${current_count}. In line 4 we simply move the present directory to the new one via the mv command. That’s it!

Clearly, you can use a loop of this kind for many other purposes potentially saving you a lot of work and grief.

[1] http://oreilly.com/catalog/9780596003302/
[2] http://www.bashcookbook.com/