handyfloss

In UNIX-like environments, such as GNU/Linux, command line is often used to operate on a bunch of files, such as:

rm -f *.dat

In the command above, “*.dat” is expanded by the shell (the command interpreter), to all matching files in the present directory (e.g.: “file1.dat file2.dat dont_delete_me.dat this_file_is_rubbish.dat“). However, this expansion is performed as a first step, and then the expanded command line is executed, e.g.:

rm -f file1.dat file2.dat dont_delete_me.dat this_file_is_rubbish.dat

This behaviour can potentially fail if a lot of files match the *.dat, because there is an upper limit to how wide a command line can be (brutally high, but finite). This can happen, for example, if you try to delete the contents of a directory with 100,000 files, and use rm -f * (yes, this can happen). For example, a ls in a directory with 100,000 files works fine, but an rm * does not:

Bart[~/kk]: rm *
/bin/rm: Argument list too long.

To avoid this problem, we can make use of xargs and echo (since echo does not seem to suffer from this argument number limitation), in the following way:

echo * | xargs rm -f

Now, xargs takes care of the argument list, and processes it so that rm does not see an endless argument list.

xargs can also be given other uses. For example, consider the following example: We want to convert some OGG files to MP2 (I won’t be caught dead near an MP3, due to its patent issues), so that a Windows luser can hear them. We can use oggdec to convert OGG to WAV, then toolame to convert WAV to MP2. Now, oggdec accepts multiple arguments as input files to convert, so the following works:

oggdec *.ogg

The above generates a .wav for each .ogg in the argument list. However, toolame does not work like that; it expects a single argument, or, if it finds two, the second one is interpreted as the desired output file, so the following fails (too many arguments):

toolame *.wav

This is where xargs can come in handy, with its -n option (see the xargs man page). This option tells xargs how many arguments to process at the same time. If we want each single argument to be processed separately, the following will do the trick:

echo *.wav | xargs -n 1 toolame

In the above example, toolame is called once per WAV file, and sees a single WAV file as argument each time, so it generates a .mp2 file per .wav file, as we wanted.