In UNIX-like environments, such as GNU/Linux, command line is often used to operate on a bunch of files, such as:
rm -f *.dat
In the command above, “*.dat” is expanded by the shell (the command interpreter), to all matching files in the present directory (e.g.: “file1.dat file2.dat dont_delete_me.dat this_file_is_rubbish.dat
“). However, this expansion is performed as a first step, and then the expanded command line is executed, e.g.:
rm -f file1.dat file2.dat dont_delete_me.dat this_file_is_rubbish.dat
This behaviour can potentially fail if a lot of files match the *.dat
, because there is an upper limit to how wide a command line can be (brutally high, but finite). This can happen, for example, if you try to delete the contents of a directory with 100,000 files, and use rm -f *
(yes, this can happen). For example, a ls
in a directory with 100,000 files works fine, but an rm *
does not:
Bart[~/kk]: rm *
/bin/rm: Argument list too long.
To avoid this problem, we can make use of xargs
and echo
(since echo
does not seem to suffer from this argument number limitation), in the following way:
echo * | xargs rm -f
Now, xargs
takes care of the argument list, and processes it so that rm
does not see an endless argument list.
xargs
can also be given other uses. For example, consider the following example: We want to convert some OGG files to MP2 (I won’t be caught dead near an MP3, due to its patent issues), so that a Windows luser can hear them. We can use oggdec
to convert OGG to WAV, then toolame
to convert WAV to MP2. Now, oggdec
accepts multiple arguments as input files to convert, so the following works:
oggdec *.ogg
The above generates a .wav for each .ogg in the argument list. However, toolame
does not work like that; it expects a single argument, or, if it finds two, the second one is interpreted as the desired output file, so the following fails (too many arguments):
toolame *.wav
This is where xargs
can come in handy, with its -n
option (see the xargs man page). This option tells xargs
how many arguments to process at the same time. If we want each single argument to be processed separately, the following will do the trick:
echo *.wav | xargs -n 1 toolame
In the above example, toolame
is called once per WAV file, and sees a single WAV file as argument each time, so it generates a .mp2 file per .wav file, as we wanted.