Dynamic file read with Perl
July 6th 2006

GNU/Linux command-line users, programmers and hackers worldwide have probably come to know and love the wonderful tail shell command, together with cat, head, grep, awk and sed, easily one of the single most usefull commands.

A killer feature of tail is the -f (--follow) argument, which outputs the last lines of a file and then keeps waiting for new lines that might keep appearing in the file, and show them on the screen when they do. This is invaluable to keep track of, e.g., logfiles where new entries are being added all the time, and one does not want to be doing a tail by hand.

Since I am a great fan of Perl, and use its scripts for anything short of cooking dinner (but wait...), I have found myself in situations where I had to tail the last lines of a file. This can be done in several ways:

system "tail $file";


my $str = `tail $file`;
print $str;

or with a open() statement, then reading the whole file (or a part), and printing it. The first example with system is the most "direct" one, but reading the file (or a part) into a variable is very handy for doing with it all the nifty things Perl does so well to text strings (substituting, deleting, including, reordering, comparing...).

However, when tail -f was needed (i.e., keep on tracking the file and operate on the output as it appears), I kept using system calls, and all the formatting had to be done in the shell spawned by the system call, not by Perl. This was sad.

So, I was so happy when I discovered a simple trick to make open() read dynamically. There are better ways of doing it, more efficiently and correctly, but this one works, and is quite simple. If efficience is vital for you, this is not probably the place to learn about it. Actually, if you look for efficiency, you shouldn't be using Perl at all :^)

Example of Perl code that reads dynamically a file "$in":

open(INFILE,"tail -0f $in |") || die "Failed!\n";
while(my $line = <INFILE>)
  do whatever to $line;

Update: Explanation to the code above:

The open() call pipes the output of the tail command (notice the -f flag. Do a man tail to know more) to the file tag "INFILE". The "||" sign is an OR, and means "do the thing on my right side if the thing on my left didn't end successfully (but ONLY in that case!)".

Next, we perform a while loop over the lines in the pipe. The "<INLINE>" construct extracts elements in INLINE, treating it as an array. As you can see, these elements are assigned to a new variable $line, and the loop continues while $line has some non-false value, i.e. while there are lines in INFILE.

The paragraph inside the curled keys is pseudocode, obviously; you put there your code. And, for tidiness, once we exit the loop, and INFILE is exhausted of lines, we close it.

Tags: , , ,


4 Responses to “Dynamic file read with Perl”

  1. Dynamic file read with Perl « handyfloss on 24 Oct 2008 at 13:10 pm #

    [...] Entry available at: http://handyfloss.net/2006.07/dynamic-file-read-with-perl/ [...]

  2. Rob Harrigan on 12 Jan 2009 at 23:42 pm #

    You sure isn't supposed to be INFILE?

  3. Rob Harrigan on 12 Jan 2009 at 23:43 pm #

    Last comment was botched becasue of the tags, sorry.
    You sure INLINE isn’t supposed to be INFILE?

  4. isilanes on 13 Jan 2009 at 11:05 am #

    Thanks, Rob! It is a typo. Fixing it in 1 sec.

Trackback URI | Comments RSS

Leave a Reply

Subscribe without commenting

« | »

  • The contents of this blog are under a Creative Commons License.

    Creative Commons License

  • Meta