Read entire file then print when editing inplace?

Question

Most examples of inplace editing are one-liners that iterate through a file or files, reading and printing one line at a time.

I can't find any examples of reading an entire file into an array, modifying the array as needed, and then printing the array while using the ^I switch to do an inplace edit. When I try to read the entire file from the diamond operator, edit the contents and print the entire contents, I find that the print goes to STDOUT instead of ARGVOUT and that ARGVOUT is closed. I can open the same file for output and then print to it, but I'm not sure I understand why that is necessary. Here is an example:

#!/usr/bin/perl
use strict;
use warnings;
use 5.010;

my $filename = 'test.txt';

push @ARGV, $filename;

$^I = ".bk";

my @file = <>; #Read all records into array
chomp @file;
push @file, qw(add a few more lines);

print join "\n", @file; #This prints to STDOUT, and ARGVOUT is closed. Why?

Running the above makes a backup of the test.txt file as expected, but leaves the edited test.txt empty, printing the edited contents to STDOUT instead.

The reason why you can't find any examples is because it's generally considered bad practice in Perl to read in the entire file, only to do line-by-line processing. :) There are many better ways to handle the reading. See answers below for a few specific reasons why. — Robert P, Feb 02 '11 at 21:39
Sorry @Robert P, but there are many line processing tasks where it is easiest to load all of the lines first. What if you want to remove the line in the exact middle of a file? Remove a line containing a pattern that is between 700 and 750 lines *before* a line containing another pattern? Process the input after sorting it, and then removing some lines at the top and/or bottom before printing? — mob, Feb 02 '11 at 23:04
Great answers. Both @mob's and @ephemient's do exactly what I wanted, so a toss-up really, which to accept. — d5e5, Feb 03 '11 at 19:15
@mob: Like I said, *in general* it's better to do line by line processing in Perl. I didn't say it couldn't, or shouldn't be done at all (although, the first two tasks you suggest could still be handled with line-by-line processing, if you had a second read file hanlde!) :-) — Robert P, Feb 03 '11 at 21:53

mob · Accepted Answer · 2011-02-02T21:37:43.467

See perlrun.

When the -i switch has been invoked, perl starts the program using ARGVOUT as the default file handle instead of STDOUT. If there are multiple input files, then every time the <> or <ARGV> or readline(ARGV) operation finishes with one of the input files, it closes ARGVOUT and reopens it to write to the next output file name.

Once all the input from <> is exhausted (when there are no more files to process), perl closes ARGVOUT and restores STDOUT as the default file handle again. Or as perlrun says

#!/usr/bin/perl -pi.orig
s/foo/bar/;

is equivalent to

#!/usr/bin/perl
$extension = '.orig';
LINE: while (<>) {
    if ($ARGV ne $oldargv) {
        if ($extension !~ /\*/) {
            $backup = $ARGV . $extension;
        }
        else {
            ($backup = $extension) =~ s/\*/$ARGV/g;
        }
        rename($ARGV, $backup);
        open(ARGVOUT, ">$ARGV");
        select(ARGVOUT);
        $oldargv = $ARGV;
    }
    s/foo/bar/;
}
continue {
    print;  # this prints to original filename
}
select(STDOUT);

Once you say my @file = <> and consume all the input, Perl closes the filehandle to the backup files and starts directing output to STDOUT again.

The workaround, I think, is to call <> in scalar context and check eof(ARGV) after each line. When eof(ARGV)=1, you have read the last line in that file and you get one chance to print before you call <> again:

my @file = ();
while (<>) {
    push @file, $_;
    if (eof(ARGV)) {
        # done reading current file
        @processed_file = &do_something_with(@file);
        # last chance to print before ARGVOUT gets reset
        print @processed_file;
        @file = ();
    }
}

score 3 · Answer 2 · answered Feb 02 '11 at 21:28

my @file = <>; #Read all records into array

is bad. Now you're done slurping all the records, *ARGV is closed, and $^I replacement doesn't have anything to work on.

my @file;
while (<>) {
    push @file, $_;
}
continue {
    if (eof ARGV) {
        chomp @file;
        push @file, qw(add a few more lines);
        print join "\n", @file;
        @file = ();
    }
}

This read the file(s) line-at-a-time, and at the end of each file (before it's closed), performs the manipulation.

undef $/;
while (<>) {
    my @file = split /\n/, $_, -1;
    push @file, qw(add a few more lines);
    print join "\n", @file;
}

This reads entire files at a time as single records.

The continue statement wasn't in my vocabulary, but it is now. Also, next time I want to slurp it all into one string and do a global substitution while editing inplace, I'll know how. Thanks. — d5e5, Feb 03 '11 at 19:08

toolic · Answer 3 · 2011-02-02T21:31:03.510

2

Tie::File can also be used to edit a file in-place. It does not leave a backup copy of the original file, however.

use warnings;
use strict;
use Tie::File;

my $filename = 'test.txt';
tie my @lines, 'Tie::File', $filename or die $!;
push @lines, qw(add a few more lines);
untie @lines;

edited Feb 02 '11 at 21:31

answered Feb 02 '11 at 21:24

toolic

57,801
17
75
117

This could come in handy too, for when I don't need or already have a backup. Thanks. – d5e5 Feb 03 '11 at 19:10

DavidGamba · Answer 4 · 2013-01-14T16:47:12.850

1

Perl's inplace editing is much simpler than any of the answers:

sub edit_in_place
{
    my $file       = shift;
    my $code       = shift;
    {
        local @ARGV = ($file);
        local $^I   = '';
        while (<>) {
            &$code;
        }
    }
}

edit_in_place $file, sub {
    s/search/replace/;
    print;
};

if you want to create a backup then change local $^I = ''; to local $^I = '.bak';

edited Jan 14 '13 at 16:47

answered Jan 14 '13 at 16:39

DavidGamba

3,503
2
30
46

Read entire file then print when editing inplace?

4 Answers4