Why is Perl's chomp affecting the output of my print?

Question

It's been a couple months since I've been Perling, but I'm totally stuck on why this is happening...

I'm on OSX, if it matters.

I'm trying to transform lines in a file like

08/03/2011 01:00 PDT,1.11

into stdout lines like

XXX, 20120803, 0100, KWH, 0.2809, A, YYY

Since I'm reading a file, I want to chomp after each line is read in. However, when I chomp, I find my printing gets all messed up. When I don't chomp the printing is fine (except for the extra newline...). What's going on here?

while(<SOURCE>) {
    chomp;
    my @tokens = split(' |,');     # @tokens now [08/03/2011, 01:00, PDT, 1.11]

    my $converted_date = convertDate($tokens[0]);
    my $converted_time = convertTime($tokens[1]);

print<<EOF;
$XXX, $converted_date, $converted_time, KWH, $tokens[3], A, YYY
EOF
}

With the chomp call in there, the output is all mixed up:

, A, YYY10803, 0100, KWH, 1.11

Without the chomp call in there, it's at least printing in the right order, but with the extra new line:

XXX, 20110803, 0100, KWH, 1.11
, A, YYY

Notice that with the chomp in there, it's like it overwrites the newline "on top of" the first line. I've added the $|=1; autoflush, but don't know what else to do here.

Thoughts? And thanks in advance....

Where do you get the data file from? Is it possible it has odd line ending characters? — Ernest Friedman-Hill, May 24 '12 at 13:58
It's the common problem Windows file on Unix again. See earlier discussions: http://stackoverflow.com/questions/881779/neatest-way-to-remove-linebreaks-in-perl http://stackoverflow.com/questions/1836217/perl-or-something-else-m-problem http://stackoverflow.com/questions/7534591/cr-vs-lf-perl-parsing — daxim, May 24 '12 at 14:02

score 10 · Accepted Answer · edited May 24 '12 at 15:36

10

The lines of your input ends with CR LF. You're removing the LF only. A simple solution is to use the following instead of chomp:

s/\s+\z//;

You could also use the dos2unix command line tool to convert the files before passing them to Perl.

edited May 24 '12 at 15:36

Greg Bacon

134,834
32
188
245

answered May 24 '12 at 14:14

ikegami

367,544
15
269
518

Duh! Thanks so much. That was it. – user311121 May 24 '12 at 14:23

score 1 · Answer 2 · answered May 24 '12 at 14:24

The problem is that you have DOS line-endings and are running on a Unix build of Perl.

One solution to this is to use PerlIO::eol. You may have to install it but there is no need for a use line in the program.

Then you can write

binmode ':raw:eol(LF)', $filehandle;

after which, regardless of the format or source of the file, the lines read will be terminated with the standard "\n".

Why is Perl's chomp affecting the output of my print?

2 Answers2