HTML sorting with Perl regex

Question

I have an HTML file consisting of an HTML table with links to Scientific Papers and Authors and with their year of publishing. The html is sorted from oldest to newest. I need to resort the table by parsing the file and getting a new file with the sorted source code from newest to oldest.

Here is a small perl script that should be doing the job but it produces semi-sorted results

local $/=undef;
open(FILE, "pubTable.html")  or die "Couldn't open file: $!";
binmode FILE;
my $html = <FILE>; 
open (OUTFILE, ">>sorted.html") || die "Can't oupen output file.\n";
map{print OUTFILE "<tr>$_->[0]</tr>"} 
sort{$b->[1] <=> $a->[1]} 
map{[$_, m|, +(\d{4}).*</a>|]}
$html =~ m|<tr>(.*?)</tr>|gs;
close (FILE);  
close (OUTFILE);

And here is my input file: link

and what I get as an output: link

From the output you can see the order is going well but then I get the year 1993 after the year 1992 and not in the beginning of the list.

[Have you tried using an HTML parser instead?](http://stackoverflow.com/a/1732454/725418) — TLP, May 02 '12 at 08:23
Why sort at all? If the input is oldest to newest, and you need newest to oldest, just *reverse* the list. Then you don't even have to parse the contents of the rows; just move them whole. — Rob Kennedy, May 03 '12 at 01:44

score 2 · Accepted Answer · answered May 02 '12 at 08:48

There was a problem with the regex in the map because of the following lines in the html.

<a href="http://www.icp.uni-stuttgart.de/~hilfer/publikationen/pdfo/">,{UCLA}-Report 982051,Los Angeles,,1989,</a></td>   </tr>

and

<a href="http://www.icp.uni-stuttgart.de/~hilfer/publikationen/pdfo/">Phys.Rev.Lett., <b> 60</b>, 1514, 1988</a></td>   </tr>
<a href="http://www.icp.uni-stuttgart.de/~hilfer/publikationen/pdfo/">Phys. Rev. B, <b> 45</b>, 7115, 1992</a></td>   </tr>
<a href="http://www.icp.uni-stuttgart.de/~hilfer/publikationen/pdfo/">J.Chem.Phys., <b> 96</b>, 2269, 1992</a></td>   </tr>

In the 1989 line the year includes a comma at the end and there's no whitespace in front. Because of that, the script threw a lot of warnings and always put that line in the bottom.

The other three lines have a four-digit number (\d{4}) with something behind it .* (the year). So the sorting used the other numbers (7115, 2269, 1514) to sort and those were mixed up with the years.

You need to adjust the regex accordingly to fix those issues.

Before:

map{[$_, m|, +(\d{4}).*</a>|]}

After:

map{[$_, m|, *(\d{4}),?</a>|]}

score 1 · Answer 2 · answered May 02 '12 at 11:47

And a solution with XML::Twig, which can also be used to process HTML. It's fairly robust: it won't process other tables in the file, it will accommodate typos like the one in the year in the UCLA report...

#!/usr/bin/perl 

use strict;
use warnings;

use XML::Twig;

my $IN  = 'sort_input.html';
my $OUT = 'sort_output.html';

my $t= XML::Twig->new( twig_handlers => { 'table[@class="pubtable"]' => \&sort_table,
                                        },
                       pretty_print => 'indented',
                     )
       ->parsefile_html( $IN)
       ->print_to_file( $OUT);

sub sort_table
  { my( $t, $table)= @_;
   $table->sort_children( sub { if($_[0]->last_child( 'td')->text=~ m{(\d+)\D*$}s) { $1; } },
                          type => 'numeric', order => 'reverse'
                        );
  }

score 0 · Answer 3 · answered May 02 '12 at 08:53

Solution with a robust HTML parsing/manipulation library:

use strictures;
use autodie qw(:all);
use Web::Query qw();
my $w = Web::Query->new_from_file('pubTable.html');
$w->find('table')->html(
    join q(),
    map { $_->[0]->html }
    sort { $a->[1] <=> $b->[1] }
    @{
        $w->find('tr')->map(sub {
            my (undef, $row) = @_;
            my ($year) = $row->find('.pubname')->text =~ /(\d\d\d\d) ,? \s* \z/msx;
            return [$row => $year];
        })
    }
);

open my $out, '>:encoding(UTF-8)', 'sorted.html';
print {$out} $w->html;
close $out;

HTML sorting with Perl regex

3 Answers3

Linked