Regex get column information based on other column

Question

I have this:

<table border="1" cellspacing="1" cellpadding="0">

<tbody>

<tr><th class="align-left" style="text-align: left;">Name</th><th>Type</th><th>Size</th><th>Values</th><th>Description</th><th>Attributes</th><th>Default</th></tr>

<tr>

<td>E-mail</td>

<td>text</td>

<td>60</td>

<td>test@test.com</td>

<td>&#160;</td>

<td>M</td>

<td>test@test.com</td>

</tr>

<tr>

<td>Phone</td>

<td>text</td>

<td>20</td>

<td>01-250 481 00</td>

<td>&#160;</td>

<td>&#160;</td>

<td>&#160;</td>

</tr>

</tbody>

</table>

This is how the code looks like:

I want to pull out the information from (values) based on the left from (name) with regex/regexp but I don't know if this is possible...

For example, I want to search for "phone" and get "01-250 481 00"

What do you think?

Answer 1

Don't use regexes to parse HTML. Use a HTML parser to convert the HTML to a DOM tree. Then perform your operations in the DOM domain. E.g.

use HTML::TreeParser;

my $parser = HTML::TreeParser->new;
my $root   = $parser->parse_content($html_string);

my $table = $root->look_down(_tag => 'table');
my @rows  = $table->look_down(_tag => 'tr');
for my $row (@rows) {
    # perform your row operation here using HTML::Element methods
    # search, replace, insert, modify content...

    my @columns = $row->look_down(_tag => 'tr');

    # we need 1st (Name) and 4th (Values) column
    if (@columns >= 4) {
        if ($column[0]->as_trimmed_text() eq "Phone") {
            my $number = $column[3]->as_trimmed_text();
            ...
        }
    }
}

# if you need to dump the modified tree again...
print $root->as_HTML();

# IMPORTANT: must be done after done with DOM tree!
$root->delete();

Answer 2

A Mojo::DOM option:

use strict;
use warnings;
use Mojo::DOM;
use List::Util 'first';

my $dom = Mojo::DOM->new($html);
my $query = 'phone';

my @cols = $dom->at('tr')->find('th')->map('text')->each;
my $name_col = 1 + first { $cols[$_] eq 'Name' } 0..$#cols;
my $values_col = 1 + first { $cols[$_] eq 'Values' } 0..$#cols;

my $row = $dom->find('tr')->first(sub {
  my $name = $_->at("td:nth-of-type($name_col)");
  defined $name and $name->text =~ m/\Q$query\E/i;
});

if (defined $row) {
  my $values = $row->at("td:nth-of-type($values_col)")->text;
  print "Values: $values\n";
} else {
  print "Not found\n";
}

Answer 3

Parsing HTML with a regex is a terrible idea. My current weapon of choice when parsing HTML is Web::Query.

My approach would be to parse the table into a suitable data structure which then allows you to extract the data you want.

Something like this, perhaps...

#!/usr/bin/perl

use strict;
use warnings;
use feature 'say';

use Data::Dumper;
use Path::Tiny;
use Web::Query;

# Get the HTML. I'm reading it from a file, you might
# need to make an HTTP request.
my $html = path('table.html')->slurp;

# Parse the HTML.
my $wq = wq($html);

# Extract the text from all of the <th> elements.
# These will be the keys of our hash.
my @cols;
$wq->find('th')->each(sub { push @cols, $_->text });

# A hash to store our data
my %data;

# Find all of the <tr> elements in the HTML
$wq->find('tr')->each(sub {
  # Each table row will be a sub-hash in our hash
  my %rec;
  # Find each <td> element in the row.
  # For each element, get the text and match it with the column header.
  # Store the key/value pair in a hash.
  $_->find('td')->each(sub {
    my ($i, $elem) = @_;
    my $key = $cols[$i];
    $rec{$key} = $elem->text;
  });
  # If we have data, then store it in the main hash.
  $data{$rec{Name}} = \%rec if $rec{Name};
});

# Show what we've made.
say Dumper \%data;

# A sample query which extracts all of the values
# from the data structure.
for (keys %data) {
  say "$_ is $data{$_}{Values}";
}