Parsing HTML page using C# having problems

Question

I've been trying to accomplish something using HtmlAgilityPack, Fizzler and Regular Expressions but had no luck.

The page I am trying to scrape and parse to elements is here http://www.sczg.unizg.hr/student-servis/vijest/2015-04-14-poslovi-u-administraciji/

Example of an item in item list:
<p> 
  <b>1628/ SomeBoldedTitle
  </b> 
    Some Description. 
    Some price 20,00kuna. 
  <strong>Contact somenumber
       098/1234-567 some mail
  </strong> 
</p>

I would like to parse this item to:

4/5 digit ID >1628/ in the b element
Title >SomeBoldedTitle in the b>element
Description >after /b
Contact number and link > sometimes in strong>element sometimes in b

Here is some code I tried to get at least some output, I expected all p elements with b's but nothing has come out.

 using System;
    using HtmlAgilityPack;
    using Fizzler.Systems.HtmlAgilityPack;

namespace Sample
{
    class Program
    {

        static void Main(string[] args)
        {
            var web = new HtmlWeb();
            var document = web.Load("http://www.sczg.unizg.hr/student-servis/vijest/2015-04-14-poslovi-u-administraciji/");
            var page = document.DocumentNode;
                foreach (var item in page.QuerySelectorAll("p.item"))
            {
                Console.WriteLine(item.QuerySelector("p:has(b)").InnerHtml);
            }
        }
    }
}

Here is the link to the fizzler "documentation" I used to get this code https://fizzlerex.codeplex.com/

"I 've been trying to acomplish something..." What have you been trying? — SamWhan, May 03 '16 at 12:53

score 1 · Answer 1 · answered May 04 '16 at 01:29

Forward

I recommend using an HTML parsing module because HTML can lead to some crazy edge cases that will really skew your data. But if you're in control of your source text and still want/need to use a regex, I offer this possible solution.

Description

Given the following text

Example of an item in item list:
<p> 
  <b>1628/ SomeBoldedTitle
  </b> 
    Some Description. 
    Some price 20,00kuna. 
  <strong>Contact somenumber
       098/1234-567 some mail
  </strong> 
</p>

This Regex

<p>(?:(?!<p>).)*<b>([0-9]+)/\s*((?:(?!</b>).)*?)\s*</b>\s*((?:(?!<strong>|<b>).)*?)\s*<(?:strong|b)>\s*((?:(?!</).)*?)\s*</

Will parse your text into the following capture groups:

Group 0 will be most of the string
Group 1 will be the multi digit code
Group 2 will be the title
Group 3 will be the description
Group 4 will be the phone number

Capture Groups

[0][0] = <p> 
  <b>1628/ SomeBoldedTitle
  </b> 
    Some Description. 
    Some price 20,00kuna. 
  <strong>Contact somenumber
       098/1234-567 some mail
  </
[0][1] = 1628
[0][2] = SomeBoldedTitle
[0][3] = Some Description. 
    Some price 20,00kuna.
[0][4] = Contact somenumber
       098/1234-567 some mail

Explained

Regular expression visualization

Note: right click the image and select view in new window.

NODE                     EXPLANATION
----------------------------------------------------------------------
  <p>                      '<p>'
----------------------------------------------------------------------
  (?:                      group, but do not capture (0 or more times
                           (matching the most amount possible)):
----------------------------------------------------------------------
    (?!                      look ahead to see if there is not:
----------------------------------------------------------------------
      <p>                      '<p>'
----------------------------------------------------------------------
    )                        end of look-ahead
----------------------------------------------------------------------
    .                        any character
----------------------------------------------------------------------
  )*                       end of grouping
----------------------------------------------------------------------
  <b>                      '<b>'
----------------------------------------------------------------------
  (                        group and capture to \1:
----------------------------------------------------------------------
    [0-9]+                   any character of: '0' to '9' (1 or more
                             times (matching the most amount
                             possible))
----------------------------------------------------------------------
  )                        end of \1
----------------------------------------------------------------------
  /                        '/'
----------------------------------------------------------------------
  \s*                      whitespace (\n, \r, \t, \f, and " ") (0 or
                           more times (matching the most amount
                           possible))
----------------------------------------------------------------------
  (                        group and capture to \2:
----------------------------------------------------------------------
    (?:                      group, but do not capture (0 or more
                             times (matching the least amount
                             possible)):
----------------------------------------------------------------------
      (?!                      look ahead to see if there is not:
----------------------------------------------------------------------
        </b>                     '</b>'
----------------------------------------------------------------------
      )                        end of look-ahead
----------------------------------------------------------------------
      .                        any character
----------------------------------------------------------------------
    )*?                      end of grouping
----------------------------------------------------------------------
  )                        end of \2
----------------------------------------------------------------------
  \s*                      whitespace (\n, \r, \t, \f, and " ") (0 or
                           more times (matching the most amount
                           possible))
----------------------------------------------------------------------
  </b>                     '</b>'
----------------------------------------------------------------------
  \s*                      whitespace (\n, \r, \t, \f, and " ") (0 or
                           more times (matching the most amount
                           possible))
----------------------------------------------------------------------
  (                        group and capture to \3:
----------------------------------------------------------------------
    (?:                      group, but do not capture (0 or more
                             times (matching the least amount
                             possible)):
----------------------------------------------------------------------
      (?!                      look ahead to see if there is not:
----------------------------------------------------------------------
        <strong>                 '<strong>'
----------------------------------------------------------------------
       |                        OR
----------------------------------------------------------------------
        <b>                      '<b>'
----------------------------------------------------------------------
      )                        end of look-ahead
----------------------------------------------------------------------
      .                        any character
----------------------------------------------------------------------
    )*?                      end of grouping
----------------------------------------------------------------------
  )                        end of \3
----------------------------------------------------------------------
  \s*                      whitespace (\n, \r, \t, \f, and " ") (0 or
                           more times (matching the most amount
                           possible))
----------------------------------------------------------------------
  <                        '<'
----------------------------------------------------------------------
  (?:                      group, but do not capture:
----------------------------------------------------------------------
    strong                   'strong'
----------------------------------------------------------------------
   |                        OR
----------------------------------------------------------------------
    b                        'b'
----------------------------------------------------------------------
  )                        end of grouping
----------------------------------------------------------------------
  >                        '>'
----------------------------------------------------------------------
  \s*                      whitespace (\n, \r, \t, \f, and " ") (0 or
                           more times (matching the most amount
                           possible))
----------------------------------------------------------------------
  (                        group and capture to \4:
----------------------------------------------------------------------
    (?:                      group, but do not capture (0 or more
                             times (matching the least amount
                             possible)):
----------------------------------------------------------------------
      (?!                      look ahead to see if there is not:
----------------------------------------------------------------------
        </                       '</'
----------------------------------------------------------------------
      )                        end of look-ahead
----------------------------------------------------------------------
      .                        any character
----------------------------------------------------------------------
    )*?                      end of grouping
----------------------------------------------------------------------
  )                        end of \4
----------------------------------------------------------------------
  \s*                      whitespace (\n, \r, \t, \f, and " ") (0 or
                           more times (matching the most amount
                           possible))
----------------------------------------------------------------------
  </                       '</'

Parsing HTML page using C# having problems

1 Answers1

Forward

Description

Explained