HtmlAgilityPack and HtmlDecode

Question

I am currently using HtmlAgilityPack with a console application to scrape a website. Since the html is encoded (it returns encoded characters like ') I have to decode before I save the content to my database.

Is there a way to decode the returned html using HtmlAgilityPack without having to use HttpUtility.HtmlDecode? I want to avoid adding System.Web to my console application if possible.

Why won't you add System.Web? As it is a default .net library it should be available on any client machine anyway... — Christoph Fink, Jul 12 '11 at 14:11

score 118 · Accepted Answer · edited Dec 11 '18 at 07:44

The Html Agility Pack is equiped with a utility class called HtmlEntity. It has a static method with the following signature:

/// <summary>
/// Replace known entities by characters.
/// </summary>
/// <param name="text">The source text.</param>
/// <returns>The result text.</returns>
public static string DeEntitize(string text)

It supports well-known entities (like  ) and encoded characters such as ' as well.

score 18 · Answer 2 · answered May 11 '15 at 21:15

Just adding my 2 cents: I've ran some performance tests using StopWatch class and found that HttpUtility.HtmlDecode is about 15-20% faster than the DeEntitize method. Also DeEntitize has some bugs (see comments above).

So maybe referencing System.Web is not that bad after all.

If you're writing an app that already targets ".NET full" (opposed to ".NET Client Profile" - which is a lightweight version) - I'd go for referencing System.Web.

What "comments above"? – Mathias Lykkegaard Lorenzen May 07 '18 at 07:53 — Mathias Lykkegaard Lorenzen, May 07 '18 at 07:53

score 9 · Answer 3 · answered Jul 22 '18 at 18:03

9

Use WebUtility that doesn't need any special reference.

answered Jul 22 '18 at 18:03

wezten

2,126
3
25
48

HtmlAgilityPack and HtmlDecode

3 Answers3

Linked