How to search and extract text from html file?

Question

I have a bunch of 40k lines HTML files and I need to extract only sentences from it, so I want to automate this process. Text is located inside such blocks

<div class="text">...</div>

How do I search for these blocks and extract data between them to another file?

Maybe answer on your question is here: https://stackoverflow.com/questions/56107/what-is-the-best-way-to-parse-html-in-c — Viacheslav Yankov, Jan 26 '20 at 20:01

score 1 · Answer 1 · answered Jan 26 '20 at 20:03

If the files are truly HTML files (e.g. They are the source of an actual webpage). Your best bet is to use HtmlAgilityPack which, despite it's age, is still incredibly robust (https://html-agility-pack.net/).

Your code to load the file and get all divs with the class of text would be :

var doc = new HtmlDocument();
doc.Load(filePath);
doc.DocumentNode.SelectNodes("//div[@class='text']");

SelectNodes simply takes an XPath string so it's easy enough to manipulate and the documentation is pretty good!

How to search and extract text from html file?

1 Answers1