How to parse HTML nodes

Question

My Website flow.

1. Authenticated user will upload docx.
2. I am using OpenXmlPowerTools API to convert this docx to HTML
3. Save the file
4. Save each node of the html page into database.

Database:-

tblNodeCollection

• NodeId
• Node Type (Expected values - <p> , <h1>, <h3> , <table>)
• NodeContent (Expected Value - <p> This is p content </p>

No issues till Step #3. But I am clueless on how to save the nodes collection into the table.

I googled & found HTMLAgiiltiyPack but don't know much about it.

using DocumentFormat.OpenXml.Packaging;
using HtmlAgilityPack;
using OpenXmlPowerTools;

namespace ExportData 
{
public class ExportHandler 
{
public void GenerateHTML()
    {
        byte[] byteArray = File.ReadAllBytes(@"d:\test.docx");
        using (MemoryStream memoryStream = new MemoryStream())
        {
            memoryStream.Write(byteArray, 0, byteArray.Length);
            using (WordprocessingDocument doc =
                WordprocessingDocument.Open(memoryStream, true))
            {
                HtmlConverterSettings settings = new HtmlConverterSettings()
                {
                    PageTitle = "My Page Title"
                };
                XElement html = HtmlConverter.ConvertToHtml(doc, settings);

                File.WriteAllText(@"d:\Test.html", html.ToStringNewLineOnAttributes());


            }
        }

        //now how do I proceed from here
    }
 }

Any type of help/guidance highly appreciated.

Answer 1

From the discussion we've had in the comments, and the part you seem to be stuck on, I'd recommend the following:

This Question here on SO may provide some help with how to convert to html.

Of course, you still face the issue of needing to be able to split each page (as you mentioned in the comments), you may be able to export each page to html individually.

As for your database structure, I'd recommend something akin to:

[Document Table]
  - Document ID
  - Document Name
  - Any other data you need per-document

[Node Table]
  - Node ID
  - Document ID (foreign key)
  - Node Content (string)

Make sure you've got sensible indexes on the node table as you're going to potentially be seeking across thousands if not millions of rows as time goes on (particularly one on the document id).

It might also be useful to have an index property against each node (e.g. a bigint position) so you can reconstitute a document by putting the nodes back together in order.

Overall though, my advice would be to try and make your boss see reason and really push against this silly design decision.

Answer 2

Here is the simplified procedure how to parse html and save it to database. I hope this will help you and/or give you an idea how to solve your problem

        HtmlWeb h = new HtmlWeb();
        HtmlAgilityPack.HtmlDocument doc = h.Load("http://stackoverflow.com/questions/41183837/how-to-store-html-nodes-into-database");
        HtmlNodeCollection tableNodes = doc.DocumentNode.SelectNodes("//table");
        HtmlNodeCollection h1Nodes = doc.DocumentNode.SelectNodes("//h1");
        HtmlNodeCollection pNodes = doc.DocumentNode.SelectNodes("//p");
        //get other nodes here

        foreach (var pNode in pNodes)
        {
            string id = pNode.Id;
            string content = pNode.InnerText;
            string tag = pNode.Name;

            //do other stuff here and then save to database

            //just an example...
            SqlConnection conn = new SqlConnection("here goes conection string");
            SqlCommand cmd = new SqlCommand();
            cmd.Connection = conn;
            cmd.CommandText = "INSERT INTO tblNodeCollection (Tag, Id, Content) VALUES (@tag, @id, @content)";
            cmd.Parameters.Add("@tag", tag);
            cmd.Parameters.Add("@id", id);
            cmd.Parameters.Add("@content", content);

            cmd.ExecuteNonQuery();
        }