25

I'm attempting to parse HTML code with DOMDocument, do stuff like changes to it, then assemble it back to a string which I send to the output.

But there a few issues regarding parsing, meaning that what I send to DOMDocument does not always come back in the same form :)

Here's a list:

  1. using ->loadHTML:

    • formats my document regardless of the preserveWhitespace and formatOutput settings (loosing whitespaces on preformatted text)
    • gives me errors when I have html5 tags like <header>, <footer> etc. But they can be supressed, so I can live with this.
    • produces inconsistent markup - for example if I add a <link ... /> element (with a self-closing tag), after parsing/saveHTML the output will be <link .. >

  2. using ->loadXML:

    • encodes entities like > from <style> or <script> tags: body > div becomes body &gt; div
    • all tags are closed the same way, for example <meta ... /> becomes <meta...></meta>; but this can be fixed with an regex.

I didn't try HTML5lib but I'd prefer DOMDocument instead of a custom parser for performance reasons

Update:

So like the Honeymonster mentioned using CDATA fixes the main problem with loadXML.

Is there any way I could prevent self closing of all empty HTML tags besides a certain set, without using regex?

Right now I have:

$html = $dom->saveXML($node);

$html = preg_replace_callback('#<(\w+)([^>]*)\s*/>#s', function($matches){

       // ignore only these tags
       $xhtml_tags = array('br', 'hr', 'input', 'frame', 'img', 'area', 'link', 'col', 'base', 'basefont', 'param' ,'meta');

       // if a element that is not in the above list is empty,
       // it should close like   `<element></element>` (for eg. empty `<title>`)
       return in_array($matches[1], $xhtml_tags) ? "<{$matches[1]}{$matches[2]} />" : "<{$matches[1]}{$matches[2]}></{$matches[1]}>";
}, $html);

which works but it will also do the replacements in the CDATA content, which I don't want...

Alex
  • 66,732
  • 177
  • 439
  • 641
  • 3
    Do you have a test snippet we can play with? – John Conde May 23 '12 at 01:58
  • How do you know html5lib is any slower than DOMDocument if you didn't even try it? – Brad May 28 '12 at 18:46
  • 4
    I assume that because it's written in PHP.. DOMDocument is an PHP extension written in C – Alex May 28 '12 at 20:58
  • *"loosing whitespaces on preformatted text"* I doubt that. Please provide an example demonstrating that issue. And what's actually wrong with the "self closing tag"? I mean, HTML5 does not need to have it, right? Or do you mean XHTML5? – hakre May 30 '12 at 13:12
  • Actually certain elements do need to have them, at least in some browsers – Alex May 30 '12 at 13:25

6 Answers6

14

Use html5lib. It can parse html5 and produce a DOMDocument. Example:

require_once '/path/to/HTML5/Parser.php';
$dom = HTML5_Parser::parse('<html><body>...');

Documentation

Francis Avila
  • 31,233
  • 6
  • 58
  • 96
12

10 years has been passed but the problem still exists on PHP DOMDocument, I found 2 ways to fix the issue.

Solution 1

Add LIBXML_NOERROR as option to the loadHTML method like this:

<?php

$dom = new DOMDocument();

$dom->loadHTML('<header data-attribute="foo">bar<', LIBXML_NOERROR);

echo $dom->saveHTML();
// outputs the html with valid closing tag without any error
?>

Solution 2

Add libxml_use_internal_errors(true) before loading the HTML

<?php

$dom = new DOMDocument();

libxml_use_internal_errors(true);

$dom->loadHTML('<header data-attribute="foo">bar<');

echo $dom->saveHTML();
// outputs the html with valid closing tag without any error
?>
Obaydur Rahman
  • 723
  • 5
  • 15
11

If you want to support HTML5, do not touch DOMDocument at all.

Currently the best option seems to be https://github.com/Masterminds/html5-php

Previously the best option was https://github.com/html5lib/html5lib-php but as the description says, it's "currently unmaintained". And this has been status for since October 2011 so I'm not holding my breath anymore.

I haven't used html5-php in production so I cannot provide any real world experiences about that. I've used html5lib-php in production and I would say that it's parsing well formed documents correctly but it has unexpected errors with some simple syntax errors. On the other hand, it seems to implement adoption agency algorithm and some other weird corner cases correctly. If html5lib-php were still maintained, I'd still prefer it. However, as things currently stand, I'd prefer using html5-php and possibly help fixing remaining bugs there.

Mikko Rantalainen
  • 14,132
  • 10
  • 74
  • 112
  • Since PHP 7.2 extension `html-tidy` should have some kind of support for HTML5. I don't yet have any experience with it so I cannot speak for the quality of the implementation. – Mikko Rantalainen Sep 25 '20 at 12:09
  • I've used PHP `tidy` extension and in my experience the parser seems to be fast but if you request DOM from `tidy`, that's surprisingly slow. It's actually so slow that it's probably faster to use `tidy` to do conversion from HTML5 to XHTML (string->string conversion) and then use XML parser to create the DOM. – Mikko Rantalainen Sep 22 '22 at 09:04
7

Unfortunately, or possibly fortunately, domdocument is designed to not try to preserve formatting from the original document. This is to make the parser's internal state easier to manage by keeping all elements the same style. Afaik most parsers will create a tree representation in memory and not worry about the textual formatting until the user requests such. This is why your self closed tags are output with separate closing tags. The good news is that it doesn't matter.

As to style tags and script tags getting <> converted to &lt;&gt;, you may be able to avoid the conversion by surrounding the contents of the element in question with the recommended cdata tags thusly:

<style>
  /*<![CDATA[*/
    body > div {
      width: 50%;
    }
  /*]]>*/
</style>

The comment /* */ around the cdata declarations are to allow for broken clients which don't know about cdata sections and instead treat the declarations as CSS code. If you're using the document internally only then you may omit the /* */ comment surrounds and have the cdata declaration only. You may encounter issues with the aforementioned broken clients if you manipulate the document and then send it to the browser without checking to ensure the /* */ comments are retained; I am unsure whether domdocument will retain these or not.

John Carter
  • 53,924
  • 26
  • 111
  • 144
Lucy Llewellyn
  • 1,011
  • 6
  • 5
  • 1
    wow i can't believe I didn't think of using CDATA :) thanks, that solves many issues with the xml parser, which i wanted to use ;) – Alex May 28 '12 at 19:15
  • Note that you still cannot skip encoding data by yourself if you're going to embed untrusted user input. The user data may easily contain substring `]]>` or `/*]]>*/`. – Mikko Rantalainen Sep 25 '20 at 12:13
4

I tried both html5lib and html5php but neither worked with the HTML I was provided with. An alternative that was able to parse the HTML was: https://github.com/ivopetkov/html5-dom-document-php

The main class extends PHP's native DomDocument.

Potherca
  • 13,207
  • 5
  • 76
  • 94
  • 1
    My main takeaway from this particular library was this: `Allows querying the DOM with CSS selectors (currently avaiable: *, tagname, tagname#id, #id, tagname.classname, .classname, tagname[attribute="value"], [attribute="value"], tagname[attribute], [attribute])` - so you can now `$foo = $dom->querySelectorAll('img[srcset]');` - very helpful. – frumbert Mar 26 '18 at 12:38
-10

When initialising domDocument, do as follows:

$dom = new DOMDocument(5, 'UTF-8');
Justin Levene
  • 1,630
  • 19
  • 17