How do I make JTIdy make HTML documents well-formed?

Question

I'm using JTidy v. r938. I'm using this code to attempt to clean up a page …

final Tidy tidy = new Tidy();
tidy.setQuiet(false);
tidy.setShowWarnings(true);
tidy.setShowErrors(0);
tidy.setMakeClean(true);
Document document = tidy.parseDOM(conn.getInputStream(), null);

But when I parse this URL -- http://www.chicagoreader.com/chicago/EventSearch?narrowByDate=This+Week&eventCategory=93922&keywords=&page=1, things aren't getting cleaned up. For example, the META tags on the page, like

<META http-equiv="Content-Type" content="text/html; charset=UTF-8">

remain as

<META http-equiv="Content-Type" content="text/html; charset=UTF-8">

instead of having a "</META>" tag or appearing as "<META http-equiv="Content-Type" content="text/html; charset=UTF-8"/>". I confirm this by outputting the resulting JTidy org.w3c.dom.Document as a String.

What can I do to make JTidy truly clean up the page -- i.e. make it well-formed? I realize there are other tools out there, but this question specifically relates to using JTIdy.

Ever find a solution to this? – Drazen Bjelovuk Aug 25 '14 at 20:23 — Drazen Bjelovuk, Aug 25 '14 at 20:23

Paul Vargas · Answer 1 · 2012-04-30T22:11:57.417

8

You need specify several flags to Tidy if you want XML format

private String cleanData(String data) throws UnsupportedEncodingException {
    Tidy tidy = new Tidy();
    tidy.setInputEncoding("UTF-8");
    tidy.setOutputEncoding("UTF-8");
    tidy.setWraplen(Integer.MAX_VALUE);
    tidy.setPrintBodyOnly(true);
    tidy.setXmlOut(true);
    tidy.setSmartIndent(true);
    ByteArrayInputStream inputStream = new ByteArrayInputStream(data.getBytes("UTF-8"));
    ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
    tidy.parseDOM(inputStream, outputStream);
    return outputStream.toString("UTF-8");
}

Or simply if want XHTML form

Tidy tidy = new Tidy();
tidy.setXHTML(true);

edited Apr 30 '12 at 22:11

answered Apr 30 '12 at 22:04

Paul Vargas

41,222
15
102
148

3

I tried this with both "setXmlOut" and "setXHTML" but neither generated a well-formed doc returned by "document = tidy.parseDOM(...)". Also, JTidy is spitting out the message, "This document has errors that must be fixed before using HTML Tidy to generate a tidied up version." – Dave May 01 '12 at 13:53

score 3 · Answer 2 · answered Feb 12 '13 at 16:22

3

use tidy.setXmlTags(true); to parse XML instead of HTML

answered Feb 12 '13 at 16:22

Brian

402
3
12

It worked for one of my HTML parsing Issue with tidy.. Thanks :) – CFUser Sep 28 '15 at 06:40

score 2 · Answer 3 · answered Aug 05 '13 at 21:27

2

Use Tidy.setForceOutput(true) (at your own risk) to generate the output even if errors are found.

answered Aug 05 '13 at 21:27

Adam Mackler

1,980
1
18
32

score 1 · Answer 4 · answered Feb 06 '14 at 06:01

I parse the HTML 2 times to get well formed xml

  BufferedReader br = new BufferedReader(new StringReader(str));
  StringWriter sw = new StringWriter();

  Tidy t = new Tidy();
  t.setDropEmptyParas(true);
  t.setShowWarnings(false); //to hide errors
  t.setQuiet(true); //to hide warning
  t.setUpperCaseAttrs(false);
  t.setUpperCaseTags(false);
  t.parse(br,sw);
  StringBuffer sb = sw.getBuffer();
  String strClean = sb.toString();
  br.close();
  sw.close();

  //do another round of tidyness
  br = new BufferedReader(new StringReader(strClean));
  sw = new StringWriter();

  t = new Tidy();
  t.setXmlTags(true);
  t.parse(br,sw);
  sb = sw.getBuffer();
  String strClean2 = sb.toString();
  br.close();
  sw.close();

How do I make JTIdy make HTML documents well-formed?

4 Answers4

Linked