4

I want to remove ONLY html tags from text with JSOUP. I used solution from here (my previous question about JSOUP) But after some checkings I discovered that JSOUP gets JAVA heap exception: OutOfMemoryError for big htmls but not for all. For example, it fails on html 2Mb and 10000 lines. Code throws an exception in the last line (NOT on Jsoup.parse): 

public String StripHtml(String html){
  html = html.replace("&lt;", "<").replace("&gt;", ">");
  String[] tags = getAllStandardHtmlTags;
  Document thing = Jsoup.parse(html);
  for (String tag : tags) {
      for (Element elem : thing.getElementsByTag(tag)) {
          elem.parent().insertChildren(elem.siblingIndex(),elem.childNodes());
          elem.remove();
      }
  }
  return thing.html();
}

Is there a way to fix it?

Community
  • 1
  • 1
Rougher
  • 834
  • 5
  • 19
  • 46

5 Answers5

8

Alternatively, you can give a try to Jsoup cleaning capabilities. The code below will remove ALL html tags located in the passed html string.

public String StripHtml(String html) {
    return Jsoup.clean(html, Whitelist.none());
}

The whitelist (Whitelist.none()) tells the Jsoup cleaner which tags are allowed. As you can see, none html tags are allowed here. Any tags not referenced in the whitelist will be removed.

You may be interested by other provided whitelists:

Those base whitelists can be customized by adding tags (see addTags method) or by removing tags (see removeTags method).

If you want to create your own whitelist (be careful !), here is the way to go:

Whitelist myCustomWhitelist = new Whitelist();
myCustomWhitelist.addTags("b", "em", ...);

See details here: Jsoup Whitelists

Jsoup 1.8.3

Stephan
  • 41,764
  • 65
  • 238
  • 329
1

After many searching in google and after some attempts to implement html stripper by myself, my solution is to use HTMLStripCharFilter class of Solr with replacing escapedTags to blackList with standard html tags.

  1. HTMLStripCharFilter is faster than JSOUP library and regexes for big size files
  2. HTMLStripCharFilter hasn't memory problem like JSOUP (Out of memory exception) for big size files
  3. HTMLStripCharFilter isn't entering to "catastrophic backtracking" like regexes
Rougher
  • 834
  • 5
  • 19
  • 46
  • I tested `jsoup:1.9.2` with a 4.8MB HTML file and had no issue. Test was not perceivably slower than hello world basic test. – P.Brian.Mackey Sep 26 '17 at 20:21
0

I see two solutions:

  1. Increase the Java Heap space. It seems that generating the html as string needs more memory than you allow. Increasing the maximum JAVA heap can be done with the -Xmx command line parameter to the JVM:

    java -Xmx512m parsing.java

  2. You could switch from DOM based JSoup to a SAX based parser like nekohtml Such parsers can deal with any size html documents because they never build the complete DOM in memory.

luksch
  • 11,497
  • 6
  • 38
  • 53
  • 1. Didn't work for me. I tried to set 2048 Mb in Tomcat7w.exe, but it didn't help. 2. I'll check. – Rougher Jan 02 '16 at 11:14
  • @Rougher Did you use the CATALINA_OPTS variable for Tomcat? See http://stackoverflow.com/questions/2718786/how-to-increase-java-heap-space-for-a-tomcat-app in case you did not know that setting this for tomcat works a bit different. – luksch Jan 02 '16 at 11:18
0

for me was sufficient to use combination of Jsoup methods:

Jsoup.clean(Jsoup.parse(htmlString).text(), Whitelist.simpleText())

whitelist you may choose...

Peter
  • 260
  • 5
  • 3
0

Jsoup 1.14.2, just use:

doc.select(cssQuery).text();

Strips all tags and returns a nice text content. In my case I had a ul, li pairs, it nicely stripped them all.

AlpharettaTechy
  • 350
  • 2
  • 14