Can not get the actual line in my code causing the error

Question

With the following example code I get a huge amount of stacktrace errors which I know comes from page.save(saveFolder) line - but nowhere in the errors does it give this line as the error. So when you have code that is 10 times as long - it is hard to know where the error is.

import com.gargoylesoftware.htmlunit.html.*;
import java.io.File;
import java.io.*;
import com.gargoylesoftware.htmlunit.WebClient;

public class download_to_send_to_stackoverflow {
    public static void main(String args[]) {
        System.out.println("Hello world!");
        String url = "https://dont want to put actual web site down here";
        HtmlPage page = null;

        WebClient webClient = new WebClient();
        webClient.getOptions().setRedirectEnabled(true);
        webClient.getOptions().setJavaScriptEnabled(true);
        webClient.getOptions().setCssEnabled(false);
        webClient.getOptions().setUseInsecureSSL(true);
        webClient.getOptions().setTimeout(30000);
        webClient.setJavaScriptTimeout(30000); //e.g. 50s
        webClient.waitForBackgroundJavaScript(15000) ;   // added 2017/1/24
        webClient.waitForBackgroundJavaScriptStartingBefore(30000) ;   // added 2017/1/24

        try {
            page = webClient.getPage( url );
            File saveFolder = new File("spool/_");
            page.save(saveFolder);     // this is the line that is causing the error.
        } catch( Exception e ) {

        }
    }
}

There error lines are as follows : -

Aug 19, 2020 10:28:28 AM com.gargoylesoftware.htmlunit.javascript.DefaultJavaScriptErrorListener scriptException
SEVERE: Error during JavaScript execution
======= EXCEPTION START ========
Exception class=[java.lang.RuntimeException]
com.gargoylesoftware.htmlunit.ScriptException: Exception invoking setInnerHTML
    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:929)
    at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:619)
    at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:537)
    at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.callSecured(HtmlUnitContextFactory.java:354)
    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:858)
    at com.gargoylesoftware.htmlunit.javascript.host.xml.XMLHttpRequest.setState(XMLHttpRequest.java:213)
    at com.gargoylesoftware.htmlunit.javascript.host.xml.XMLHttpRequest.doSend(XMLHttpRequest.java:804)
    at com.gargoylesoftware.htmlunit.javascript.host.xml.XMLHttpRequest.access$000(XMLHttpRequest.java:105)
    at com.gargoylesoftware.htmlunit.javascript.host.xml.XMLHttpRequest$1.run(XMLHttpRequest.java:636)
    at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:619)
    at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:537)
    at com.gargoylesoftware.htmlunit.javascript.background.JavascriptXMLHttpRequestJob.run(JavascriptXMLHttpRequestJob.java:36)
    at com.gargoylesoftware.htmlunit.javascript.background.JavaScriptJobManagerImpl.runSingleJob(JavaScriptJobManagerImpl.java:429)
    at com.gargoylesoftware.htmlunit.javascript.background.DefaultJavaScriptExecutor.run(DefaultJavaScriptExecutor.java:148)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Exception invoking setInnerHTML
    at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:214)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptableObject$GetterSlot.setValue(ScriptableObject.java:323)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptableObject.putImpl(ScriptableObject.java:2868)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptableObject.put(ScriptableObject.java:536)
    at com.gargoylesoftware.htmlunit.javascript.HtmlUnitScriptable.put(HtmlUnitScriptable.java:151)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptableObject.putProperty(ScriptableObject.java:2539)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.setObjectElem(ScriptRuntime.java:1721)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.setObjectElem(ScriptRuntime.java:1706)
    at net.sourceforge.htmlunit.corejs.javascript.Interpreter.doSetElem(Interpreter.java:2460)
    at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1488)
    at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:1013)
    at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:111)
    at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:427)
    at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.doTopCall(HtmlUnitContextFactory.java:340)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3640)
    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$3.doRun(JavaScriptEngine.java:851)
    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:914)
    ... 14 more
Caused by: java.lang.NullPointerException
    at com.gargoylesoftware.htmlunit.html.HtmlScript$2.execute(HtmlScript.java:227)
    at com.gargoylesoftware.htmlunit.html.HtmlScript.onAllChildrenAddedToPage(HtmlScript.java:256)
    at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoDOMBuilder.endElement(HtmlUnitNekoDOMBuilder.java:560)
    at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
    at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoDOMBuilder.endElement(HtmlUnitNekoDOMBuilder.java:514)
    at net.sourceforge.htmlunit.cyberneko.HTMLTagBalancer.callEndElement(HTMLTagBalancer.java:1192)
    at net.sourceforge.htmlunit.cyberneko.HTMLTagBalancer.endElement(HTMLTagBalancer.java:1132)
    at net.sourceforge.htmlunit.cyberneko.filters.DefaultFilter.endElement(DefaultFilter.java:219)
    at net.sourceforge.htmlunit.cyberneko.filters.NamespaceBinder.endElement(NamespaceBinder.java:312)
    at net.sourceforge.htmlunit.cyberneko.HTMLScanner$ContentScanner.scanEndElement(HTMLScanner.java:3189)
    at net.sourceforge.htmlunit.cyberneko.HTMLScanner$ContentScanner.scan(HTMLScanner.java:2114)
    at net.sourceforge.htmlunit.cyberneko.HTMLScanner.scanDocument(HTMLScanner.java:937)
    at net.sourceforge.htmlunit.cyberneko.HTMLConfiguration.parse(HTMLConfiguration.java:443)
    at net.sourceforge.htmlunit.cyberneko.HTMLConfiguration.parse(HTMLConfiguration.java:394)
    at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
    at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoDOMBuilder.parse(HtmlUnitNekoDOMBuilder.java:760)
    at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoHtmlParser.parseFragment(HtmlUnitNekoHtmlParser.java:158)
    at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoHtmlParser.parseFragment(HtmlUnitNekoHtmlParser.java:112)
    at com.gargoylesoftware.htmlunit.javascript.host.Element.parseHtmlSnippet(Element.java:868)
    at com.gargoylesoftware.htmlunit.javascript.host.Element.setInnerHTML(Element.java:920)
    at com.gargoylesoftware.htmlunit.javascript.host.html.HTMLElement.setInnerHTML(HTMLElement.java:676)
    at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:188)
    ... 30 more
Enclosed exception: 
java.lang.RuntimeException: Exception invoking setInnerHTML
    at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:214)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptableObject$GetterSlot.setValue(ScriptableObject.java:323)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptableObject.putImpl(ScriptableObject.java:2868)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptableObject.put(ScriptableObject.java:536)
    at com.gargoylesoftware.htmlunit.javascript.HtmlUnitScriptable.put(HtmlUnitScriptable.java:151)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptableObject.putProperty(ScriptableObject.java:2539)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.setObjectElem(ScriptRuntime.java:1721)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.setObjectElem(ScriptRuntime.java:1706)
    at net.sourceforge.htmlunit.corejs.javascript.Interpreter.doSetElem(Interpreter.java:2460)
    at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1488)
    at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:1013)
    at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:111)
    at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:427)
    at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.doTopCall(HtmlUnitContextFactory.java:340)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3640)
    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$3.doRun(JavaScriptEngine.java:851)
    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:914)
    at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:619)
    at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:537)
    at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.callSecured(HtmlUnitContextFactory.java:354)
    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:858)
    at com.gargoylesoftware.htmlunit.javascript.host.xml.XMLHttpRequest.setState(XMLHttpRequest.java:213)
    at com.gargoylesoftware.htmlunit.javascript.host.xml.XMLHttpRequest.doSend(XMLHttpRequest.java:804)
    at com.gargoylesoftware.htmlunit.javascript.host.xml.XMLHttpRequest.access$000(XMLHttpRequest.java:105)
    at com.gargoylesoftware.htmlunit.javascript.host.xml.XMLHttpRequest$1.run(XMLHttpRequest.java:636)
    at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:619)
    at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:537)
    at com.gargoylesoftware.htmlunit.javascript.background.JavascriptXMLHttpRequestJob.run(JavascriptXMLHttpRequestJob.java:36)
    at com.gargoylesoftware.htmlunit.javascript.background.JavaScriptJobManagerImpl.runSingleJob(JavaScriptJobManagerImpl.java:429)
    at com.gargoylesoftware.htmlunit.javascript.background.DefaultJavaScriptExecutor.run(DefaultJavaScriptExecutor.java:148)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
    at com.gargoylesoftware.htmlunit.html.HtmlScript$2.execute(HtmlScript.java:227)
    at com.gargoylesoftware.htmlunit.html.HtmlScript.onAllChildrenAddedToPage(HtmlScript.java:256)
    at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoDOMBuilder.endElement(HtmlUnitNekoDOMBuilder.java:560)
    at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
    at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoDOMBuilder.endElement(HtmlUnitNekoDOMBuilder.java:514)
    at net.sourceforge.htmlunit.cyberneko.HTMLTagBalancer.callEndElement(HTMLTagBalancer.java:1192)
    at net.sourceforge.htmlunit.cyberneko.HTMLTagBalancer.endElement(HTMLTagBalancer.java:1132)
    at net.sourceforge.htmlunit.cyberneko.filters.DefaultFilter.endElement(DefaultFilter.java:219)
    at net.sourceforge.htmlunit.cyberneko.filters.NamespaceBinder.endElement(NamespaceBinder.java:312)
    at net.sourceforge.htmlunit.cyberneko.HTMLScanner$ContentScanner.scanEndElement(HTMLScanner.java:3189)
    at net.sourceforge.htmlunit.cyberneko.HTMLScanner$ContentScanner.scan(HTMLScanner.java:2114)
    at net.sourceforge.htmlunit.cyberneko.HTMLScanner.scanDocument(HTMLScanner.java:937)
    at net.sourceforge.htmlunit.cyberneko.HTMLConfiguration.parse(HTMLConfiguration.java:443)
    at net.sourceforge.htmlunit.cyberneko.HTMLConfiguration.parse(HTMLConfiguration.java:394)
    at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
    at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoDOMBuilder.parse(HtmlUnitNekoDOMBuilder.java:760)
    at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoHtmlParser.parseFragment(HtmlUnitNekoHtmlParser.java:158)
    at com.gargoylesoftware.htmlunit.html.parser.neko.HtmlUnitNekoHtmlParser.parseFragment(HtmlUnitNekoHtmlParser.java:112)
    at com.gargoylesoftware.htmlunit.javascript.host.Element.parseHtmlSnippet(Element.java:868)
    at com.gargoylesoftware.htmlunit.javascript.host.Element.setInnerHTML(Element.java:920)
    at com.gargoylesoftware.htmlunit.javascript.host.html.HTMLElement.setInnerHTML(HTMLElement.java:676)
    at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:188)
    ... 30 more
======= EXCEPTION END ========

Question is : how can I get my code to always tell me which line of my code is causing the problem.

You will need to have a meaningful `try-catch` block to start with. — PM 77-1, Aug 19 '20 at 01:43
Does this answer your question? [What is a NullPointerException, and how do I fix it?](https://stackoverflow.com/questions/218384/what-is-a-nullpointerexception-and-how-do-i-fix-it) — PM 77-1, Aug 19 '20 at 01:46
Look at the method: HtmlScript.onAllChildrenAddedToPage() Unfortunately, you didn't include that in to code you shared. — NomadMaker, Aug 19 '20 at 01:48
It does not matter if a put in a printstacktrace() in my catch or not - the stacktrace is coming from the page.save(folder) command and even if I put commands in my catch - they are never executed. the stacktrace is coming from page.save command. with my level of knowledge I can not catch it. — Adam Kaye-Smith, Aug 19 '20 at 02:18
The question is not a matter of how to fix - ie how to solve a NullPointerException - but that the error output should tell me which line of code is causing the problem - in code of 1000 lines long - this is important — Adam Kaye-Smith, Aug 19 '20 at 02:19
You don't seem to have included the full stack trace, just the 'caused by' part, which indicates that an exception was caught, wrapped and regrown. — tgdavies, Aug 19 '20 at 02:46
It tells you the line in the bottom `Caused by`: something is null in HtmlScript.java on line 227 — Player One, Aug 19 '20 at 03:11
Dont think so - my code is called download_to_send_to_stackoverflow.java and it is only 51 lines long. HtmlScript.java is one of the many libraries that is called which I have never written. — Adam Kaye-Smith, Aug 19 '20 at 03:15
The original question was - what line of my code is causing the problem and that be indicated in the error output - because if you have a huge amount of code - this is very helpful to know which line you wrote is causing the error — Adam Kaye-Smith, Aug 19 '20 at 03:55
During a complex program's execution, it is sometimes difficult for the exact line to be pinpointed. The method I'd look into is HtmlScript.onAllChildrenAddedToPage(), line 256. — NomadMaker, Aug 19 '20 at 04:28
I dont believe that - all the code had to do is print the full stacktrace and then we would know exactly which line of code is the problem. but instead java has chosen to summarize the stacktrace with the line ... 30 more - indicating that there is 30 more lines in the stack trace that is not printed out. — Adam Kaye-Smith, Aug 19 '20 at 05:15
I think you don't understand what `...30 more` means, maybe you should read [this](https://stackoverflow.com/questions/2970361/how-to-print-the-full-stacktrace-in-java). — haoyu wang, Aug 19 '20 at 07:11
So I have read the recommended link - my question remains - what is the line of my code that started the fault - I would the line of my code printed as a line number from my code - do you have a command that can tell me which is the line of code that initiated this partial listing of the stack. . I dont have this problem in perl - it tells you exactly where the error. — Adam Kaye-Smith, Aug 19 '20 at 07:38
Frequently in my java code - that stacktrace completes with not ...30 more but the last line of the stacktrace is the name of my java class that I wrote and the line which started of the error. I have read plenty of online stuff saying it is the Exception is an array and you have to cycle through the array to get all the stack trace information - but I am not in control of the object htmlpage which is throwing and catching the error and this htmlpage code does not allow me to catch that exception and cycle thru the array because it has already been caught. — Adam Kaye-Smith, Aug 19 '20 at 07:43

score 1 · Answer 1 · answered Aug 19 '20 at 07:53

The reason page.save(saveFolder) is not on the stack trace is that it started a new thread and the exception is thrown in the new thread. In the new thread none of your code is executed so it won't show you which line of you code goes wrong.

In fact in this case JVM can't know which line of you code is the cause. you may think page.save(saveFolder); is the cause but more likely it is because you forget to set some of the pararmeter before you call page.save(saveFolder);.

The only thing you can do is check the exception to see what is going wrong, knowing which line of your code cause the exception won't help you much. This is why debugging multi-thread programe is difficult.

Is it possible to enforce your application to be single threaded in order to debug your code — Adam Kaye-Smith, Aug 19 '20 at 08:03
I think it's impossible. If the library use multi-thread, how could you make it single-thread. — haoyu wang, Aug 19 '20 at 09:25

Can not get the actual line in my code causing the error

1 Answers1