I am working on a crawler project using crawler4j and on top of it, I have a swing interface. I have 2 different cases, namely the controller.java (also containing the SWING components) and crawler.java. I am attempting to append output processed by the crawler.java and append it in the controller class which also holds the swing components. I tried adding an instance of the controller class into crawler class but the swing component is not showing the updates.
controller.java
//instantiate the crawler
CrawlController controller = new CrawlController(config, pageFetcher, robotstxtServer);
controller.addSeed("http://pastebin.com/");
controller.startNonBlocking(crawler.class, 1);
I also have the following method in the controller class to update the text pane:
public void setTextOP(String t){
outputPane.setText(t);
}
In crawler.java
public class crawler extends WebCrawler {
controller c;
public crawler(controller c) {
this.c = c;
}
@Override
public void visit(Page page) {
int docid = page.getWebURL().getDocid();
String url = page.getWebURL().getURL();
String domain = page.getWebURL().getDomain();
String subDomain = page.getWebURL().getSubDomain();
int parentDocid = page.getWebURL().getParentDocid();
System.out.println("Docid: " + docid);
System.out.println("URL: " + url);
System.out.println("Domain: '" + domain + "'");
System.out.println("Sub-domain: '" + subDomain + "'");
if (page.getParseData() instanceof HtmlParseData) {
HtmlParseData htmlParseData = (HtmlParseData)page.getParseData();
String text = htmlParseData.getText();
String html = htmlParseData.getHtml();
List<WebURL> links = htmlParseData.getOutgoingUrls();
c.setTextOP(text);
}
}
Would really appreciate some advice on this. Thank you.
PS: the text output is printing fine in the IDE console.