0

I have users input their list of URLs that they want to crawl for the most recent updates. I'm familiar with Java, so I was trying to make a web crawler in Java, but I can't quite figure out how to get this array from javascript to Java. What would be the most compatible language so that each of the URLs in the array could be crawled?

cmagic13
  • 25
  • 1
  • 1
  • 6

1 Answers1

0

Based on my understanding, the only format to send an array from javascript is JSON. You can then parse the JSON on java side.

Refer to this link How to parse JSON in Java

For the web crawler in java you can refer to this link.

Unfortunately Java doesn't come with all of the tools to make an HTTP request and parse the page in a super easy way. Fortunately there's a really lightweight and super easy to use package called jsoup that makes this very easy.

There's about 700 lines of code to form the HTTP request and the response, and a few thousand lines of code to parse the response. But because this is all neatly bundled up in this package for us, we just have to write a few lines of code ourselves.

So there you go. Hope it helps!

Community
  • 1
  • 1
Joshua H
  • 754
  • 8
  • 18
  • I'm very much a n00b, so help me out here, please. So I changed the array into a JSON object, and now how do I get that into java and use it as a parameter? The link doesn't quite answer my question. But the web crawler part helps. – cmagic13 Jan 17 '16 at 20:06
  • 1
    JSON is a js object which can contain everything including array. So you dont need to convert the array to JSON. You do however need to use JSON.stringify in order for java to read it. You can send the object via AJAX or normal HTTP request as you would do in normal form submission. Use this link to get you started with java servlet http://hmkcode.com/java-servlet-send-receive-json-using-jquery-ajax/ . Hope it helps! – Joshua H Jan 18 '16 at 01:11