1

I am trying to parse a website, specifically this one It does not provide a api for that, like it does for bf4 or other titles, but the owner said that I should just parse the data.

The problem I have is that using jSoup, it retrieves the data, but if you look carefully, the website makes a new httpget and only after that the search is completed. From what I could gather, i think it sends some paramethers in the header to.

If i just use jSoup to call that like I get some data, and where the search should be I get the message:

  Please activate Javascript to see the search results.

Is there is a way to get the data? I really need this, any help is very much appreciated.

Please help

Mihai
  • 1,212
  • 16
  • 25

1 Answers1

0

You need a javascript-capable client, e.g., HtmlUnit or Selenium.

masa
  • 2,762
  • 3
  • 21
  • 32
  • But I cannot for the life of me install htmlUnit on android, do you know a tutorial or something so i can install it? – Mihai Aug 06 '14 at 16:36
  • 1
    Sorry, I missed that you are on Android. Please see [this Stack Overflow question](http://stackoverflow.com/questions/11093130/scraping-dynamically-generated-html-inside-android-app) for further advice (on cloud scraping, sounds interesting). BTW, I tested that the data is accessible with Selenium, it is a pity that that info is now useless... – masa Aug 06 '14 at 17:47
  • Bobik does look like something I need, but their website is down... Are there other alternatives? Or do you know some workaround? – Mihai Aug 07 '14 at 07:37
  • [This posting](http://stackoverflow.com/questions/17399055/android-web-scraping-with-a-headless-browser) might be useful. It discourages to use HtmlUnit, but offers Android's built-in WebView as one option. I am not familiar with WebView. – masa Aug 07 '14 at 08:09