0

I want to download results from search a engine in JSON, XML or other format using R but don't know where to start. Think of a general search for example of a company name and some other key words, like "bhp billiton iron ore production".

  • Google Search API is deprecated
  • I seems to be to difficult to "convert" my search phrases to Yahoo's YQL
  • Not sure that Yahoo BOSS would do the job (would it?)

I'm just looking for a simple tool to feed with search phrases and receive the results in a reasonable format. It is also necessary to be able to do a bit more advanced search like using AND/OR, 'within n words', doing it web and or news search, etc.

Can you give me any advice what to use?

czeinerb
  • 359
  • 2
  • 5
  • 1
    For R scraping , Take a look at some packages like `XML` , `rvest` ( You should have some **xpath** skills) ,`rjson` to format your json results. Otherwise If you have some python skills , better to use something like `scrapy` ( It is the work-horse) for scraping. – agstudy Mar 10 '15 at 20:25
  • This question will be rather closed because it asking for a tool and not a real question. – agstudy Mar 10 '15 at 20:27
  • This is a Q&A site. Asking for tools / general information is not appropriate. Regardless, [THIS](http://gastonsanchez.com/work/webdata/) is an excellent resource. – JasonAizkalns Mar 10 '15 at 20:27
  • Thank you for the help and sorry that it is so general, newbie mistake. Most links are dead but gives an idea where to look. I found a similar thread here: [link](http://stackoverflow.com/questions/6099663/) – czeinerb Mar 10 '15 at 22:14

0 Answers0