Multiple JSON Requests Ruby

Question

Im using rails for with the crunchbase api to pull down the info for companies and I've been battling this error all night. It completes one request successfully and after constructing the second request it crashes and I receive this error:

Error/Users/Rich/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/psych.rb:205:in `parse':
(<unknown>): control characters are not allowed at line 1 column 1 (Psych::SyntaxError)

I can post as much code as necessary. All help is much appreciated as I'm a ruby newbie.

def get_data(company_links)
  while i < company_links.length
    puts "USING URL #{URL}#{company_links[i]}.js?#{API_Key}"
    resp = RestClient.get("#{URL}#{company_links[i]}.js?#{API_Key}")
    arr = Crack::JSON.parse(resp)
    i += 1
  end
end

calling the method

links_array = ["dropbox","twitter"]
get_data(links_array)

Edit Answer Found Don't really know why it works but im not complaining that its solved. Thanks to everyone who provided assistance I really appreciate it :)

def get_data(links_array)
    links_array.each do |company|
    puts "USING URL #{URL}#{company}.js?#{API_Key}"
    resp = RestClient.get("#{URL}#{company}.js?#{API_Key}")
    arr = JSON.parse(resp)
    puts arr["name"]
  end
end

Is `def get_data(...)` the first line of your file? The error message suggests that the error occurs in line 1, column 1 — Patrick Oscity, May 31 '13 at 06:56
Is that referring to the first line of the file or the response? My first line is require 'rubygems' — rich, May 31 '13 at 07:39

score 2 · Answer 1 · answered May 31 '13 at 07:25

2

It looks like your problem is in the JSON you're getting back, not the fetching of it. Diagnose with:

def get_data(company_links)
  company_links.each do |company|
    puts "USING URL #{URL}#{company}.js?#{API_Key}"
    resp = RestClient.get("#{URL}#{company}.js?#{API_Key}")
    puts "RETURNS: #{resp}"
    arr = Crack::JSON.parse(resp)
  end
end

Also try

links_array = ["twitter","dropbox"]
get_data(links_array)

and see if it fails on the first or second request.

Edit: I'm betting you've got some character Crack/Psych doesn't like at the very beginning of your JSON string, and that it's a "control character" ;)

Edit2: If the JSON printed to the console is valid, try arr = Crack::JSON.parse(resp.to_s) or arr = Crack::JSON.parse(resp.inspect). I'm fuzzy on a lot of the important details here.

answered May 31 '13 at 07:25

Narfanator

5,595
3
39
71

Really appreciate the help I've been going through your suggestions and its a really weird error. The json it prints for one company is perfectly valid and then it dies. Edit 2 produced a different error though: /Users/Rich/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/psych.rb:205:in `parse': (): found unknown escape character while parsing a quoted scalar at line 1 column 1 (Psych::SyntaxError) – rich May 31 '13 at 07:38
Run each get manually in the console and add the results of `puts resp` to your question. Too hard to diagnose from "here". – Narfanator May 31 '13 at 07:45
Found the answer, no idea why this worked but it does by replacing:Crack::JSON.parse(resp) with just JSON.parse(resp) – rich May 31 '13 at 07:46

score 1 · Answer 2 · edited May 23 '17 at 12:11

1

I am not sure why exactly this happening but, instead of using while loop, you might consider using threads to run multiple requests. I personally do not think looping requests is a good idea.

Take a look at this example:

Ruby on Rails Multiple HTTP request at the same time?

edited May 23 '17 at 12:11

Community

1
1

answered May 31 '13 at 07:05

whizzkid

674
1
8
22

Multiple JSON Requests Ruby

2 Answers2