EOFError: end of file reached issue with Net::HTTP

Question

I am using ruby-1.8.7-p302/Rails 2.3.11. I am trying to use FQL (Facebook API) to get stats for a link. Here's my code:

def stats(fb_post_url)
  url = BASE_URI + "?query=#{URI.encode("select like_count from link_stat where url=\"#{fb_post_url}\"")}"
  parsed_url = URI.parse(url)
  http = Net::HTTP.new(parsed_url.host, parsed_url.port)
  request = Net::HTTP::Get.new(parsed_url.request_uri)

  response = http.request(request)
  response.inspect
end

And here's the error:

EOFError: end of file reached
from /home/rahul/.rvm/rubies/ruby-1.8.7-p302/lib/ruby/1.8/net/protocol.rb:135:in `sysread'
from /home/rahul/.rvm/rubies/ruby-1.8.7-p302/lib/ruby/1.8/net/protocol.rb:135:in `rbuf_fill'
from /home/rahul/.rvm/rubies/ruby-1.8.7-p302/lib/ruby/1.8/timeout.rb:67:in `timeout'
from /home/rahul/.rvm/rubies/ruby-1.8.7-p302/lib/ruby/1.8/timeout.rb:101:in `timeout'
from /home/rahul/.rvm/rubies/ruby-1.8.7-p302/lib/ruby/1.8/net/protocol.rb:134:in `rbuf_fill'
from /home/rahul/.rvm/rubies/ruby-1.8.7-p302/lib/ruby/1.8/net/protocol.rb:116:in `readuntil'
from /home/rahul/.rvm/rubies/ruby-1.8.7-p302/lib/ruby/1.8/net/protocol.rb:126:in `readline'
from /home/rahul/.rvm/rubies/ruby-1.8.7-p302/lib/ruby/1.8/net/http.rb:2028:in `read_status_line'
from /home/rahul/.rvm/rubies/ruby-1.8.7-p302/lib/ruby/1.8/net/http.rb:2017:in `read_new'
from /home/rahul/.rvm/rubies/ruby-1.8.7-p302/lib/ruby/1.8/net/http.rb:1051:in `request'
from /home/rahul/.rvm/rubies/ruby-1.8.7-p302/lib/ruby/1.8/net/http.rb:1037:in `request'
from /home/rahul/.rvm/rubies/ruby-1.8.7-p302/lib/ruby/1.8/net/http.rb:543:in `start'
from /home/rahul/.rvm/rubies/ruby-1.8.7-p302/lib/ruby/1.8/net/http.rb:1035:in `request'
from /home/rahul/Work/Radr/lib/fb_stats.rb:13:in `stats'
from (irb):10

This seems to be happening only in case of the Facebook API. Also, I saw it suggested in some post this could be a bug in Net::HTTP.

Answer 1

If the URL is using https instead of http, you need to add the following line:

parsed_url = URI.parse(url)
http = Net::HTTP.new(parsed_url.host, parsed_url.port)
http.use_ssl = true

Note the additional http.use_ssl = true.

And the more appropriate code which would handle both http and https will be similar to the following one.

url = URI.parse(domain)
req = Net::HTTP::Post.new(url.request_uri)
req.set_form_data({'name'=>'Sur Max', 'email'=>'some@email.com'})
http = Net::HTTP.new(url.host, url.port)
http.use_ssl = (url.scheme == "https")
response = http.request(req)

See more in my blog: EOFError: end of file reached issue when post a form with Net::HTTP.

Answer 2

I had a similar problem with a request to a non-SSL service.

This blog suggested vaguely to try URI encoding the URL that is passed to 'get': http://www.loudthinking.org/2010/02/ruby-eoferror-end-of-file-reached.html

I took a shot at it, based on desperation, and in my limiting testing this seems to have fixed it for me. My new code is:

@http = Net::HTTP.new('domain.com')  
@http = @http.start    
url = 'http://domain.com/requested_url?blah=blah&etc=1'
req = Net::HTTP::Get.new(URI.encode(url))
req.basic_auth USERNAME, API_KEY
res = @http.request(req) 

Note that I use @http.start as I want to maintain the HTTP session over multiple requests. Other than that, you might like to try the most relevant part which is: URI.encode(url) inside the get call

Answer 3

I find that I run into Net::HTTP and Net::FTP problems like this periodically, and when I do, surrounding the call with a timeout() makes all of those issues vanish. So where this will occasionally hang for 3 minutes or so and then raise an EOFError:

res = Net::HTTP.post_form(uri, args)

This always fixes it for me:

res = timeout(120) { Net::HTTP.post_form(uri, args) }

Answer 4

I had the same problem, ruby-1.8.7-p357, and tried loads of things in vain...

I finally realised that it happens only on multiple calls using the same XMLRPC::Client instance!

So now I'm re-instantiating my client at each call and it just works:|

Answer 5

After doing some research, this was happening in Ruby's XMLRPC::Client library - which uses NET::HTTP. The client uses the start() method in NET::HTTP which keeps the connection open for future requests.

This happened precisely at 30 seconds after the last requests - so my assumption here is that the server it's hitting is closing requests after that time. I'm not sure what the default is for NET::HTTP to keep the request open - but I'm about to test with 60 seconds to see if that solves the issue.

Answer 6

I ran into this recently and eventually found that this was caused by a network timeout from the endpoint we were hitting. Fortunately for us we were able to increase the timeout duration.

To verify this was our issue (and actually not an issue with net http), I made the same request with curl and confirmed that the request was being terminated.

Answer 7

In Ruby on Rails I used this code, and it works perfectly:

req_profilepic = ActiveSupport::JSON.decode(open(URI.encode("https://graph.facebook.com/me/?fields=picture&type=large&access_token=#{fb_access_token}")))

profilepic_url = req_profilepic['picture']