2

I’m using Rails 4.2.7. Currently Im’ requesting web pages through a SOCKS proxy like so

begin
   ...
    res1 = Net::HTTP.SOCKSProxy('127.0.0.1', 50001).start(uri.host, uri.port) do |http|
      puts "launching #{uri}"
      resp = http.get(uri)
      status = resp.code
      content = resp.body
      content_type = resp['content-type']
      content_encoding = resp['content-encoding']
    end
  ...
rescue OpenURI::HTTPError => ex
   ...
rescue SocketError, Net::OpenTimeout, Zlib::BufError => e
   ...
end

and occasionally I will get the following error ..

Error during processing: Failed to open TCP connection to otherwebsite.com:80 (general SOCKS server failure)
/Users/mikeb/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/net/http.rb:882:in `rescue in block in connect'
/Users/mikeb/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/net/http.rb:879:in `block in connect'
/Users/mikeb/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/timeout.rb:91:in `block in timeout'
/Users/mikeb/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/timeout.rb:101:in `timeout'
/Users/mikeb/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/net/http.rb:878:in `connect'
/Users/mikeb/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/net/http.rb:863:in `do_start'
/Users/mikeb/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/net/http.rb:852:in `start'
/Users/mikeb/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/net/http.rb:584:in `start'
/Users/mikeb/Documents/workspace/runtrax/app/helpers/webpage_helper.rb:96:in `get_content'
/Users/mikeb/Documents/workspace/runtrax/app/helpers/webpage_helper.rb:32:in `get_url'
/Users/mikeb/Documents/workspace/runtrax/app/services/onlinerr_race_finder_service.rb:42:in `get_race_list'
/Users/mikeb/Documents/workspace/runtrax/app/services/abstract_race_finder_service.rb:26:in `process_data'
/Users/mikeb/Documents/workspace/runtrax/app/services/run_crawlers_service.rb:18:in `block in run_all_crawlers'
/Users/mikeb/.rvm/gems/ruby-2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/delegation.rb:46:in `each'
/Users/mikeb/.rvm/gems/ruby-2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/delegation.rb:46:in `each'

How do I catch this error and retry my request?

Edit: How is my question different than what was asked? The duplicate is referring to a buffer error, this quesiton deals with a socket error.

  • 1
    rescuing `StandardError` should do I assume, exception is coming from [here](https://github.com/ruby/ruby/blob/v2_3_1/lib/net/http.rb#L882) which reraises whatever `TCPSocket.open` throws which I assume should be a child of `StandardError` so just `begin; #body of your code; rescue => e; #your retry logic end` should do – bjhaid Oct 10 '16 at 21:05
  • Possible duplicate of [How do I catch the "Error during processing: buffer error" in Ruby when getting a web page?](http://stackoverflow.com/questions/39919198/how-do-i-catch-the-error-during-processing-buffer-error-in-ruby-when-getting) – Scott Bartell Oct 12 '16 at 05:37
  • @bjhaid, per my comment to the answer, "rescue => e" is just catching any generic error. I want to only catch the specific error mentioned in my question and retry. ScottBartell, how are those questions in any way similar? Those are two completely different errors –  Oct 12 '16 at 13:43
  • @Mike http://betterlogic.com/roger/2009/04/all-the-exceptions-that-can-be-raised-from-rubys-tcpsocketopen/ – bjhaid Oct 12 '16 at 14:04
  • Thanks for this link. I'm not clear which of these errors applies to my situation. Which of these corresponds to what I listed in my stack trace? –  Oct 12 '16 at 19:25
  • @Mike try putting this first to get the class of that specific error: `rescue => e puts e.class end` But what would you do if an error with a different class happens? I don't think you need to be that specific here. – Halil Özgür Oct 16 '16 at 16:06

1 Answers1

4

You can retry:

retries = 0
begin
  res1 = Net::HTTP.SOCKSProxy('127.0.0.1', 50001).start(uri.host, uri.port) do |http|
    puts "launching #{uri}"
    resp = http.get(uri)
    status = resp.code
    content = resp.body
    content_type = resp['content-type']
    content_encoding = resp['content-encoding']
  end
rescue => e
  retries += 1
  if retries < 3
    retry # <-- Jumps to begin
  else
    # Error handling code, e.g.
    logger.warn "Couldn't connect to proxy: #{e}"
  end
end
Halil Özgür
  • 15,731
  • 6
  • 49
  • 56
  • But here you are just catching the general error. I only want to deal with the error I specified in my question. If other errors occur, I do not want to retry. –  Oct 12 '16 at 13:41
  • @Mike I guess you can accept the behavior of HTTP library as a public interface. It doesn't discriminate exceptions. Being at a higher level, you too shouldn't need to. I guess it can be anything like a timeout, general connection error or whatever. Do you need to handle each of those error cases differently? My guess would be [`SocketError`](http://ruby-doc.org/stdlib-2.3.1/libdoc/socket/rdoc/SocketError.html) – Halil Özgür Oct 13 '16 at 22:07
  • Hi, I edited my question to reflect the fact that I gave catching "SocketError" a shot, but execution is not passing to this branch when the erorr in my question is thrown. –  Oct 16 '16 at 14:21
  • 1
    _NOTE!_ **retries = 0** should be placed out of the **begin end** scope – Valentine Konov Jul 10 '17 at 09:58