Does Ruby support nonblocking HTTP requests without a third-party library?

Question

I am writing an API to access resources on one of my servers. Part of this API will make HTTP requests. In the name of good software design, I don't want my API to be blocking.

I am fairly new to Ruby, but in Java I would provide an Asynchronous API that returns Futures of the response. In JavaScript I would take callbacks in my methods.

I've searched for other Stack Overflow questions, and https://github.com/eventmachine/em-http-request seems to do what I want. However, I am hesitant to rely on a third-party library. Is there a Ruby native way to solve this problem, or do most developers rely on third-party libraries?

Any reason you can't just use [threads](http://stackoverflow.com/questions/6643964/asynchronous-http-request-in-ruby)? — PinnyM, Jun 17 '15 at 18:06
you can always fall back to `IO.select` to implement nonblocking functionality: http://stackoverflow.com/questions/6165735/understanding-io-select-when-reading-socket-in-ruby if you want your api to respond faster, you might be much better off by using something like sidekiq for asynchronous tasks! — phoet, Jun 17 '15 at 19:03
Consider this: Ruby is a third-party library/package, as are many other things you add to a machine after the initial OS installation. It's just how OSS works. You have to decide to what extent you want to go with "other" software, but it's very common to rely on well-written and well-tested libraries/gems, whether they come from the core developers or not. (And, just because something comes from the core team doesn't automatically mean it's the best choice, it's just what is bundled.) — the Tin Man, Jun 17 '15 at 19:12
As far as Event Machine, or other gems, look to see their activity level, whether they're currently supported, etc. I'd recommend looking at the [Typhoeus/Hydra](https://github.com/typhoeus/typhoeus) duo. It's very powerful and fast and supports multiple request in parallel and it's easy to work with. — the Tin Man, Jun 17 '15 at 19:17

score 5 · Accepted Answer · answered Jun 17 '15 at 19:37

Based on my tests, MRI does support nonblocking HTTP requests simply by using a thread. Threads don't always allow parallelism in Ruby due to the GIL, but net/http appears to be one of the exceptions:

require 'net/http'
require 'benchmark'

uri = URI('http://stackoverflow.com/questions/30899019/')

n = 10
Benchmark.bm do |b|
  b.report { n.times { Net::HTTP.get(uri) } }
  b.report { threads = Array.new(n) { Thread.new { Net::HTTP.get(uri) } }; threads.each(&:join) }
end
#     user     system      total        real
# 0.010000   0.010000   0.020000 (  0.102024)
# 0.020000   0.010000   0.030000 (  0.025904)

score 1 · Answer 2 · answered Oct 31 '21 at 22:47

Since Ruby 3.0 the standard library also was made compatible with Fibers and Async gem so you can do the following:

require 'async'
require 'benchmark'
require 'open-uri'
require 'httparty'

n = 3
Benchmark.bm(20) do |b|

  b.report "Sequential" do
    n.times do |i|
      HTTParty.get("https://httpbin.org/delay/1.6")
    end
  end

  b.report "Threads + HTTParty" do
    threads = Array.new(n) { |i|
      Thread.new {
        HTTParty.get("https://httpbin.org/delay/1.6")
      }
    }
    threads.each(&:join)
  end

  b.report "Async + HTTParty" do
    Async do |task|
      n.times do |i|
        task.async do
          HTTParty.get("https://httpbin.org/delay/1.6")
        end
      end
    end
  end

  b.report "Async + open-uri" do
    Async do |task|
      n.times do |i|
        task.async do
          URI.open("https://httpbin.org/delay/1.6")
        end
      end
    end
  end

end

I ran this code in a docker limited to one CPU and got 6 seconds for the first, and 2 seconds for the others — Edmond Lafay-David, Sep 27 '22 at 09:40

Does Ruby support nonblocking HTTP requests without a third-party library?

2 Answers2