I want to use Ruby EventMachine and em-http-request to run parallel HTTP synchronous requests triggered from different threads.
The idea is to run a single reactor in its own thread and push HTTP requests to complete on its event queue thanks to EM.next_tick.
The pattern for every call could be
def complete_request(url, options)
Thread.new { EM.run } unless EM.reactor_running?
EM.run do
EM.next_tick do
con = EventMachine::HttpRequest.new(url, options[:connection_headers])
http = com.setup_request(verb, head: options[:headers], body: options[:body])
http.errback { }
http.callback { }
end
end
# wait for request completion (but how?)
...
end
reqs = []
responses = []
reqs << Thread.new { responses << complete_request('http://www.stackoverflow.com', verb: get) }
reqs << Thread.new { responses << complete_request('http://www.flickr.com', verb: get) }
reqs.each { |req| req.join }
To make the requests synchronous, I tried to use Fibers but unsuccessfully. Either the request fails to connect or it completes but never exits the event loop.
I don't want to call EM.stop in the callbacks because it would screw up other requests being executed in parallel I guess, and would also stop the reactor while I want it to run until I decide no more requests should be treated.
Does anyone already try to use EventMachine and em-http-request this way ? Can EventMachine support this use case ?
