1

I have an API which uses a Service, in which I have used Ruby thread to reduce the response time of the API. I have tried to share the context using the following example. It was working fine with Rails 4, ruby 2.2.1

Now, we have upgraded rails to 5.2.3 and ruby 2.6.5. After which service has stopped working. I can call the service from Console, it works fine. But with API call, service becomes unresponsive once it reaches CurrencyConverter.new. Any Idea what can be the issue?


class ParallelTest
  def initialize
    puts "Initialized"
  end

  def perform
    # Our sample set of currencies
    currencies = ['ARS','AUD','CAD','CNY','DEM','EUR','GBP','HKD','ILS','INR','USD','XAG','XAU']
# Create an array to keep track of threads
    threads = []

    currencies.each do |currency|
      # Keep track of the child processes as you spawn them
      threads << Thread.new do
        puts currency
        CurrencyConverter.new(currency).print
      end
    end
# Join on the child processes to allow them to finish
    threads.each do |thread|
      thread.join
    end
    { success: true }
  end
end


class CurrencyConverter
  def initialize(params)
    @curr = params
  end

  def print

    puts @curr
  end
end

If I remove the CurrencyConverter.new(currency), then everything works fine. CurrencyConverter is a service object that I have.

Found the Issue Thanks to @anothermh for this link https://guides.rubyonrails.org/threading_and_code_execution.html#wrapping-application-code https://guides.rubyonrails.org/threading_and_code_execution.html#load-interlock

As per the blog, When one thread is performing an autoload by evaluating the class definition from the appropriate file, it is important no other thread encounters a reference to the partially-defined constant.

Only one thread may load or unload at a time, and to do either, it must wait until no other threads are running application code. If a thread is waiting to perform a load, it doesn't prevent other threads from loading (in fact, they'll cooperate, and each perform their queued load in turn, before all resuming running together).

This can be resolved by permitting concurrent loads. https://guides.rubyonrails.org/threading_and_code_execution.html#permit-concurrent-loads

Rails.application.executor.wrap do
  urls.each do |currency|
    threads << Thread.new do
        CurrencyConverter.new(currency)
        puts currency
    end
    ActiveSupport::Dependencies.interlock.permit_concurrent_loads do
      threads.map(&:join)
    end
  end
end

Thank you everybody for your time, I appreciate.

mukesh4139
  • 76
  • 9
  • What application server are you using? Unicorn, puma, ... ? – ZedTuX Nov 28 '19 at 12:50
  • @ZedTuX I am using Puma – mukesh4139 Nov 28 '19 at 12:51
  • Try adding Thread.current.abort_on_exception = true inside the thread and report what you get – nPn Nov 28 '19 at 12:56
  • @nPn Server didn't respond anything, nor it prints anything. I added Thread.current.abort_on_exception = true just below the Thread.new – mukesh4139 Nov 28 '19 at 13:02
  • you can also try just adding Thread.abort_on_exception = true at the top of your file. Exceptions in threads will cause the thread to end but the main program to continue to run, so I think you are getting an exception in your call to CurrencyConverter.new – nPn Nov 28 '19 at 13:09
  • Have you upgraded Puma? What contains the changelog of Puma between the previous and the new version? – ZedTuX Nov 28 '19 at 13:20
  • Does this help? https://guides.rubyonrails.org/threading_and_code_execution.html#wrapping-application-code – anothermh Nov 28 '19 at 21:40
  • Is there anything specific to `CurrencyConverter` that might cause this? – lacostenycoder Nov 29 '19 at 00:29
  • @lacostenycoder there is nothing fancy in `CurrencyConverter` It only contains puts statement. I checked the same with passenger and webrick too. All behaves same. – mukesh4139 Nov 29 '19 at 04:39
  • @ZedTuX previously we were using passenger in production and webrick in development. Now, with rails 5 we have puma in dev & prod. – mukesh4139 Nov 29 '19 at 04:58

1 Answers1

-3

Don't re-invent the wheel and use Sidekiq instead.

From the project's page:

Simple, efficient background processing for Ruby.

Sidekiq uses threads to handle many jobs at the same time in the same process. It does not require Rails but will integrate tightly with Rails to make background processing dead simple.

With 400+ contributors, and 10k+ starts on Github, they have build a solid parallel job execution process that is production ready, and easy to setup.

Have a look at their Getting Started to see it by yourself.

ZedTuX
  • 2,859
  • 3
  • 28
  • 58
  • the example that I have used is just to give the context. The actual requirement is different. I am creating three HTTP requests and merging results of each request and then returning the result. Since external requests is taking 4-5 secs each, I have used threads to make sure the response time of my API is less. So, I cannot use `Sidekiq` for this purpose. – mukesh4139 Nov 29 '19 at 05:54
  • I see. What about something like that then: https://gist.github.com/yob/5895100. The code looks good and seem easy to test. – ZedTuX Nov 29 '19 at 06:15
  • @mukesh4139 have you checked what I suggested you in my last comment? – ZedTuX Apr 12 '21 at 03:37