I am trying to use the pusher-client gem to receive a Pusher from a website (near real-time data). However, the script tends to die a lot.
If I post the Pusher data to my screen, then the client runs for as long as I care to let it run. However, if I save the Pusher data to a file, or pass the data to a Gearman job, using the gearman gem, or put the data into Memcache, using the dalli gem, then my client script crashes.
My server is not overloaded in any way. It has 8 cores and 24 GB of RAM. Typically no single core is at 100% and no more than 6 are in use at any time and no more than 12 GB of the RAM is ever in use.
My current incarnation of the client script is attempting to pass the data off to a Gearman job, and I hope that this will offload the saving of the data and prevent missing any updates provided by Pusher, which can often come very quickly. This is a representative error the script kicks out:
ruby-2.1.1/gems/gearman-ruby-3.0.7/lib/gearman/taskset.rb:99:in `handle_job_created': Got unexpected job_created notification with handle H:hawk898
from /home/gabe/.rvm/gems/ruby-2.1.1/gems/gearman-ruby-3.0.7/lib/gearman/taskset.rb:222:in `read_packet'
from /home/gabe/.rvm/gems/ruby-2.1.1/gems/gearman-ruby-3.0.7/lib/gearman/taskset.rb:67:in `add_task_internal'
from /home/gabe/.rvm/gems/ruby-2.1.1/gems/gearman-ruby-3.0.7/lib/gearman/taskset.rb:29:in `add_task'
from /var/www/.../utils/pusher_utils.rb:81:in `send_to_gearman'
from /var/www/.../utils/pusher_utils.rb:10:in `export_to_gearman'
from /var/www/.../async_pusher.rb:59:in `block in <main>'
from /home/gabe/.rvm/gems/ruby-2.1.1/gems/pusher-client-0.4.0/lib/pusher-client/channel.rb:30:in `call'
from /home/gabe/.rvm/gems/ruby-2.1.1/gems/pusher-client-0.4.0/lib/pusher-client/channel.rb:30:in `block in dispatch'
from /home/gabe/.rvm/gems/ruby-2.1.1/gems/pusher-client-0.4.0/lib/pusher-client/channel.rb:29:in `each'
from /home/gabe/.rvm/gems/ruby-2.1.1/gems/pusher-client-0.4.0/lib/pusher-client/channel.rb:29:in `dispatch'
from /home/gabe/.rvm/gems/ruby-2.1.1/gems/pusher-client-0.4.0/lib/pusher-client/channel.rb:22:in `dispatch_with_all'
from /home/gabe/.rvm/gems/ruby-2.1.1/gems/pusher-client-0.4.0/lib/pusher-client/socket.rb:201:in `send_local_event'
from /home/gabe/.rvm/gems/ruby-2.1.1/gems/pusher-client-0.4.0/lib/pusher-client/socket.rb:73:in `block (2 levels) in connect'
from /home/gabe/.rvm/gems/ruby-2.1.1/gems/pusher-client-0.4.0/lib/pusher-client/socket.rb:67:in `loop'
from /home/gabe/.rvm/gems/ruby-2.1.1/gems/pusher-client-0.4.0/lib/pusher-client/socket.rb:67:in `block in connect'
I don't want to dig into the pusher-client gem and figure out why this is happening. I simply want to move on, so I'm tempted to use rescue Exception. I am saving the data provided by Pusher for downstream logic and action. The data is fast moving and I can either determine if the job actually made it to Gearman, possibly missing the next update or delaying it, or I can simply mark the currently stored data as stale/invalid and wait for the next update from Pusher. My preference is to mark it as stale/invalid and wait.
I've read "Why is it a bad style to `rescue Exception => e` in Ruby?" and know that rescue Exception is bad practice. However, I'm not doing this to cover up a bug in my code - I've looked at the workloads being passed to Gearman at the time of the error and they are fine.
I'm running:
Ubuntu 12.04
RVM and Ruby 2.1.1
Here is some sample code that sends a heartbeat workload to the Gearman job server:
def heartbeat(site, marketId, taskset)
startTime = timeID()
workload = myhash()
workload['marketId'] = marketId
workload['site'] = site
workload['src'] = 'pusher'
workload['time'] = startTime
workload['staleTime'] = startTime + 1
send_to_gearman('update_heartbeat', json_encode(workload), taskset)
endTime = timeID()
delta =endTime - startTime
puts sprintf("Heartbeat Task Handoff took: %9.4f msec", delta * 1E3)
end # def heartbeat
def send_to_gearman(wrkr, workload, taskset)
pp workload
begin
task = Gearman::Task.new(wrkr, workload, { :priority => :high, :background => true })
taskset.add_task(task)
taskset.wait
rescue => e
pp e
end # begin
end # def send_to_gearman
The entirety of the code runs fine for minutes if not hours before failing. Specifically every line of code is being exercised repeatedly, except those after rescue, without failing prior to the handle_job_created error.
So, is it safe to use rescue Exception in this case? Is there some subset of Exception I could use instead that I'm missing? Any better suggestions or gotchas I may have overlooked?
