Webhook firing multiple times, causing heavy API calls

Question

My app has some heavy callback validations when I create a new customer. Basically I check multiple APIs to see if there's a match before creating a new customer record. I don't want this to happen after create, because I'd rather not save the record in the first place if there aren't any matches.

I have a webhook setup that creates a new customer. The problem is that, because my customer validations take so long, the webhook continues to fire because it doesn't get the immediate response.

Here's my Customer model:

  validates :shopify_id, uniqueness: true, if: 'shopify_id.present?'
  before_validation :get_external_data, :on => :create 


def get_external_data
  ## heavy API calls that I don't want to perform multiple times
end

My hook:

        customer = shop.customers.new(:first_name => first_name, :last_name => last_name, :email => email, :shopify_url => shopify_url, :shopify_id => id)
        customer.save

        head :ok

customer.save is taking about 20 seconds.

To clarify, here's the issue:

Webhook is fired
Heavy API Calls are made
Second Webhook is fired (API calls still being made from first webhook). Runs Heavy API Calls
Third Webhook is fired

This happens until finally the first record is saved so that I can now check to make sure shopify_id is unique

Is there a way around this? How can I defensively program to make sure no duplicate records start to get processed?

I just make an API call to Clearbit to see if there's a match. Otherwise I have no use for the customer — Jackson Cunningham, Oct 10 '15 at 01:12

score 2 · Answer 1 · edited May 23 '17 at 11:44

What an interesting question, thank you.

Asynchronicity

The main issue here is the dependency on external web hooks.

The latency required to test these will not only impact your save times, but also prevent your server from handling other requests (unless you're using some sort of multi processing).

It's generally not a good idea to have your flow dependent on more than one external resource. In this case, it's legit.

The only real suggestion I have is to make it an asynchronous flow...

--

Asynchronous vs synchronous execution, what does it really mean?

When you execute something synchronously, you wait for it to finish before moving on to another task. When you execute something asynchronously, you can move on to another task before it finishes.

In JS, the most famous example of making something asynchronous is to use an Ajax callback... IE sending a request through Ajax, using some sort of "waiting" process to keep user updated, then returning the response.

I would propose implementing this for the front-end. The back-end would have to ensure the server's hands are not tied whilst processing the external API calls. This would either have to be done using some other part of the system (not requiring the use of the web server process), or separating the functionality into some other format.

Ajax

I would most definitely use Ajax on the front-end, or another asynchronous technology (web sockets?).

Either way, when a user creates an account, I would create a "pending" screen. Using ajax is the simplest example of this; however, it is massively limited in scope (IE if the user refreshes the page, he's lost his connection).

Maybe someone could suggest a way to regain state in an asynchronous system?

You could handle it with Ajax callbacks:

#app/views/users/new.html.erb
<%= form_for @user, remote: true do |f| %>
   <%= f.text_field ... %>
   <%= f.submit %>
<% end %>

#app/assets/javascripts/application.js
$(document).on("ajax:beforeSend", "#new_user", function(xhr, settings){
   //start "pending" screen
}).on("ajax:send", "#new_user", function(xhr){
   // keep user updated somehow
}).on("ajax:success", "#new_user", function(event, data, status, xhr){
   // Remove "pending" screen, show response
});

This will give you a front-end flow which does not jam up the server. IE you can still do "stuff" on the page whilst the request is processing.

--

Queueing

The second part of this will be to do with how your server processes the request.

Specifically, how it deals with the API requests, as they are what are going to be causing the delay.

The only way I can think of at present will be to queue up requests, and have a separate process go through them. The main benefit here being that it will make your Rails app's request asynchronous, instead of having to wait around for the responses to come.

You could use a gem such as Resque to queue the requests (it uses Redis), allowing you to send the request to the Resque queue & capture its response. This response will then form your response to your ajax request.

You'd probably have to set up a temporary user before doing this:

#app/models/user.rb
class User < ActiveRecord::Base
   after_create :check_shopify_id
   private

   def check_shopify_id
      #send to resque/redis
   end
end

Of course, this is a very high level suggestion. Hopefully it gives you some better perspective.

Gavin Miller · Answer 2 · 2015-10-10T00:44:52.930

This is a tricky issue since your customer creation is dependant on an expensive validation. I see a few ways you can mitigate this, but it will be a "lesser of evils" type decision:

Can you pre-call/pre-load the customer list? If so you can cache the list of customers and validate against that instead of querying on each create. This would require a cron job to keep a list of customers updated.
Create the customer and then perform the customer check as a "validation" step. As in, set a validated flag on the customer and then run the check once in a background task. If the customer exists, merge with the existing customer; if not, mark the customer as valid.

Either choice will require work arounds to avoid the expensive calls.

Webhook firing multiple times, causing heavy API calls

2 Answers2