Elasticsearch-rails : response is empty at times, second request it returns proper response

Question

New to elasticsearch-rails. It is acting werid. When I call my API for the first time, at times, it responds with empty array but calling the same API again, returns proper response.

API Output - For the first time

API Output- Second time

My model :

class Consultation < ApplicationRecord
  include Elasticsearch::Model
  include Elasticsearch::Model::Callbacks

  after_save :set_index
  Consultation.import force: true

  def set_index
    self.__elasticsearch__.index_document
  end
end

My controller :

 def search
      required_params_present = check_required_params(%i[search])
      if required_params_present
        searched_result = Elasticsearch::Model.search("*#{params[:search]}*", [Consultation]).records.records
        data = ActiveModel::ArraySerializer.new(searched_result, each_serializer: ConsultationSerializer)
        send_response(HTTP_STATUS_CODE_200, true, I18n.t('search'), data)
      else
        send_response(HTTP_STATUS_CODE_801, false, I18n.t('params_missing.error'))
      end
 rescue => e
      send_response(HTTP_STATUS_CODE_500, false, e.message)
 end

Response is empty only for the first time. Is it that Elasticsearch take times to respond for the first time?

Any help or ideas will be really appreciated?

Answer 1

Newly indexed documents are not yet searchable immediately (within ~1 second), for performance reasons. See reference

You'll need to do a manual "refresh" on the index, if you want realtime search result. However, I quote below

While a refresh is much lighter than a commit, it still has a performance cost. A manual refresh can be useful when writing tests, but don’t do a manual refresh every time you index a document in production; it will hurt your performance. Instead, your application needs to be aware of the near real-time nature of Elasticsearch and make allowances for it.

In test environment, this is perfectly acceptable to do a "refresh" just so you could test immediately that the documents are already searchable.

Because it seems that you are on development, I would advise against this, but you may still free to do so with something like below

def set_index
  config = (Rails.env.development? || Rails.env.test?) ? { refresh: true } : {}
  self.__elasticsearch__.index_document config
end