has_many associations with `limit` fetching all records instead of provided limit

Question

I have a chat model:

class Chat < ApplicationRecord
  # Associations
  has_many :chat_recipients
  has_many :recipients, through: :chat_recipients, source: :person
  has_many :messages, dependent: :destroy
  has_many :latest_5_messages, -> { order(created_at: :desc).limit(5) }, class_name: Message.name
  # Associations
end

and a message model:

class Message < ApplicationRecord
  # Associations
  belongs_to :person
  belongs_to :chat
  # Associations
end

these two associations:

has_many :messages, dependent: :destroy
has_many :latest_5_messages, -> { order(created_at: :desc).limit(5) }, class_name: Message.name

are pointing to the same model/class. What I expect is that only 5 messages should be fetched to reduce the server load. I've been developing Rails apps for more than 4 years now and I noticed something which I found somewhat strange. When I ran this command on the rails console: 2.3.1 :080 > chat = current_person.chats.includes(:recipients, :latest_5_messages).find(1) This is the output:

Chat Load (0.8ms)  SELECT  "chats".* FROM "chats" INNER JOIN "chat_recipients" ON "chats"."id" = "chat_recipients"."chat_id" WHERE "chat_recipients"."person_id" = $1 AND "chats"."id" = $2 LIMIT $3  [["person_id", 1], ["id", 1], ["LIMIT", 1]]
ChatRecipient Load (0.4ms)  SELECT "chat_recipients".* FROM "chat_recipients" WHERE "chat_recipients"."chat_id" = 1
Person Load (0.7ms)  SELECT "people".* FROM "people" WHERE "people"."id" IN (1, 2)
Message Load (1.0ms)  SELECT "messages".* FROM "messages" WHERE "messages"."chat_id" = 1 ORDER BY "messages"."created_at" DESC

It can be clearly seen that a query to fetch ALL messages was made. But I wanted to fetch only 5 records, right? What's the point of applying a limit on a has_many association if it's fetching all the records regardless. But this isn't it, now that I run: chat.latest_5_messages.count this is the output:

2.3.1 :082 > chat.latest_5_messages.count
    (0.9ms)  SELECT COUNT(count_column) FROM (SELECT  1 AS count_column FROM "messages" WHERE "messages"."chat_id" = $1 LIMIT $2) subquery_for_count  [["chat_id", 1], ["LIMIT", 5]]
 => 5

I mean what's going on? I love Rails, I absolutely do, but I think I'm still catching up on under the hood workings of ActiveRecord after developing for more than 4 years. Why another query? When I did chat.latest_5_messages I saw exactly 5 records in an Associations::CollectionProxy but the thing was when I looped on this collection, there were 80+ iterations (it fetched all associated records contrary to the limit(5) provided). Take a look:

2.3.1 :087 > chat.latest_5_messages.map(&:id)
 => [190, 189, 188, 187, 186, 185, 184, 183, 182, 181, 180, 179, 178, 177, 176, 175, 174, 172, 170, 169, 167, 166, 165, 164, 163, 162, 161, 160, 159, 158, 157, 156, 155, 152, 151, 150, 149, 148, 147, 146, 145, 144, 143, 142, 141, 140, 139, 138, 137, 136, 135, 134, 133, 132, 131, 130, 129, 128, 127, 126, 125, 124, 123, 122, 121, 120, 119, 118, 117, 116, 115, 114, 113, 112, 111, 110, 109, 108, 107, 106, 105, 104, 103, 102]

and I had to do chat.latest_5_messages.all to get my desired 5 records, but at the cost of another DB query, which I wanted to avoid.

I'm sorry If this question sounds dump to some people, I might be unaware of some parts of ActiveRecord, but it would be highly appreciated if someone explains.

P.S: Can this only be achieved using a raw SQL query or is there a way to do it the Rails way.

score 1 · Answer 1 · edited May 23 '17 at 12:00

It sounds like you're trying to eager load an association with a specified limit, which is not supported by ActiveRecord. See http://api.rubyonrails.org/classes/ActiveRecord/Associations/ClassMethods.html#module-ActiveRecord::Associations::ClassMethods-label-Eager+loading+of+associations

In particular:

If you eager load an association with a specified :limit option, it will be ignored, returning all the associated objects

This is also discussed in quite a few other places e.g. Rails 4 Eager load limit subquery, Rails Eager Load and Limit

I can't explain why map gives you all of the records though - that definitely seems odd. Are you by any chance overriding the definition of map on ApplicationRecord? Does upgrading to the latest minor version of Rails/ActiveRecord have any effect on it?

No, I'm not overriding the definition of map anywhere. And about the limit option being ignored is not mentioned in Rails 5 docs http://guides.rubyonrails.org/association_basics.html Nonetheless it's a weird thing. Thanks for answering though. — Sarmad Sabih, Mar 10 '17 at 05:02

score 0 · Answer 2 · answered Mar 09 '17 at 11:05

0

The best implementation of your idea is scopes

class Chat < ApplicationRecord
  has_many :chat_recipients
  has_many :recipients, through: :chat_recipients, source: :person
  has_many :messages, dependent: :destroy

  scope :latest_5_messages, -> { order(created_at: :desc).limit(5) }
end

answered Mar 09 '17 at 11:05

Dmitriy Gusev

151
1
6

Hey Dmitriy, thanks for answering. But this scope shouldn't be on Chat model, and even if I place it on Message model, it doesn't solve the problem. – Sarmad Sabih Mar 10 '17 at 04:56
Ok, so you should write that in Chat model scope :latest_5_messages, -> { messages.order(created_at: :desc).limit(5) } – Dmitriy Gusev Mar 10 '17 at 06:46

has_many associations with `limit` fetching all records instead of provided limit

2 Answers2