I have about 2mil records in my db, and I need to frequently get random sets of x records. When my db consisted of about 50 records I used:
Product.all.sample(5)
However this is EXTREMELY costly with my records count.
How can I efficiently find a small sample of records given the size of my db.
One option would be to get a much smaller set of Products. What you're doing now is costly because you're grabbing all Products first then selecting 5 of them at random with.
Product.limit(100).sample
This isn't ideal though because you'll return the same set of 100 Products.
Instead you can remove the .sample all together and just ask ActiveRecord for a random set of 5 Products
Product.order("RAND()").limit(5)
While you've move the performance issue out of Rails you've moved it into MySQL where 'RAND()' is still pretty slow for large datasets.
This question has a bunch on answers related to your problem that should prove useful.
