Improve performance of find_by method matching strings?

Question

Pardon my Ruby/Rails ignorance, I'm still learning.

Details:
I have two mongodb tables. people has 214 records, people_org has 107, this will only increase in the future. The only unique connection between these tables is a person's name, which is on average ~15 characters. I do not have control over this DB structure at this time.

Currently I'm looping through each people object, then using find_by to find the matching record, printing an ID that I need, and printing some information.

<%@people.each do |p|%>
...
    <%=@people_org.find_by(name: p.name).id%>
    ...[priting a few paragraphs of text]...
    <%end%>
...
<%end%>

Problem:
The performance is quite slow (~10 seconds).

Potential Solutions:
(1) Our team may adapt to a structure that does not rely on string matching and use integers instead (I assume this would improve performance).

(2) Perhaps there is a more efficient method?

(3) Perhaps there is a way to prepare or alter the data for a more effcient sort?

Thank you in advance for your help.

Do you have any assocition between `people` and `people_org` ? — Vishal, Aug 02 '17 at 13:29
You can use the lookup aggregation provided by MongoDB: https://stackoverflow.com/a/35948843/2100645 — Lucas Costa, Aug 02 '17 at 13:30
quite slow? 10s is abysmal, especially for a few hundred records. — Eric Duminil, Aug 02 '17 at 13:39
(3) : You can grab all the `PeopleOrg` records before displaying the page and try to look for them in the view: `@people_orgs_grouped_by_name = PeopleOrg.where(name: @people.pluck(:name)).group_by(&:name)` and then in the view use `if @people_orgs_grouped_by_name[p.name].present?` etc. (1) yes, you should define a relation between the 2 models without using the name, because Bob (id 1) is not the same as Bob (id 2) but will still be grabbed by this code — MrYoshiji, Aug 02 '17 at 13:40

score 0 · Answer 1 · answered Sep 08 '17 at 01:08

As mentioned in the comment, you can utilise MongoDB Aggregation operator $lookup to perform lookup operation to another collection.

For example, having MongoDB documents in people collection such below:

{ "name": "Brandon", 
  "phone": 123456789 }

also an example of documents in people_org collection as below:

{ "organisation": "NBCC", 
  "contact": "Brandon"}

You can use MongoDB Ruby Aggregation example as below to lookup people by name to people_org:

cursor = collection.aggregate([
                   {"$match":
                          {"name":"Brandon"}}, 
                   {"$lookup": 
                          {"from": "people_org", 
                           "localField": "name", 
                           "foreignField": "contact", 
                           "as": "organisation"}
                   }])
cursor.each do |document|
    puts document
end

Note that $lookup is available only since MongoDB v3.2+

Improve performance of find_by method matching strings?

1 Answers1