Searching in Firebase without server side code

Question

I am trying to get all the users having the name that contains a given string from Firebase. For example, if I have these users:

Devid, Andy, Bob

I would like to get all the users having the name that contains a 'D' so I expect this as result:

Devid, Andy

This is my Firebase's structure at the moment:

Since Firebase is case sensitive I've created an attribute name_ that contains the lowercase name.

Using startAt and endAt I can get all the users with the name starting with a defined string

ref.orderByChild("name_").startAt(text).endAt(text+"\uf8ff").on('value', ...);

But this gives me only the users having the name that starts with a given string, for example if text is 'D' I'll get:

Devid

1) At the moment my query means, "give me all the users having name_ that starts with a given string" is there a way to make it mean "give me all the users which name contains a given string"? EDIT: NO

Firebase Queries don't have anything similar to full-text search operators. To accomplish those, you'll either have to integrate an external full-text search engine, or come up with a very elaborate custom indexing scheme. Firebase and indexing/search

2) At the moment I don't want to have server side code, what can be a good and efficient way to implement custom indexes?

Thanks

when using regexp, you don't need to use startAt/endAt... that is the whole point — NirMH, Nov 23 '15 at 13:09
You are right, the problem is that we can only use startAt/endAt/equalsTo to do queries. Here is the doc https://www.firebase.com/docs/web/guide/retrieving-data.html#section-queries — Devid Farinelli, Nov 23 '15 at 13:17
Firebase Queries don't have anything similar to full-text search operators. To accomplish those, you'll either have to integrate an external full-text search engine, or come up with a very elaborate custom indexing scheme. This has been asked before, so I'll find a duplicate. — Frank van Puffelen, Nov 23 '15 at 15:34
Possible duplicate of [FireBase and indexing/search](http://stackoverflow.com/questions/10559191/firebase-and-indexing-search) — Frank van Puffelen, Nov 23 '15 at 15:34
Thanks Frank! I've already seen that question, is very old, so I was hoping that with the introduction of state/endAt/equalsTo/etc... searching would become possible. I'll take that response into consideration! — Devid Farinelli, Nov 24 '15 at 08:23

Jay · Accepted Answer · 2015-11-24T14:10:52.510

11

Ok - there's no way to do exactly what you want with your current structure.

However this just popped into my head:

users:
  user_1234
    first_name: "Devid"
    components:
       "D": true
       "e": true
       "v": true
       "i": true
       "d": true
  user_5678
    first_name: "Andy"
    components:
       "A": true
       "n": true
       "d": true
       "y": true
  user_1010
    first_name: "Bob"
    components:
       "B": true
       "o": true
       "b": true

and here's some ObjC Code to make it happen (and it's tested!)

Firebase *ref = [myRootRef childByAppendingPath:@"users"];

FQuery *q1 = [ref queryOrderedByChild:@"components/b"];
FQuery *q2 = [q1 queryEqualToValue:@1];

[q2 observeEventType:FEventTypeChildAdded withBlock:^(FDataSnapshot *snapshot) {

    NSLog(@"%@", snapshot.value);

}];

This code returns Bob.

To get all of the 'd' people, change the "components/b" to "components/d"

Edit:

You can get really crazy and add more combinations to expand your search capability

users:
  user_1234
    first_name: "Devid"
    components:
       "D": true
       "e": true
       "v": true
       "i": true
       "d": true
       "De": true
       "Dev": true
       "Devi": true
       "Devid": true
       "ev": true
       "evi": true
       "evid": true
       ... etc

It would pretty simple to code up a few lines of code to iterate over the name and write out the combinations.

Obviously it would be way more efficient (if you have a limited data set) to just read all of the first names into snapshot, dump them into an array and (in ObjC) use an NSPredicate to pull out what you need.

edited Nov 24 '15 at 14:10

answered Nov 23 '15 at 22:21

Jay

34,438
18
52
81

Thanks for your reply Jay! I was thinking to something similar. Is there a way to use this solution to search for 2 characters like "De"? – Devid Farinelli Nov 24 '15 at 09:15
I think that something like {"devid", "evid", "vid", "id", "d"} should work, I was also thinking about a solution like this {"d": {"e": {"v": {"i": {"d": true } } } } }. The problems are: 1) Is this solution efficient? Is there a way to tell Firebase to index these datas? (I have to study how indexing in Firebase works) 2) Is it possible to use these indexes in a Firebase query so that I don't have to download all the data to the client but only the search results? – Devid Farinelli Nov 24 '15 at 14:28
1

1) Is it efficient? No real way to answer that. It depends on your dataset size, frequency of searches and a lot of other factors. However, while visually it's a bit messy, disk space is cheap so it shouldn't be an issue. 2) Indexing, yes. See the firebase guide and examples. When you query, all of the data in the node (or nodes) would be included in the snapshot so yes they are downloaded to the client. The impact and amount of data would again depend on your dataset size. – Jay Nov 24 '15 at 14:39
2

Great answer Jay. I never expected anyone to take the "elaborate custom indexing scheme" part of my comment seriously, but you're definitely heading in that direction. Note that what you're building here is essentially what a full-text search engine will also do, although those use a specialized data structure for storing the search terms. – Frank van Puffelen Nov 26 '15 at 19:07
Thanks both for the help! I've decided to accept this answer because it provides an example of how a text search can be implemented without server side code, that was the purpose of the question. Custom indexes are funny to implement, but I'll probably use a server side script that handles the search. – Devid Farinelli Dec 07 '15 at 10:17

user3191409 · Answer 2 · 2017-02-11T01:43:36.823

oxyzen library in github does that given you do inserts and updates with some wrapped firebase

for the indexing part basically the function:

JSON stringifies a document.
removes all the property names and JSON to eave only the data (regex).
removes all xml tags (therefore also html) and attributes (remember old guidance, "data should not be in xml attributes") to leave only the pure text if xml or html was present.
removes all special chars and substitute with space (regex)
substitutes all instances of multiple spaces with one space (regex)
splits to spaces and cycles:
for each word adds refs to the document in some index structure in your db tha basically contains childs named with words with childs named with an escaped version of "ref/inthedatabase/dockey"
then inserts the document as a normal firebase application would do

in the oxyzen implementation, subsequent updates of the document ACTUALLY reads the index and updates it, removing the words that don't match anymore, and adding the new ones.

subsequent searches of words can easily find documents in the words child. multiple words searches are implemented using hits

Searching in Firebase without server side code

2 Answers2

Linked