0

I'm fetching longitude and latitude values from a database using a home address.

The structure of each of the documents in the collection is as follows:

"Ext": string
"Lat": string
"Long": string
"Parish": string
"Postal": string
"Street": string
"Number": string

There are a handful of different where() conditions, all wrapped in if statements.

The street number where() condition always causes the query to return no matches, and all of the other where()'s find matches correctly.

I have tried:

  • commenting out the street number where() condition
  • hard coding values copied from the database
  • using parseInt() and toString() in the where() call to force typing
  • logging data about the string to the console to check typing, leading/trailing whitespace, etc.
function addressToCoords() {
  // returns an array of coordinates if successful, false otherwise

  const ADD = document.getElementById("address").value;
  let NUM;
  let STREET;
  if (ADD) {
    NUM = ADD.match(/^[0-9]+/)[0];
    STREET = ADD.replace(/^[0-9]+\s/, "");

    if (NUM === STREET) {
      console.log("NUM = STREET. regex split failed");
      STREET = null;
      return false;
    }
  }

  const CTY = document.getElementById('parish').value;
  const POS = document.getElementById('postalCode').value.replace(/\s/, "");

  let coords = [0.0, 0.0];
  let query = f.addresses;

  if (CTY) { console.log("CTY: ", CTY); query = query.where("Parish", "==", CTY); }

  if (STREET) { console.log("STREET: ", STREET); query = query.where("Street", "==", STREET); }

  if (POS) { console.log("POS: ", POS); query = query.where("Postal", "==", POS); }

  // -FIXME- doesn't match
  if (NUM) {
    // Usual test case outputs: 
    //           NUM:     1          string              1
    console.log("NUM: ", NUM, " ", typeof(NUM), " ", NUM.length);
    query = query.where("Number", "==", NUM);
  } 

  query.limit(5)
    .get()
    .then(querySnapshot => {
      if (!querySnapshot.empty) {
        querySnapshot.docs.forEach (ele => {
          console.log("doc found!: ", ele.data());
        });
        coords[0] = querySnapshot.docs[0].Lat;
        coords[1] = querySnapshot.docs[0].Long;
      } else {
        console.log("no doc found!");
      }
  }).catch(function(error) {
    console.log(error);
  });

  document.getElementById("Latitude").value = coords[0];
  document.getElementById('Longitude').value = coords[1];

  return coords;
}

Any insight is appreciated.

Edit: I'm currently also going back and forth with firebase support, and apparently my code successfully queries the example database the rep built and functions correctly.

That seems to leave some kind of issue with the database structure, but...

  • I'm not querying to a different collection
  • All documents have the same fields with the same types
  • The testing I've done doesn't seem to allow the possibility of a type mismatch in the query

This is probably my ignorance talking, but the problem seems pretty bizarre.

Jacob M.
  • 31
  • 4
  • Please amend your question and show the actual query that you end up running. – Lee Taylor Aug 22 '19 at 17:56
  • 1
    Please edit the question to show the output of the log (since we can't see what NUM actually is). Also please describe the documents that you expect to match. Does the Number field actually a number type data? – Doug Stevenson Aug 22 '19 at 17:57
  • Unless I misunderstand you, the actual query is there. All of the components are optional so are inside if statements. Look for if (CTY)... as that's the first of them and the get() is below. The output varies depending on which address I'm looking up, but I'll add the example I usually use and the output. Sorry it wasn't included initially. – Jacob M. Aug 22 '19 at 18:00
  • @DougStevenson Is there anything else about the structure of the collection or the documents that I should add? – Jacob M. Aug 22 '19 at 20:00
  • I just realized I added the information for a database in a different project (is it Friday yet?). It's now corrected. – Jacob M. Aug 22 '19 at 20:25

1 Answers1

2

Firebase support is very helpful. This answer is thanks to the support rep that figured it out.

Thanks Carlos!

So it turns out my code was correct, there was a remarkably subtle problem with my data.

I had downloaded the data in this database from another database in csv format, verified the contents with Excel, and at some point saved it. Excel, being Excel, decided that any good UTF-8 encoded csv file needs to begin with a Byte-Order-Mark.

For those who don't know (like I didn't), Byte-Order-Mark's are invisible characters (zero width) in the unicode character set. They'll show up as weird question mark symbols in some text editors, but usually just don't get displayed at all.

Some further reading.

Specifically, BOM's are used for specifying the endianness of a string of bytes so that they can be properly interpreted.

The BOM ended up being the first character in the name of the database Number field because that was the first column specified in the csv file and, counter-intuitively enough, it wasn't the value that wasn't matching, it was the field name.

There were a couple indications of this that I lacked the background to recognize:

  • The firestore console placing the "Number" field at the bottom of the lexicographically sorted list of fields
  • Number being the only field name surrounded by quotation marks in the firestore console

The solution was to either:

Update each and every entry in the collection (since the csv column name became the property name in my csv to json conversion, then the firestore field name when I uploaded),

or

Delete the collection entirely and use some kind of find and replace tool (VS Code's search did the job) to replace all of the BOM's with an empty string (simply not specifying a replacement value in VS Code search and hitting "replace all" worked).

If the data had some other problems with zero-width (or other undesirable) characters they could be filtered out with a whitelist filtering tool. Blacklisting would work in theory, but with a character set as large as unicode's it wouldn't be practical unless you were only filtering out zero-width characters.

Thanks to everyone who took a look!

Jacob M.
  • 31
  • 4