3

Context

I have a list of items and a list of stores. I need to ask each store which of the items they have.

image showing scheme

I know that, according to Frank van Puffelen, parallel queries shouldn't take long. In fact, it should take the same amount as a regular one, in which "the time it takes to run a query is proportional to the number of results you get back, not the number of docs you're searching through."

Problem

However, the bigger the number of items in the list (hence, the bigger the number of queries executed), the longer it takes to get results back. I'm getting the following durations for each quantity of items:

Table describing durations

How I'm doing it

In order to get all the results at the same time, I'm using a DispatchGroup. I iterate through each store asking which items they have, and in each store I iterate through the list querying for such item in the store.

func fetchStoresProducts() {

    let productSearchGroup = DispatchGroup()

    for i in 0...(comparisonResults.count - 1) {
      productSearchGroup.enter()
      DataService.instance.getPrices(ofList: state.shoppingList.items, inStore: comparisonResults[i].store) { (productsFound) in
        self.comparisonResults[i].products = productsFound
        productSearchGroup.leave()
      }
    }

    productSearchGroup.notify(queue: .main) {
      // continue with next steps
    }

  }

And in the getPrices function, I query for each item considering its conditions:

func getPrices(ofList list: [Item], inStore store : Store, completion: @escaping (_ productsFound : [Product]) -> ()) {

    var products = [Product]()
    let listGroup = DispatchGroup()

    list.forEach { (item) in
      listGroup.enter()

      var query = historyRef.whereField("genericName", isEqualTo: item.genericName)
      query = query.whereField("storeID", isEqualTo: store.uid)

      if let type = item.genericType {
        query = query.whereField("genericType", isEqualTo: type)
      }

      if item.brandSearchOption == nil {
        if let brand = item.brand {
          query = query.whereField("brand", isEqualTo: brand)
        }
      }

      query = query.order(by: "price").limit(to: 1)

      query.getDocuments { (snapshot, error) in

        if let snapshot = snapshot {
          if let document = snapshot.documents.first {
            if var product = Product(data: document.data(), documentID: document.documentID) {
              product.setQuantity(to: item.quantity)
              products.append(product)
            }
          }
        } else {
          print(error!.localizedDescription)
        }

        listGroup.leave()
      }
    }

    listGroup.notify(queue: .main) {
      completion(products)
    }

  }

Notice that each of these queries has a limit of 1, which should give the fastest possible query duration. And they are also being executed together, almost in parallel - I don't wait for a result to ask for the next one.

Question

What am I doing wrong? Why is the querying time increasing proportionally to the number of items in the list?

Update

Interesting fact: if I change the historyRef for a smaller collection storesRef (with 100 or so documents), maintaining all other variables (number of stores and number of items), the speed of the queries is lightining fast. It averages 1.3s for 16 items in the list.

Could this issue be related to the collection, then? The size of it or how many composite indexes it has? historyRef has 15k+ documents with 10 composite indexes, storesRef has 100+ documents with 1 composite index.

rfarias
  • 1,296
  • 10
  • 15

0 Answers0