0

I'm working with DocumentDB in AWS, and I've been having troubles when I try to read from the same collection simultaneously from different aggregation queries.

The issue is not that I cannot read from the database, but rather that it takes a lot of time to complete the queries. It doesn't matter if I trigger the queries simultaneously or one after the other.

I'm using a Lambda Function with NodeJS to run my code. And I'm using mongoose to handle the connection with the database.

Here's a sample code that I put together to illustrate my problem:

query1() {
     return Collection.aggregate([...])
}
 
query2() {
  return Collection.aggregate([...])
}

query3() {
  return Collection.aggregate([...])
}

It takes the same time if I run it using Promise.all

Promise.all([ query1(), query2(), query3() ])

Than if I run it waiting for the previous one to finish

query1().then(result1 => query2().then(result3 => query3()))

While if I run each query in different Lambda Executions, it takes significantly less time for each individual query to finish (Between 1 and 2 seconds).

So if they were running in parallel the execution should be finished with the time of the query that takes the most time (2 seconds), and not take 7 seconds, as it does now.

So my guessing is that the instance of DocumentDB is running the queries in sequence no matter how I send them. In the collection there are around 19,000 documents with a total size of almost 25Mb.

When I check the metrics of the instance, the CPUUtilization is barely over 8% and the RAM available only drops by 20Mb. So I don't think the problem of the delay has to do with the size of the instance.

Do you know why DocumentDB is behaving like this? Is there a configuration that I can change to run the aggregations in parallel?

Juan Jouglard
  • 3
  • 3
  • This appears to be how the Javascript promise works. This [SO post](https://stackoverflow.com/questions/30823653/is-node-js-native-promise-all-processing-in-parallel-or-sequentially) describes might help in addressing what you are observing – vmachan Jun 17 '21 at 15:59
  • From my understanding, when you use Promise.all, the promises are run in parallel. What the Promise.all method does is that it just waits for all of them to finish. That should not affect the execution time of each individual query. Thanks for the reply! – Juan Jouglard Jun 17 '21 at 17:50
  • Have you tried your code using MongoDB? Would help diagnose where the issue is. – tmcallaghan Jul 02 '21 at 16:48

0 Answers0