4

I am trying to benchmarking task for some queries in RethinkDB. I really did not get good answer of a question Why RethinkDB's count() operation is so slow?

I have a query with 2GB of data:

r.db("2GB").table("table").between(40, r.maxval, {index:"price"})

The query is executed in 5 milliseconds But once I would like count the number items like 

r.db("2GB").table("table").between(40, r.maxval, {index:"price"}).count()

It took more than 6 seconds Every query that uses count operation is very slow. I have seen many issues in github but could not get exact reason.

Update: it's not just between() but all other like filter ....the count() is horribly slow

Jorge Silva
  • 4,574
  • 1
  • 23
  • 42
Prakash Thapa
  • 1,285
  • 14
  • 27

1 Answers1

5

When you call between you get back a cursor, which loads data off disk lazily as you iterate over it. So the amount of time necessary to return the cursor is the amount of time necessary to read the first batch of your data, not all of your data. count, at the other hand, has to look at the whole table before it can return, so it takes time proportional to the size of your table.

mlucy
  • 5,249
  • 1
  • 17
  • 21
  • Thank you, question is updated. that means every thing is sent as a cursor then use count? I thought all the queries are executed in server... – Prakash Thapa Jul 11 '15 at 22:42
  • 1
    Everything is executed on the server, but most stream operations (including `between` and `filter`) are lazy. When you type `r.table('test').FOO().run(...)` where `FOO` is a lazy operation, the `run` returns as soon as the first batch of data is available; it doesn't load the whole table off disk at once. It will load the rest of the data as you iterate over the cursor so that it doesn't keep too much in memory at once. So you're comparing a query that returns as soon as it's loaded a very small amount of data to a query that doesn't return until it's loaded all the data in your table. – mlucy Jul 11 '15 at 22:58
  • Thank you @mlucy, now I am completely satisfied with the question that I was asking for more than one and half year !!! – Prakash Thapa Jul 13 '15 at 06:57