-1

We are trying to overhaul a scheduling application at the moment. It is written in Python/Django and using DRF to power a React frontend.

I just have a quick question- apologies if this has been answered already.

I have seen Dietrich Epp's answer to this problem on this thread.

I am just wondering if I have to check if a time is between two datetime objects over 100k records, what the fastest way to achieve this is?

I have considered indexing all of the datetimes in Haystack so that Elasticsearch can deal with the searching but do not want to overcomplicate if it can be solved simply.

Thanks all!

user2288983
  • 63
  • 6

1 Answers1

0

In my opinion, there is no need for haystack, you don't need overcomplicate things. With a database like postgresql or mysql 100k records are not so much.

A query like that should resolve in database in less than 0.1 second, and faster using an index, you should only try to improve the performance if the query is going to be executed more than once a second, so:

  • just leverage the work to the database
  • try to create an index on the date column in database
  • try not to do it on python
  • and be certain not doing 100k queries (common issue in django)

Use this approach https://stackoverflow.com/a/4668718/912450

jperelli
  • 6,988
  • 5
  • 50
  • 85
  • If you include a link to another question that is exactly the same - just mark this question as a duplicate... – Dekel Aug 23 '17 at 00:11
  • I would do that if I think is a duplicate, but I think it is not because of the 100k performance concern. – jperelli Aug 23 '17 at 00:12
  • Did he say he has a performance issue? Do you think that 100K records should cause any performance issue when searching for dates? – Dekel Aug 23 '17 at 00:13
  • If you read the question, he says: "what the fastest way [...]?". Reading it, clearly he has concerns about performance. The link to the other answer is accessory information, the answer to his question is all the other words I wrote before – jperelli Aug 23 '17 at 00:16
  • Also, obviously, feel free to flag the question, and/or delete it, you have the rep – jperelli Aug 23 '17 at 00:17
  • It is definitely a performance concern as this comparison on the 100k records will need to be done a dozen times per hour. – user2288983 Aug 23 '17 at 00:33
  • I edited the answer to let you know, a dozen of times per hour is not so much. You'll need to improve performance if this is a 1/sec or more query. – jperelli Aug 23 '17 at 00:39