0

I have a big database on .txt file, I'd like to know what's faster, reading the data from the file in every time I want to access to it or loading all the data into variables when the program starts so I can access to it from the variable it self.

PS: perhaps it's important to know that I code in java, but it's more a general question.

John
  • 13
  • 5
  • 2
    Probably will be extended by someone else's answer, but under most if not all circumstances, a single read to memory (a.k.a assign values to variables) done from a single read to the file is much faster than reading the file every time you need a variable. You can set up an example yourself easily. The only exception I can image is if you will only read the file once anyway, for example if you only need one value once and will never open that file again. And even then, you will somehow read that value to a variable anyway.. – HaroldH Jan 10 '21 at 21:10
  • 1
    Does this answer your question? [How much faster is the memory usually than the disk?](https://stackoverflow.com/questions/1371400/how-much-faster-is-the-memory-usually-than-the-disk) – Ole V.V. Jan 10 '21 at 21:15

2 Answers2

2

Assume for a second you have perfect recall, but you can choose to forget.

Now, what is faster?

  1. Read the book remembering it all, then recall from memory as you need information.

  2. Don't read the book. When you need some information, skim the book, looking only for the small piece of information you need and forgetting the rest. When you need more information, skim the book again, but you don't even know where in the book the information is, since you remember none of it, so every time you need to start reading the book from the beginning.

Obviously, #1 is way, WAY faster. Sure it requires your brain to be able to remember it all, but performance-wise, there is no comparison at all.

Exception: If you only ever need one piece of information, #2 will be faster, since you can stop reading as soon as you find the information you need, i.e. you don't have to read the whole book.

Andreas
  • 154,647
  • 11
  • 152
  • 247
0

Short answer

Variable

Long answer

Reading a file is quite a slow operation. It involves accessing your disk, which is noticeably slower than accessing a variable you already have in memory. And beware that when you read the file you'll need to store it somewhere in memory, so you'll pay as well the time for accessing memory

You can play around with some examples reading a file and measuring how long did it took. Remember to run the code several times so you get a more accurate result

Other point you should consider is the usage you're doing of your DB. If you're just storing a couple of values, it's fine to go for a txt file. But as soon as your storage layer gets more complex you'll probably need a proper DB (e.g. MySQL, DynamoDB, Mongo)

Alberto S.
  • 7,409
  • 6
  • 27
  • 46