0

I am writing an application to read from a list of files, line by line and do some processing. I want to use as little RAM as I can. I came across this question https://stackoverflow.com/a/41741702/3531263

Where the poster is saying readString uses more RAM than readLine and they have posted some code. What I don't understand is how one uses more RAM? Because ultimately, the way their code is written, they are still writing an entire line to their buffer. So would that not mean if they had just used readString, it would have been the same thing?

user3531263er
  • 423
  • 1
  • 4
  • 20

2 Answers2

4

the way their code is written, they are still writing an entire line to their buffer

Their code, yes. Your code might not need the whole line to be in memory at the same time. For example, your program is filtering a log file by request id, which is in the beginning of the line. It doesn't need to read the whole line which may be a few megabytes or more, only to reject it due to wrong request id. But with ReadString you don't have the luxury of choice.

Sergio Tulentsev
  • 226,338
  • 43
  • 373
  • 367
2

I 'gree with Sergio. Also, have a look at the current implementation in the standard library. ReadLine calls ReadSlice('\n') once, then runs through a few branches to make sure the appropriate sentinel values or errors are returned with the converted data. On the other hand, ReadBytes and ReadString both loop over repeated calls to ReadSlice(delim), so it follows that they would necessarily be copying at least as much data into memory as ReadLine, and potentially much more if the delimiter wasn't found in the first call.

Jesse Amano
  • 800
  • 5
  • 16