0

According to the documentation

AWS_S3_MAX_MEMORY_SIZE(optional; default is 0 do not roll over) The maximum amount of memory (in bytes) a file can take up before being rolled over into a temporary file on disk.

Can someone explain this a bit more? Is this a way I could throttle upload sizes? What does "being rolled over" refer to?

Thank you

Justin Shakergayen
  • 71
  • 7

1 Answers1

2

System memory is considered limited, while disk space is usually not (practically speaking). Storing a file in memory is a trade-off where you get better speed of access, but use up more of your memory to do so. Say you have a large file of 1GB, is it really worth using up so much memory just to access that file faster? Maybe if you have a lot of memory and that file is accessed very frequently, but maybe not. That is why there are configurable limits like this. At some point, the trade-off is not worth it.

"Rolling over" would refer to when the in-memory file has gone over the set limit, and then gets moved into file-on-disk.

Hymns For Disco
  • 7,530
  • 2
  • 17
  • 33
  • Thank you! This was very helpful. – Justin Shakergayen Jan 06 '21 at 20:24
  • One question though. How could I cap the size of an image so that if it was too big the upload would stop. I have a validator written in my models and forms but it reads the file first and then determines if the file is too big. Instead of "rolling over" could I end the transfer and send a warning? – Justin Shakergayen Jan 06 '21 at 20:30
  • @JustinShakergayen that is something I do not know. You can see this question, which has various different answers for how to do this. I'd recommend reading each of them https://stackoverflow.com/questions/2472422/django-file-upload-size-limit – Hymns For Disco Jan 06 '21 at 21:15