0

Setup:

  1. Django (2.2.18) and Postgres (9.5) in one server, same disk.
  2. Migrations are altering 1 table (adding and removing fields).

I'm not sure if my question is correct but I ran into a problem with my setup above while applying migrations. Error is from psycopg2:

DiskFull: could not extend file "base/167296/1842042": wrote only 4096 of 8192 bytes at block 161
HINT:  Check free disk space.

Current disk usage is 14G out of 20G. We resolved the issue by deleting a few files and running the migrations again. We observed that the disk usage temporarily increased to about 96% of 20G then went back to 14G. Now we need to do something so the issue doesn't happen again. My question is:

  • if we move the database to something like RDS, will it be enough?

I ask because I'm not sure if it's the database size that temporarily inflates or if it's django that creates temporary files on disk. In the second case, it would mean that moving the database will not actually solve the issue.

I'd appreciate references as I couldn't find anything on this on the django docs (or I'm blind).

munsu
  • 1,914
  • 19
  • 24
  • 1
    I don't know about Django. But DBMS like Postgres use transactions. In brief, these bind a bunch of operations together as sort of one big operation, that is either executed completely or not at all. For that different logs need to be kept to undo changes already done or redo them if they weren't yet done completely. And theses logs of course (temporarily) need space. Depending on how much is done in a transaction that can be quite something. And there might also be the need to temporarily flush out some data to the disk for certain operations. So your observation isn't that surprising. – sticky bit Feb 03 '21 at 03:50

1 Answers1

0

Please refer to https://stackoverflow.com/a/33769195/6733421.

As mentioned over there, if you have any default values or not null constraints in the new fields that you're adding, PostgreSQL does not do the operation in place. So, you can remove the defaults, constraints and create the column. Add them afterwards as mentioned in the above mentioned link.

As for the RDS question, There are 2 types which you can use. RDS Postgres and RDS Aurora.

RDS Aurora has a serverless option. If you choose that, you don't really need to worry about the storage

Sai Chander
  • 829
  • 1
  • 6
  • 15