1

At first some context. Currently I'm running some python scripts which are collecting some data from various sources. Since I expect to get a lot of data I'm a bit worried about how well my machine can handle big file sizes so I keep track of how the database evolves. At the current stage it should be no problem but I noticed that my main database (sqlite3) is not changing in size at all. After some research I found that the filesize might stay the same if the database had more entries before (Source), which most likely happended in the test stage of my scripts.

I'm backing up my database every day at midnight and noticed that the size of the compressed zip file is getting smaller every day. I'm using a shell script for the backup:

zip -r /backup/$(date +\%Y-\%m-\%d).zip /data

The directory /data contains a few other small files, which should not have been modified in any of the compressed zip files.

Why is the file size of the compressed ZIP getting smaller?

Vadim Kotov
  • 8,084
  • 8
  • 48
  • 62
Ian Fako
  • 1,148
  • 1
  • 15
  • 34

1 Answers1

2

If the database file is not growing when you add data, it means that SQLite is reusing free space : database pages which contained rows that were later deleted. These pages are not erased but only marked as free. SQLite does not care about the content (and will eventually overwrite it) but zip still archive everything.

It is possible that the data added are better compressed than the unused data they overwrite.

bwt
  • 17,292
  • 1
  • 42
  • 60