Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce size of core database #2215

Closed
mikerkelly opened this issue Dec 10, 2024 · 1 comment
Closed

Reduce size of core database #2215

mikerkelly opened this issue Dec 10, 2024 · 1 comment

Comments

@mikerkelly
Copy link
Contributor

We are currently have around 50GB free on dokku3. The core OpenCodelists database is 4.1GB, zippable to 1.1GB in backups. #2151 seems to increase the space requirement for backups by about 20GB. We should consider vacuuming](https://sqlite.org/lang_vacuum.html) the database or other approaches to reducing its size

We might also consider increasing the space available on dokku3 if that's more cost-effective.

Any work should be done locally on a backup to evaluate effectiveness before considering restoring the prod DB from the reduced-size backup.

A quick experiment with vacuuming seems to show around a ~5% space saving:

sqlite3 db.sqlite3 ".backup /storage/tmp/test_backup_db.sqlite3"
ls -lh tmp/test_backup_db.sqlite3
-rw-r--r-- 1 10003 10003 4.1G Dec 10 08:58 tmp/test_backup_db.sqlite3
sqlite3 tmp/test_backup_db.sqlite3 "VACUUM"
ls -lh tmp/test_backup_db.sqlite3 
-rw-r--r-- 1 10003 10003 3.9G Dec 10 09:03 tmp/test_backup_db.sqlite3
rm tmp/test_backup_db.sqlite3 

As the coding systems databases are read-only I don't expect VACUUM will benefit them but it would be worth experimenting on a backup of one with some of the same techniques to see if they can benefit.

@lucyb
Copy link
Contributor

lucyb commented Dec 11, 2024

I'm closing this as I believe it will be made redundant by #2225 and a move over to Digital Ocean Block Storage.

@lucyb lucyb closed this as completed Dec 11, 2024
@lucyb lucyb closed this as not planned Won't fix, can't repro, duplicate, stale Dec 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants