Database BackupManager

[jfldde]

Description

• Introduces new BackupManager to backup all databases required by a citrea node.
• Introduces two new RPCs:
  • backup_create <path> to create a node backup at path
  • backup_validate <path> to validate a node backup at path
  • backup_info <path> to get info about backup at path
• Adds new --restore-db <path> arg to restore a node from a backup at path. TBD

Linked Issues

• Fixes RocksDB backups through RPC #1641

Depends on #1714

[eyusufatik]

does this mean taking backups would lock us here and we won't be able to produce blocks?

[eyusufatik]

from my understanding coupled with WAL rocksdb::backup::BackupEngine doesn't really need you to lock the database.

once concern here maybe would be the state_db and native_db are expected to always be in the same version so we have to guarantee somehow they are backed up at the same point in time -- which is not really possible.

is that the case?

if so, we'll soon be working on fixing this incompatibility problem.

[jfldde]

This is to err on the safe side and guarantee that all databases are at an initial l2 block state.
The lock is held to access l2 block height and spawn the backup tasks in this safe state and then released. Block production should tick along while backup is on-going

[codecov]

Codecov Report

Attention: Patch coverage is 25.15464% with 363 lines in your changes missing coverage. Please review.

Project coverage is 76.3%. Comparing base (1265a26) to head (6f32feb).
Report is 1 commits behind head on nightly.

Files with missing lines	Patch %	Lines
crates/common/src/backup/manager.rs	20.0%	156 Missing ⚠️
crates/common/src/backup/utils.rs	0.0%	70 Missing ⚠️
crates/common/src/backup/rpc.rs	20.4%	66 Missing ⚠️
bin/citrea/src/rollup/mod.rs	61.1%	26 Missing ⚠️
...overeign-sdk/full-node/db/sov-schema-db/src/lib.rs	0.0%	14 Missing ⚠️
...reign-sdk/full-node/db/sov-db/src/ledger_db/mod.rs	0.0%	6 Missing ⚠️
...es/sovereign-sdk/full-node/db/sov-db/src/mmr_db.rs	0.0%	6 Missing ⚠️
...sov-prover-storage-manager/src/snapshot_manager.rs	0.0%	6 Missing ⚠️
crates/light-client-prover/src/da_block_handler.rs	0.0%	5 Missing ⚠️
crates/light-client-prover/src/runner.rs	0.0%	3 Missing ⚠️
... and 2 more

Additional details and impacted files

Files with missing lines	Coverage Δ
bin/citrea/src/rollup/bitcoin.rs	0.0% <ø> (ø)
bin/citrea/src/rollup/mock.rs	77.7% <100.0%> (+0.7%)	⬆️
crates/batch-prover/src/da_block_handler.rs	74.2% <100.0%> (+0.2%)	⬆️
crates/batch-prover/src/runner.rs	86.0% <100.0%> (+0.1%)	⬆️
crates/common/src/config.rs	94.8% <100.0%> (+<0.1%)	⬆️
crates/fullnode/src/runner.rs	82.3% <100.0%> (+0.1%)	⬆️
crates/sequencer/src/runner.rs	88.3% <100.0%> (+<0.1%)	⬆️
...dk/full-node/sov-prover-storage-manager/src/lib.rs	97.4% <100.0%> (+<0.1%)	⬆️
...ule-system/sov-modules-rollup-blueprint/src/lib.rs	100.0% <ø> (ø)
bin/citrea/src/main.rs	0.0% <0.0%> (ø)
... and 11 more

... and 2 files with indirect coverage changes