Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable pruning storage by size #1255

Open
wants to merge 39 commits into
base: mainnet-develop-0.4
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
f3dfb37
Update config templates
Alex6323 Mar 14, 2022
a485519
Refactor pruning config builder
Alex6323 Mar 14, 2022
1ffc253
Debug log actual and target storage size
Alex6323 Mar 21, 2022
fd46607
Put human-friendly sizes/durations in quotation marks in configs
Alex6323 Mar 21, 2022
5e54892
Log actual and target storage size in 'should_prune'
Alex6323 Mar 21, 2022
317be5e
Add pruning config unit test
Alex6323 Mar 30, 2022
553292f
Update docs in PruningSizeConfig
Alex6323 Apr 13, 2022
193a028
Prune milestones until final size is reached
Alex6323 Apr 13, 2022
b498e8e
Enable pruning-by-size; Cleanup
Alex6323 Apr 19, 2022
fde093c
Increase pruning index
Alex6323 Apr 19, 2022
ba3b93e
Log actual size during size pruning
Alex6323 Apr 19, 2022
356d2a6
Disable RocksDB WAL for pruning
Alex6323 Apr 20, 2022
0d88d22
Calculate batch byte size when pruning by size
Alex6323 Apr 20, 2022
9d4e8fb
Impl cooldown
Alex6323 Apr 20, 2022
b2f5f74
Format
Alex6323 Apr 20, 2022
f42b599
Clippy
Alex6323 Apr 20, 2022
dea6821
Prevent pruning-by-size to prune too close to ledger index
Alex6323 Apr 21, 2022
56293b1
Add excess to num_bytes_to_prune
Alex6323 Apr 21, 2022
dacd509
Address some todos
Alex6323 Apr 21, 2022
abc1ef9
Format
Alex6323 Apr 22, 2022
1724931
Cleanup
Alex6323 Apr 22, 2022
1ade79a
Const gen param to fn param
Alex6323 Apr 25, 2022
4cf4f73
Fix intra-doc links
Alex6323 Apr 25, 2022
87b6d55
Document unwraps
Alex6323 Apr 25, 2022
74aac6b
Make humanize-rs optional with no default features
Alex6323 Apr 25, 2022
3065524
Small renaming of enum variants
Alex6323 Apr 25, 2022
6d68c57
Make pruning options not mutually exclusive
Alex6323 Apr 26, 2022
c4cc8cf
Refactor snapshotting config and skip reasons
Alex6323 Apr 26, 2022
9fe5016
Change log
Alex6323 Apr 26, 2022
b26b28a
Better skip reason naming and consistent messages
Alex6323 Apr 26, 2022
3a124f4
Fix doc
Alex6323 Apr 26, 2022
1117a99
Better panic explanations
Alex6323 Apr 29, 2022
54687d0
previous instead of last
Alex6323 Apr 29, 2022
74b9995
Underflow panic doc
Alex6323 Apr 29, 2022
37ca08a
Reorder struct fields
Alex6323 Apr 29, 2022
54b34aa
Address some comments
Alex6323 May 5, 2022
e019d88
Add boxing helper to PruningError enum; remove redundant closures
Alex6323 May 6, 2022
9f6590e
Applied Christian awesomeness
Alex6323 May 6, 2022
680540c
Rename param
Alex6323 May 6, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Address some comments
  • Loading branch information
Alex6323 committed May 9, 2022
commit 54b34aaccc77f928c26a7f09e23630bc31640394
2 changes: 1 addition & 1 deletion bee-ledger/src/workers/pruning/batch.rs
Original file line number Diff line number Diff line change
@@ -363,7 +363,7 @@ fn prune_edge<S: StorageBackend>(
let mut byte_length = 0usize;

if by_size {
byte_length += edge.0.packed_len() + edge.1.packed_len();
byte_length += edge.packed_len();
}

Batch::<(MessageId, MessageId), ()>::batch_delete(storage, batch, edge)
6 changes: 1 addition & 5 deletions bee-ledger/src/workers/pruning/condition.rs
Original file line number Diff line number Diff line change
@@ -65,11 +65,7 @@ pub(crate) fn should_prune<S: StorageBackend>(
} else {
let actual_size = {
if let Ok(size) = storage.size() {
if let Some(size) = size {
Ok(size)
} else {
Err(PruningSkipReason::SizeMetricUnavailable)
}
size.ok_or(PruningSkipReason::SizeMetricUnavailable)
} else {
Err(PruningSkipReason::SizeMetricUnsupported)
}