feat(tool): forest-tool api generate-test-snapshot #5074

hanabi1224 · 2024-12-11T12:45:31Z

Summary of changes

This PR includes changes in #5034

Changes introduced in this pull request:

forest-tool api generate-test-snapshot
forest-tool api test
a few RPC unit tests (to be expanded with the manual steps below)

(Will use a separate PR to add below steps to dev docs)

Step 1: generating RPC test dumps

forest-tool api compare --dump-dir /var/tmp/rpc-dumps ...

Step 2: generate RPC test snapshots

forest-tool api generate-test-snapshot --db ~/.local/share/forest/calibnet/0.23.0 --chain calibnet --out-dir /var/tmp/rpc-snapshots /var/tmp/test-dumps/filecoin_stategetallallocations*.json

Step 2(b): zstd compress test snapshots

cd /var/tmp/rpc-snapshots
zstd *.json

Step 3: verify test snapshots

forest-tool api test /var/tmp/rpc-snapshots/*.json or forest-tool api test /var/tmp/rpc-snapshots/*.zst

Step 3(b): update unit test

upload test snapshots to DO space
update test_snapshots.txt
cargo test --lib -- --test rpc_regression_tests --nocapture

Reference issue to close (if applicable)

Closes #5034

Change checklist

I have performed a self-review of my own code,
I have made corresponding changes to the documentation. All new code adheres to the team's documentation standards,
I have added tests that prove my fix is effective or that my feature works (if possible),
I have made sure the CHANGELOG is up-to-date. All user-facing changes should be reflected in this document.

…m/forest-tool-api-generate-test-snapshot

src/tool/subcommands/api_cmd/generate_test_snapshot.rs

lemmih

The format of the memory-db is not stable. If someone changes the memory-db internals, this code will fail. We'll either have to deal with it right now or in the future. I vote to deal with it right now.

cloudflare-workers-and-pages · 2024-12-16T07:11:28Z

Deploying forest-docs with Cloudflare Pages

Latest commit:	`f69ad6e`
Status:	✅ Deploy successful!
Preview URL:	https://5597b3fc.forest-824.pages.dev
Branch Preview URL:	https://hm-forest-tool-api-generate.forest-824.pages.dev

View logs

hanabi1224 · 2024-12-16T08:28:18Z

The format of the memory-db is not stable. If someone changes the memory-db internals, this code will fail. We'll either have to deal with it right now or in the future. I vote to deal with it right now.

@lemmih Switched to Forest CAR DB as discussed.

…nerate-test-snapshot

LesnyRumcajs · 2025-01-09T12:49:07Z

src/db/memory.rs

+    pub async fn export_forest_car<W: tokio::io::AsyncWrite + Unpin>(
+        &self,
+        writer: &mut W,
+    ) -> anyhow::Result<()> {
+        let roots =
+            SettingsStoreExt::read_obj::<TipsetKey>(self, crate::db::setting_keys::HEAD_KEY)?
+                .context("chain head is not tracked and cannot be exported")?
+                .into_cids();
+        let blocks = {
+            let blockchain_db = self.blockchain_db.read();
+            let blockchain_persistent_db = self.blockchain_persistent_db.read();
+            blockchain_db
+                .iter()
+                .chain(blockchain_persistent_db.iter())
+                .map(|(&cid, data)| {
+                    anyhow::Ok(CarBlock {
+                        cid,
+                        data: data.clone(),
+                    })
+                })
+                .collect_vec()
+        };
+        let frames =
+            crate::db::car::forest::Encoder::compress_stream_default(futures::stream::iter(blocks));
+        crate::db::car::forest::Encoder::write(writer, roots, frames).await
+    }
+}


Can we contrive a unit test for this functionality?

Sure, unit test added.

LesnyRumcajs · 2025-01-09T12:51:15Z

src/tool/subcommands/api_cmd.rs

+        #[arg(num_args = 1.., required = true)]
+        test_dump_files: Vec<PathBuf>,


Let's have docs for this as well.

LesnyRumcajs · 2025-01-09T12:51:37Z

src/tool/subcommands/api_cmd.rs

+        #[arg(num_args = 1.., required = true)]
+        files: Vec<PathBuf>,


Let's have docs for this as well.

LesnyRumcajs · 2025-01-09T13:01:23Z

src/tool/subcommands/api_cmd/generate_test_snapshot.rs

+        CurrentNetwork::set_global(Network::Testnet);
+    }
+    let mut run = false;
+    let chain_config = Arc::new(ChainConfig::calibnet());


The chain config should depend on the chain parameter, no?

Good catch, fixed!

LesnyRumcajs · 2025-01-09T13:03:50Z

src/tool/subcommands/api_cmd/generate_test_snapshot.rs

+    }
+
+    fn setting_keys(&self) -> anyhow::Result<Vec<String>> {
+        // HACKHACK: may need some care


It would be great to elaborate on why the care might be needed as a comment.

Resolved or removed all HACKHACKs, methods that read settings are no longer supported after migrating DB snapshot to CAR format.

LesnyRumcajs · 2025-01-09T13:11:53Z

src/tool/subcommands/api_cmd/test_snapshot.rs

+    #[tokio::test]
+    async fn rpc_regression_tests() {
+        let urls = include_str!("test_snapshots.txt")
+            .trim()
+            .split("\n")
+            .filter_map(|n| {
+                Url::parse(
+                    format!("https://forest-snapshots.fra1.digitaloceanspaces.com/rpc_test/{n}")
+                        .as_str(),
+                )
+                .ok()
+            })
+            .collect_vec();
+        for url in urls {
+            print!("Testing {url} ...");
+            let tmp_dir = tempfile::tempdir().unwrap();
+            let tmp = tempfile::NamedTempFile::new_in(&tmp_dir)
+                .unwrap()
+                .into_temp_path();
+            println!("start downloading at {}", tmp.display());
+            download_to(&url, &tmp).await.unwrap();
+            println!("done downloading {}", tmp.display());
+            run_test_from_snapshot(&tmp).await.unwrap();
+            println!("  succeeded.");
+        }


If I understand correctly, each test invocation will temporarily download the snapshots. The total snapshot size currently is around 4 MB, but I can easily see it going higher once we add more snapshots. This will be a pain for someone on a worse connection just trying to run tests.

Perhaps we could add something like /.local/share/forest-dev/rpc-regression-snapshots persistent path (it would vary from system to system, we could re-use the logic we have for the DB).

Cache implemented.

LesnyRumcajs

Great stuff

…nerate-test-snapshot

hanabi1224 added 4 commits December 4, 2024 23:15

test(rpc): rpc unit tests with db snapshot

f5d98d7

Merge branch 'main' into hm/rpc-snaphsot-unittest

4df1c11

Merge branch 'main' into hm/rpc-snaphsot-unittest

6edcc28

Merge branch 'main' into hm/rpc-snaphsot-unittest

008ca0d

hanabi1224 force-pushed the hm/forest-tool-api-generate-test-snapshot branch from 7696c1a to f496aed Compare December 11, 2024 15:10

feat(tool): forest-tool api generate-test-snapshot

91e4716

hanabi1224 force-pushed the hm/forest-tool-api-generate-test-snapshot branch from f496aed to 91e4716 Compare December 11, 2024 23:46

hanabi1224 added 4 commits December 12, 2024 08:15

fix build errors

2f72b9b

Merge remote-tracking branch 'origin/hm/rpc-snaphsot-unittest' into h…

895236d

…m/forest-tool-api-generate-test-snapshot

cc

fd80f98

update test cases

bd33685

hanabi1224 marked this pull request as ready for review December 12, 2024 09:28

hanabi1224 requested a review from a team as a code owner December 12, 2024 09:28

hanabi1224 requested review from lemmih and sudo-shashank and removed request for a team December 12, 2024 09:28

hanabi1224 mentioned this pull request Dec 12, 2024

test(rpc): rpc unit tests with db snapshot #5034

Closed

4 tasks

hanabi1224 changed the title ~~[WIP] feat(tool): forest-tool api generate-test-snapshot~~ feat(tool): forest-tool api generate-test-snapshot Dec 12, 2024

track EthMappingsStore

0c7e42e

hanabi1224 commented Dec 12, 2024

View reviewed changes

src/tool/subcommands/api_cmd/generate_test_snapshot.rs Outdated Show resolved Hide resolved

lemmih requested changes Dec 12, 2024

View reviewed changes

hanabi1224 added 2 commits December 12, 2024 17:45

for_each_rpc_method

35c273a

Merge branch 'main' into hm/forest-tool-api-generate-test-snapshot

75c7a6e

hanabi1224 marked this pull request as draft December 13, 2024 12:24

Merge branch 'main' into hm/forest-tool-api-generate-test-snapshot

e54bc5e

hanabi1224 marked this pull request as ready for review December 16, 2024 08:27

switch to foreset CAR db

18089eb

hanabi1224 force-pushed the hm/forest-tool-api-generate-test-snapshot branch from 827e63a to 18089eb Compare December 16, 2024 08:31

hanabi1224 added 3 commits December 16, 2024 22:09

Merge branch 'main' into hm/forest-tool-api-generate-test-snapshot

f69ad6e

Merge remote-tracking branch 'origin/main' into hm/forest-tool-api-ge…

a70ee05

…nerate-test-snapshot

set_heaviest_tipset

1f2f7ff

hanabi1224 requested a review from elmattic December 17, 2024 01:57

hanabi1224 added 3 commits January 7, 2025 08:36

Merge remote-tracking branch 'origin/main' into hm/forest-tool-api-ge…

b4282ce

…nerate-test-snapshot

Merge remote-tracking branch 'origin/main' into hm/forest-tool-api-ge…

fb552dc

…nerate-test-snapshot

fix copyright headers

376ab9e

LesnyRumcajs reviewed Jan 9, 2025

View reviewed changes

hanabi1224 added 6 commits January 9, 2025 22:54

Merge remote-tracking branch 'origin/main' into hm/forest-tool-api-ge…

9a410e2

…nerate-test-snapshot

fix hackhack

f20cb1d

construct ChainConfig from NetworkChain

8ccbdd3

code docs

0e21f0f

test_export_forest_car

b98c0ae

local cache for rpc test snapshots

e3a7ebf

hanabi1224 requested review from LesnyRumcajs and lemmih January 10, 2025 10:42

switch to a more popular md5 crate

c469b88

hanabi1224 force-pushed the hm/forest-tool-api-generate-test-snapshot branch from 75caaf6 to c469b88 Compare January 10, 2025 12:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(tool): forest-tool api generate-test-snapshot #5074

feat(tool): forest-tool api generate-test-snapshot #5074

hanabi1224 commented Dec 11, 2024 •

edited

Loading

lemmih left a comment

cloudflare-workers-and-pages bot commented Dec 16, 2024 •

edited

Loading

hanabi1224 commented Dec 16, 2024

LesnyRumcajs Jan 9, 2025

hanabi1224 Jan 10, 2025

LesnyRumcajs Jan 9, 2025

hanabi1224 Jan 10, 2025

LesnyRumcajs Jan 9, 2025

hanabi1224 Jan 10, 2025

LesnyRumcajs Jan 9, 2025

hanabi1224 Jan 10, 2025

LesnyRumcajs Jan 9, 2025

hanabi1224 Jan 10, 2025

LesnyRumcajs Jan 9, 2025

hanabi1224 Jan 10, 2025

LesnyRumcajs left a comment

		#[arg(num_args = 1.., required = true)]
		test_dump_files: Vec<PathBuf>,

feat(tool): forest-tool api generate-test-snapshot #5074

Are you sure you want to change the base?

feat(tool): forest-tool api generate-test-snapshot #5074

Conversation

hanabi1224 commented Dec 11, 2024 • edited Loading

Summary of changes

Step 1: generating RPC test dumps

Step 2: generate RPC test snapshots

Step 2(b): zstd compress test snapshots

Step 3: verify test snapshots

Step 3(b): update unit test

Reference issue to close (if applicable)

Other information and links

Change checklist

lemmih left a comment

Choose a reason for hiding this comment

cloudflare-workers-and-pages bot commented Dec 16, 2024 • edited Loading

Deploying forest-docs with Cloudflare Pages

hanabi1224 commented Dec 16, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LesnyRumcajs left a comment

Choose a reason for hiding this comment

hanabi1224 commented Dec 11, 2024 •

edited

Loading

cloudflare-workers-and-pages bot commented Dec 16, 2024 •

edited

Loading