Merge main to nomium-main (#22)

* tp_authority_public_key update * lock version for cargo semver-checks * Use shorter relative path .. and fix some styling inconsistency * Rewrite docs in `common_messages_sv2` .. As part of the effort to improve Stratum V2 protocols docs, this commit aims to improves and make the documentation more comprehensive and accessible for contributors and end users alike. * Add README.md .. Use the template README used across the different Stratum V2 protocol crates to `common_messages_sv2` crate. * modify sv2.h to make it consistent with comments changes * Use shorter relative path * Add README file * Improve Job Declaration Documentation * Add README file * Use shorter paths * Improve `Template Distribution` subprotocol docs * modify sv2.h to make it consistent with comments changes * add binary-sv2 docs * add binary-sv2 readme.md * add binary-sv2 custom trait example * add binary-sv2 no-serde derive-codec docs * add binary-sv2 no-serde derive-codec readme * add binary-sv2 no-serde codec readme * add binary-sv2 no-serde codec docs * update sv2.h for binary_sv2 docs * change doc test in derive_codec * Minor warning and error grammar and typo fixes * bump jdc * Slice docs * Buffer docs * Lib WriteError, Write, Buffer docs * Back buffer pool docs * Front buffer pool docs * InnerMemory buffer pool docs * PoolMode docs * BufferPool docs * Top buffer_pool mod docs * Buffer pool examples * Top level crate docs + clean up * Update README * Sniffer::wait_for_message_type * fix unwrap on sv1-mining-device tcp connection.. Update roles/test-utils/mining-device-sv1/src/client.rs Co-authored-by: jbesraa <[email protected]> * remove redundant Drop implementation from ITF TemplateProvider * Add `translator_sv2` test * use macos-14 as macos runner * disable MG tproxy CI * Have JDS ping local mempool less frequently Otherwise Bitcoin Core -debug=rpc logging becomes too noisy. * Use shorter paths * Add README mining subprotocol * Rewrite mining subprotocol docs * Create a lib for `mining-sv2-proxy` * Add `mining-sv2-proxy` initializer * lock semver-checks to 37 while replacing MSRV with stable * bump serde_v2 major to 2.0.0.. since v37, cargo semver-checks enforces that removal of features are breaking changes therefore we need to update serde_v2 as a follow up to stratum-mining#1230 * bump binary_sv2 for serde_sv2 dependency.. since we bumpbed serde_sv2 to 2.0.0 * bump sv2_ffi for serde_sv2 dependency.. since we bumpted serde_sv2 to 2.0.0 * bump roles/Cargo.lock.. since we bumped serde_sv2 and binary_sv2 * bump const_sv2 major to 3.0.0.. since v37, cargo semver-checks enforces that removal of features are breaking changes therefore we need to update serde_v2 as a follow up to stratum-mining#1230 * bump noise_sv2 for const_sv2 dependency.. since we bumpbed const_sv2 to 3.0.0 * bump framing_sv2 for const_sv2 dependency.. since we bumpbed const_sv2 to 3.0.0 * bump codec_sv2 for const_sv2 dependency.. since we bumpbed const_sv2 to 3.0.0 * bump subprotocols for const_sv2 dependency.. since we bumpbed const_sv2 to 3.0.0 * bump sv2_ffi for const_sv2 dependency.. since we bumpbed const_sv2 to 3.0.0 * bump roles_logic_sv2 for const_sv2 dependency.. since we bumpbed const_sv2 to 3.0.0 * bump roles due to const_sv2 bump to 3.0.0 * bump framing_sv2 major to 3.0.0.. since v37, cargo semver-checks enforces that removal of features are breaking changes therefore we need to update framing_v2 as a follow up to stratum-mining#1230 * bump codec_sv2 for framing_sv2 dependency.. since we bumped framing_sv2 to 3.0.0 * bump framing_sv2 dendency version on roles_logic_sv2.. crate version has already been bumped before release. no need to bump again * bump roles due to framing_sv2 bump * bump benches due to framing_sv2 bump * bump message_generator_sv2 due to const_sv2 * change macos-14 to macos-13 * bump buffer_sv2.. as a follow up to stratum-mining#1230 * bump buffer_sv2 dep on roles/Cargo.lock * bump buffer_sv2 dep on utils/Cargo.lock * bump derive_codec_sv2.. as a consequence of stratum-mining#1231 * bump derive_codec_sv2 dep on roles/Cargo.lock * Add framing sv2 frame and handshake frame ex * /// -> // for private types+fns * framing mod doc cmts + clean * Add framing README * add codecov.yml to remove ci error * feat: update println! and eprintln! to info using tracing cargo. * Wait for `NewTemplate` message to arrive instead of failing if it is not there yet when we first check * ci: bump TP to 0.1.13 * test: increase TP log timestamp precision Also log rpc calls. * `common_messages_sv2` `job_declaration_sv2` `mining_sv2` `template_distribution_sv2`: remove the `no_std` feature and make them `#![no_std]` as they never use `std` anywhere. - bump their MAJOR version because of feature removal - bump the dependant crates PATCH version - updates docs * fix: clarify logging * remove readme field from Cargo.toml of crates that dont have a README.md * add release-libs.sh * release-libs.yaml use 1.75.0 toolchain * remove readme field from Cargo.toml of crates that dont have a README.md * avoid publishing protocols crates with all-features.. publishing with all-features is overengineering: - no_std is being deprecated - with_serde will be deprecated soon - for crates with relevant features, we specifically list them for publishing * min_ntime fix to use the one sent by TP * header_timestamp_value_assertion_in_new_extended_mining_job test addition * bump PATCH after timestamp-bug fix (stratum-mining#1324) * update lockfile after roles_logic patch * Test Pool role behavior if bad `coinbase_output`.. is provided * Remove bad pool config MG test * Move sniffer test to a separate file * Derive Clone+Debug for `JobDeclaratorClient` * Derive Clone+Debug for `JobDeclaratorServer` * Align all integration test start_* .. functions return signature * Make Sniffer::drop output more verbose * Add README.md to integration-tests * Move integration tests to separate GH action * Rename integration tests crate.. it's important to have a descriptive name for when this is published to crates.io * Modularize integration tests APIs.. into the following modules: - lib: with general purpose functions (e.g.: starters) - sniffer - template_provider * Remove redundant `TestPoolSv2`.. this struct is redundant and no other roles follow this pattern we can do the initialization inside `start_pool` * Fix variable naming on `wait_for_client`.. the `SocketAddr` is not a client, but the socket where we will listen to also, `listner` is a typo * Rename sniffer channel variables.. the original naming was confusing copypasta * Create `utils.rs` in `tests-integration` ..to hold utility functions used internaly only. * noise_sv2: keep current (std) API unchanged, add a no_std compliant API with `*_with_rng` and `*_with_now` * generate cargo lock --------- Co-authored-by: GitGab19 <[email protected]> Co-authored-by: plebhash <[email protected]> Co-authored-by: plebhash <[email protected]> Co-authored-by: jbesraa <[email protected]> Co-authored-by: bit-aloo <[email protected]> Co-authored-by: Gabriele Vernetti <[email protected]> Co-authored-by: Pavlenex <[email protected]> Co-authored-by: RJ Rybarczyk <[email protected]> Co-authored-by: Sjors Provoost <[email protected]> Co-authored-by: devworlds <[email protected]> Co-authored-by: Georges Palauqui <[email protected]> Co-authored-by: Gary Krause <[email protected]> Co-authored-by: Oleg Akulov <[email protected]>
nomium-org · Jan 14, 2025 · 8303654 · 8303654
1 parent 71814c1
commit 8303654
Show file tree

Hide file tree

Showing 95 changed files with 2,228 additions and 1,691 deletions.
diff --git a/.github/workflows/integration-tests.yaml b/.github/workflows/integration-tests.yaml
@@ -0,0 +1,32 @@
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+    branches:
+      - main
+
+name: Integration Tests
+
+jobs:
+  ci:
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        os:
+          - ubuntu-latest
+        include:
+          - os: ubuntu-latest
+            target: x86_64-unknown-linux-musl
+
+    steps:
+      - name: Use stable toolchain
+        uses: actions/checkout@v4
+        with:
+          profile: minimal
+          toolchain: stable
+          override: true
+
+      - name: Roles Integration Tests
+        run: |
+          cargo test --manifest-path=roles/Cargo.toml --verbose --test '*' -- --nocapture
diff --git a/.github/workflows/mg.yaml b/.github/workflows/mg.yaml
@@ -11,16 +11,6 @@ on:
       - main
 
 jobs:
-  bad-pool-config-test:
-    runs-on: ubuntu-latest
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@v4
-      - name: Install cargo-llvm-cov
-        run: cargo install cargo-llvm-cov
-      - name: Run bad-pool-config-test
-        run: sh ./test/message-generator/test/bad-pool-config-test/bad-pool-config-test.sh
-
   interop-jd-translator:
     runs-on: ubuntu-latest
     steps:
@@ -167,7 +157,6 @@ jobs:
     runs-on: ubuntu-latest
     if: always()
     needs: [
-      bad-pool-config-test,
       interop-jd-translator,
       interop-proxy-with-multi-ups,
       interop-proxy-with-multi-ups-extended,
@@ -188,8 +177,7 @@ jobs:
     steps:
       - name: Aggregate Results
         run: |
-          if [ "${{ needs.bad-pool-config-test.result }}" != "success" ] ||
-          [ "${{ needs.interop-jd-translator.result }}" != "success" ] ||
+          if [ "${{ needs.interop-jd-translator.result }}" != "success" ] ||
           [ "${{ needs.interop-proxy-with-multi-ups.result }}" != "success" ] ||
           [ "${{ needs.interop-proxy-with-multi-ups-extended.result }}" != "success" ] ||
           [ "${{ needs.jds-do-not-fail-on-wrong-tsdatasucc.result }}" != "success" ] ||
@@ -209,4 +197,4 @@ jobs:
             exit 1
           else
             echo "All MG tests completed successfully"
-          fi
+          fi
diff --git a/.github/workflows/release-libs.yaml b/.github/workflows/release-libs.yaml
@@ -1,12 +1,8 @@
 # This workflow is used to publish SV2 crates to crates.io
-# The workflow tries to update all the library crates, so if a crate is not to updated, the step will fail
-# for that each step have continue-on-error set to true.
-# Since each step can fail, the output ot the action must be manually checked to make sure that all
-# the library intended to be published are published.
-# Running cargo release in the various workspace help to prepare the version number and everything.
-# ATTENTION
-# Is very important to check the output manually cause when too many crates are updated crates.io could fail
-# and ask to rerun the action later
+# the workflow tries to publish all the library crates by running scripts/release-libs.sh
+# in case the `cargo publish` command fails, the script returns 1 and the entire workflow fails
+# the only exception is when the `cargo publish` command fails because the crate has already
+# been published, in which case the workflow continues
 
 name: Release Libs
 
@@ -28,117 +24,95 @@ jobs:
       - uses: actions/checkout@v4
       - uses: actions-rs/toolchain@v1
         with:
-            toolchain: stable
+            toolchain: 1.75.0
             override: true
       - name: Login
         run: cargo login ${{ secrets.CRATES_IO_DEPLOY_KEY }}
+
       - name: Publish crate common
-        continue-on-error: true
         run: |
-          cd common
-          cargo publish
+          ./scripts/release-libs.sh common
+
       - name: Publish crate buffer_sv2
-        continue-on-error: true
         run: |
-          cd utils/buffer
-          cargo publish
+          ./scripts/release-libs.sh utils/buffer
+
       - name: Publish crate no_serde_sv2_derive_codec
-        continue-on-error: true
         run: |
-          cd protocols/v2/binary-sv2/no-serde-sv2/derive_codec
-          cargo publish
+          ./scripts/release-libs.sh protocols/v2/binary-sv2/no-serde-sv2/derive_codec
+
       - name: Publish crate no_serde_sv2_codec
-        continue-on-error: true
         run: |
-          cd protocols/v2/binary-sv2/no-serde-sv2/codec
-          cargo publish
+          ./scripts/release-libs.sh protocols/v2/binary-sv2/no-serde-sv2/codec
+
       - name: Publish crate serde_sv2
-        continue-on-error: true
         run: |
-          cd protocols/v2/binary-sv2/serde-sv2
-          cargo publish
+          ./scripts/release-libs.sh protocols/v2/binary-sv2/serde-sv2
+
       - name: Publish crate binary_sv2
-        continue-on-error: true
         run: |
-          cd protocols/v2/binary-sv2/binary-sv2
-          cargo publish
+          ./scripts/release-libs.sh protocols/v2/binary-sv2/binary-sv2
+
       - name: Publish crate const_sv2
-        continue-on-error: true
         run: |
-          cd protocols/v2/const-sv2
-          cargo publish
+          ./scripts/release-libs.sh protocols/v2/const-sv2
+
       - name: Publish crate framing_sv2
-        continue-on-error: true
         run: |
-          cd protocols/v2/framing-sv2
-          cargo publish
+          ./scripts/release-libs.sh protocols/v2/framing-sv2
+
       - name: Publish crate noise_sv2
-        continue-on-error: true
         run: |
-          cd protocols/v2/noise-sv2
-          cargo publish
+          ./scripts/release-libs.sh protocols/v2/noise-sv2
+
       - name: Publish crate codec_sv2
-        continue-on-error: true
         run: |
-          cd protocols/v2/codec-sv2
-          cargo publish
+          ./scripts/release-libs.sh protocols/v2/codec-sv2
+
       - name: Publish crate common_messages
-        continue-on-error: true
         run: |
-          cd protocols/v2/subprotocols/common-messages
-          cargo publish
+          ./scripts/release-libs.sh protocols/v2/subprotocols/common-messages
+
       - name: Publish crate job_declaration
-        continue-on-error: true
         run: |
-          cd protocols/v2/subprotocols/job-declaration
-          cargo publish
+          ./scripts/release-libs.sh protocols/v2/subprotocols/job-declaration
+
       - name: Publish crate mining
-        continue-on-error: true
         run: |
-          cd protocols/v2/subprotocols/mining
-          cargo publish
+          ./scripts/release-libs.sh protocols/v2/subprotocols/mining
+
       - name: Publish crate template_distribution
-        continue-on-error: true
         run: |
-          cd protocols/v2/subprotocols/template-distribution
-          cargo publish
+          ./scripts/release-libs.sh protocols/v2/subprotocols/template-distribution
+
       - name: Publish crate sv2_ffi
-        continue-on-error: true
         run: |
-          cd protocols/v2/sv2-ffi
-          cargo publish --all-features
+          ./scripts/release-libs.sh protocols/v2/sv2-ffi
+
       - name: Publish crate roles_logic_sv2
-        continue-on-error: true
         run: |
-          cd protocols/v2/roles-logic-sv2
-          cargo publish
+          ./scripts/release-libs.sh protocols/v2/roles-logic-sv2
+
       - name: Publish crate v1
-        continue-on-error: true
         run: |
-          cd protocols/v1
-          cargo publish
+          ./scripts/release-libs.sh protocols/v1
+
       - name: Publish crate bip32-key-derivation
-        continue-on-error: true
         run: |
-          cd utils/bip32-key-derivation
-          cargo publish
+          ./scripts/release-libs.sh utils/bip32-key-derivation
+
       - name: Publish crate error-handling
-        continue-on-error: true
         run: |
-          cd utils/error-handling
-          cargo publish
+          ./scripts/release-libs.sh utils/error-handling
+
       - name: Publish crate key-utils
-        continue-on-error: true
         run: |
-          cd utils/key-utils
-          cargo publish
+          ./scripts/release-libs.sh utils/key-utils
+
       - name: Publish crate network_helpers_sv2
-        continue-on-error: true
         run: |
-          cd roles/roles-utils/network-helpers
-          cargo publish
+          ./scripts/release-libs.sh roles/roles-utils/network-helpers
+
       - name: Publish crate rpc_sv2
-        continue-on-error: true
         run: |
-          cd roles/roles-utils/rpc
-          cargo publish
+          ./scripts/release-libs.sh roles/roles-utils/rpc
diff --git a/.github/workflows/semver-check.yaml b/.github/workflows/semver-check.yaml
@@ -16,10 +16,10 @@ jobs:
       - name: Checkout repository
         uses: actions/checkout@v2
 
-      - name: Install Rust 1.75.0
+      - name: Install Rust stable
         uses: actions-rs/toolchain@v1
         with:
-          toolchain: 1.75.0
+          toolchain: stable
           override: true
 
       - name: Cache Cargo registry
@@ -42,7 +42,7 @@ jobs:
         run: sudo apt-get update && sudo apt-get install -y cmake
 
       - name: Install cargo-semver-checks
-        run: cargo install cargo-semver-checks --version 0.33.0 --locked
+        run: cargo install cargo-semver-checks --version 0.37.0 --locked
 
       - name: Run semver checks for common
         working-directory: common

diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml
@@ -14,10 +14,10 @@ jobs:
     strategy:
       matrix:
         os:
-          - macos-14
+          - macos-13
           - ubuntu-latest
         include:
-          - os: macos-14
+          - os: macos-13
             target: x86_64-apple-darwin
           - os: ubuntu-latest
             target: x86_64-unknown-linux-musl
@@ -38,10 +38,6 @@ jobs:
           cargo build --manifest-path=roles/Cargo.toml
           cargo build --manifest-path=utils/Cargo.toml
 
-      - name: Roles Integration Tests
-        run: |
-          cargo test --manifest-path=roles/Cargo.toml --verbose --test '*' -- --nocapture
-
       - name: Run sv1-client-and-server example
         run: |
           cargo run --manifest-path=examples/sv1-client-and-server/Cargo.toml --bin client_and_server -- 60

diff --git a/benches/Cargo.toml b/benches/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "benchmark"
-version = "1.0.0"
+version = "1.0.1"
 edition = "2021"
 
 [dependencies]
@@ -10,9 +10,9 @@ async-channel = "1.4.0"
 v1 = { path="../protocols/v1", package="sv1_api", version = "^1.0.0" }
 serde_json = { version = "1.0.64", default-features = false, features = ["alloc"] }
 iai="0.1"
-mining_sv2 = { path = "../protocols/v2/subprotocols/mining", version = "^1.0.0" }
+mining_sv2 = { path = "../protocols/v2/subprotocols/mining", version = "^2.0.0" }
 roles_logic_sv2 = { path = "../protocols/v2/roles-logic-sv2", version = "^1.0.0" }
-framing_sv2 = { version = "2.0.0", path = "../protocols/v2/framing-sv2" }
+framing_sv2 = { version = "3.0.0", path = "../protocols/v2/framing-sv2" }
 serde = { version = "1.0.89", default-features = false, features = ["derive", "alloc"] }
 num-bigint = "0.4.3"
 num-traits = "0.2.15"

diff --git a/codecov.yaml b/codecov.yaml
@@ -0,0 +1,13 @@
+coverage:
+  status:
+    project:
+      default:
+        target: auto
+        threshold: 100%
+        base: auto
+        informational: false
+    patch:
+      default:
+        target: auto
+        threshold: 100%
+        base: auto
diff --git a/protocols/v2/binary-sv2/binary-sv2/Cargo.toml b/protocols/v2/binary-sv2/binary-sv2/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "binary_sv2"
-version = "1.2.0"
+version = "1.2.1"
 authors = ["The Stratum V2 Developers"]
 edition = "2018"
 readme = "README.md"
@@ -14,7 +14,7 @@ keywords = ["stratum", "mining", "bitcoin", "protocol"]
 # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
 
 [dependencies]
-serde_sv2 = {version = "^1.0.0", path = "../serde-sv2", optional = true}
+serde_sv2 = {version = "^2.0.0", path = "../serde-sv2", optional = true}
 serde = { version = "1.0.89", features = ["derive", "alloc"], default-features = false, optional = true }
 binary_codec_sv2 = {version = "^1.0.0", path = "../no-serde-sv2/codec", optional = true}
 derive_codec_sv2 = {version = "^1.0.0", path = "../no-serde-sv2/derive_codec", optional = true}
@@ -28,4 +28,4 @@ prop_test = ["binary_codec_sv2/prop_test", "derive_codec_sv2"]
 with_buffer_pool = ["binary_codec_sv2/with_buffer_pool", "derive_codec_sv2"]
 
 [package.metadata.docs.rs]
-all-features = true
+features = ["core", "with_buffer_pool"]
diff --git a/protocols/v2/binary-sv2/no-serde-sv2/codec/Cargo.toml b/protocols/v2/binary-sv2/no-serde-sv2/codec/Cargo.toml
@@ -24,4 +24,4 @@ prop_test = ["quickcheck"]
 with_buffer_pool = ["buffer_sv2"]
 
 [package.metadata.docs.rs]
-all-features = true
+features = ["with_buffer_pool"]
diff --git a/protocols/v2/binary-sv2/no-serde-sv2/derive_codec/Cargo.toml b/protocols/v2/binary-sv2/no-serde-sv2/derive_codec/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "derive_codec_sv2"
-version = "1.1.0"
+version = "1.1.1"
 authors = ["The Stratum V2 Developers"]
 edition = "2018"
 readme = "README.md"
@@ -18,6 +18,3 @@ binary_codec_sv2 = {version = "^1.0.0", path="../codec"}
 
 [lib]
 proc-macro = true
-
-[package.metadata.docs.rs]
-all-features = true