Skip to content

Commit

Permalink
feat: publish workspace crates (#4986)
Browse files Browse the repository at this point in the history 
## Describe your changes

Workspace refactor, preparing to publish the Penumbra crates. In doing
so, renames all crates to have a prefix `penumbra-sdk-`, which is unique
and controlled by PL. There's also a custom script that handles the
publication action, which is non-standard.

The major obstacle is that the `penumbra-sdk-proof-params` crate
contains binary keyfiles, which are managed in git-lfs. We cannot upload
the raw keyfiles to crates.io, because that'd result in a 100MB crate.
Instead, we use a custom script to revert the binary keyfiles to
plaintext lfs pointers immediately prior to publishing to crates.io,
which stays under the limit, and allows third-party tools to opt into
downloading the key material via the `download-proving-keys` feature.

Given that `docs.rs` doesn't permit network access during crate build,
we default to _warning_ about the missing keys during build of
proof-params, recommending enabling the option. We could go further add
fail the build, conditionally warning only in the [docsrs build
context](https://docs.rs/about/builds). I chose to minimize
special-casing as much as possible.

## Issue ticket number and link

Towards #4978.

## Testing and review

1. Browse the recently published "alpha" crates:
https://crates.io/search?q=penumbra-sdk
2. Try updating a Rust project that depends on Penumbra, as the [hermes
fork](https://github.com/penumbra-zone/hermes) or the
[reindexer](https://github.com/penumbra-zone/reindexer), to depend on
the published crates.
3. Confirm that docs are available:
https://docs.rs/penumbra-sdk-app/latest/penumbra_sdk_app/

## Checklist before requesting a review

- [x] I have added guiding text to explain how a reviewer should test
these changes.

- [x] If this code contains consensus-breaking changes, I have added the
"consensus-breaking" label. Otherwise, I declare my belief that there
are not consensus-breaking changes, for the following reason:

> Careful attention was paid to make this a non-breaking change. All the
crates have been renamed, but the code behaves the same. Testing should
focus on making sure that key material for the proof params crate is
fetched correctly, to avoid breakage.

---------

Co-authored-by: Erwan Or <[email protected]>
  • Loading branch information
@conorsch @erwanor
conorsch and erwanor authored Jan 21, 2025
1 parent 510cabf commit a7a0853
Show file tree
Hide file tree
Showing 528 changed files with 3,976 additions and 3,310 deletions.
1,172 changes: 676 additions & 496 deletions Cargo.lock
View file

Large diffs are not rendered by default.

81 changes: 44 additions & 37 deletions Cargo.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ members = [
"crates/bin/pmonitor",
"crates/cnidarium-component",
"crates/core/app",
"crates/core/app-tests",
"crates/core/asset",
"crates/core/component/community-pool",
"crates/core/component/compact-block",
Expand Down Expand Up @@ -94,17 +95,22 @@ tag-name = "v{{version}}"
# The message is required for an annotated tag to be created.
tag-message = "Penumbra version {{version}}"
pre-release-commit-message = "chore: release version {{version}}"
# Don't push to crates.io; we're still using git branches for velocity.
publish = false
# Permit publishing to crates.io; local developers should run `cargo release --no-publish`
# to prepare the git info, then pushing to remote with tag will trigger crates.io publication.
publish = true
# Do commit, tag, and push the version bump, triggering a new release.
tag = true
# Don't push, since we're still evaluating the new release workflow.
push = false
# Edit each crate's Cargo.toml, bumping versions for other workspace crates.
dependent-version = "upgrade"
# Increase default ratelimit on publishing packages
rate-limit = { existing-packages = 50 }

[workspace.package]
authors = ["Penumbra Labs <[email protected]"]
edition = "2021"
version = "0.81.3"
version = "1.0.0-alpha.0"
repository = "https://github.com/penumbra-zone/penumbra"
homepage = "https://penumbra.zone"
license = "MIT OR Apache-2.0"
Expand Down Expand Up @@ -135,13 +141,13 @@ camino = { version = "1" }
chacha20poly1305 = { version = "0.9.0" }
chrono = { default-features = false, version = "0.4" }
clap = { version = "3.2" }
cnidarium = { version = "0.82", default-features = false}
cnidarium-component = { default-features = false, path = "crates/cnidarium-component" }
cometindex = { path = "crates/util/cometindex" }
cnidarium = { version = "0.82.1", default-features = false}
cnidarium-component = { default-features = false, version = "1.0.0-alpha.0", path = "crates/cnidarium-component" }
cometindex = { version = "1.0.0-alpha.0", path = "crates/util/cometindex" }
criterion = { version = "0.4" }
decaf377 = { default-features = false, version = "0.10.1" }
decaf377-fmd = { path = "crates/crypto/decaf377-fmd" }
decaf377-ka = { path = "crates/crypto/decaf377-ka" }
decaf377-fmd = { version = "1.0.0-alpha.0", path = "crates/crypto/decaf377-fmd" }
decaf377-ka = { version = "1.0.0-alpha.0", path = "crates/crypto/decaf377-ka" }
decaf377-rdsa = { version = "0.11.0" }
derivative = { version = "2.2" }
directories = { version = "4.0.1" }
Expand Down Expand Up @@ -169,35 +175,36 @@ once_cell = { version = "1.8" }
parking_lot = { version = "0.12.1" }
pbjson = { version = "0.7.0" }
pbjson-types = { version = "0.7.0" }
penumbra-app = { default-features = false, path = "crates/core/app" }
penumbra-asset = { default-features = false, path = "crates/core/asset" }
penumbra-community-pool = { default-features = false, path = "crates/core/component/community-pool" }
penumbra-compact-block = { default-features = false, path = "crates/core/component/compact-block" }
penumbra-custody = { path = "crates/custody" }
penumbra-auction = { default-features = false, path = "crates/core/component/auction" }
penumbra-dex = { default-features = false, path = "crates/core/component/dex" }
penumbra-distributions = { default-features = false, path = "crates/core/component/distributions" }
penumbra-fee = { default-features = false, path = "crates/core/component/fee" }
penumbra-funding = { default-features = false, path = "crates/core/component/funding" }
penumbra-governance = { default-features = false, path = "crates/core/component/governance" }
penumbra-ibc = { default-features = false, path = "crates/core/component/ibc" }
penumbra-keys = { default-features = false, path = "crates/core/keys" }
penumbra-mock-client = { path = "crates/test/mock-client" }
penumbra-mock-consensus = { path = "crates/test/mock-consensus" }
penumbra-mock-tendermint-proxy = { path = "crates/test/mock-tendermint-proxy" }
penumbra-num = { default-features = false, path = "crates/core/num" }
penumbra-proof-params = { default-features = false, path = "crates/crypto/proof-params" }
penumbra-proof-setup = { path = "crates/crypto/proof-setup" }
penumbra-proto = { default-features = false, path = "crates/proto" }
penumbra-sct = { default-features = false, path = "crates/core/component/sct" }
penumbra-shielded-pool = { default-features = false, path = "crates/core/component/shielded-pool" }
penumbra-stake = { default-features = false, path = "crates/core/component/stake" }
penumbra-tct = { default-features = false, path = "crates/crypto/tct" }
penumbra-test-subscriber = { path = "crates/test/tracing-subscriber" }
penumbra-transaction = { default-features = false, path = "crates/core/transaction" }
penumbra-txhash = { default-features = false, path = "crates/core/txhash" }
penumbra-view = { path = "crates/view" }
penumbra-wallet = { path = "crates/wallet" }
penumbra-sdk-app = { default-features = false, version = "1.0.0-alpha.0", path = "crates/core/app" }
penumbra-sdk-asset = { default-features = false, version = "1.0.0-alpha.0", path = "crates/core/asset" }
penumbra-sdk-community-pool = { default-features = false, version = "1.0.0-alpha.0", path = "crates/core/component/community-pool" }
penumbra-sdk-compact-block = { default-features = false, version = "1.0.0-alpha.0", path = "crates/core/component/compact-block" }
penumbra-sdk-custody = { version = "1.0.0-alpha.0", path = "crates/custody" }
penumbra-sdk-auction = { default-features = false, version = "1.0.0-alpha.0", path = "crates/core/component/auction" }
penumbra-sdk-dex = { default-features = false, version = "1.0.0-alpha.0", path = "crates/core/component/dex" }
penumbra-sdk-distributions = { default-features = false, version = "1.0.0-alpha.0", path = "crates/core/component/distributions" }
penumbra-sdk-fee = { default-features = false, version = "1.0.0-alpha.0", path = "crates/core/component/fee" }
penumbra-sdk-funding = { default-features = false, version = "1.0.0-alpha.0", path = "crates/core/component/funding" }
penumbra-sdk-governance = { default-features = false, version = "1.0.0-alpha.0", path = "crates/core/component/governance" }
penumbra-sdk-ibc = { default-features = false, version = "1.0.0-alpha.0", path = "crates/core/component/ibc" }
penumbra-sdk-keys = { default-features = false, version = "1.0.0-alpha.0", path = "crates/core/keys" }
penumbra-sdk-mock-client = { version = "1.0.0-alpha.0", path = "crates/test/mock-client" }
penumbra-sdk-mock-consensus = { version = "1.0.0-alpha.0", path = "crates/test/mock-consensus" }
penumbra-sdk-mock-tendermint-proxy = { version = "1.0.0-alpha.0", path = "crates/test/mock-tendermint-proxy" }
penumbra-sdk-num = { default-features = false, version = "1.0.0-alpha.0", path = "crates/core/num" }
penumbra-sdk-proof-params = { default-features = false, version = "1.0.0-alpha.0", path = "crates/crypto/proof-params" }
penumbra-sdk-proof-setup = { version = "1.0.0-alpha.0", path = "crates/crypto/proof-setup" }
penumbra-sdk-proto = { default-features = false, version = "1.0.0-alpha.0", path = "crates/proto" }
penumbra-sdk-sct = { default-features = false, version = "1.0.0-alpha.0", path = "crates/core/component/sct" }
penumbra-sdk-shielded-pool = { default-features = false, version = "1.0.0-alpha.0", path = "crates/core/component/shielded-pool" }
penumbra-sdk-stake = { default-features = false, version = "1.0.0-alpha.0", path = "crates/core/component/stake" }
penumbra-sdk-tct = { default-features = false, version = "1.0.0-alpha.0", path = "crates/crypto/tct" }
penumbra-sdk-test-subscriber = { version = "1.0.0-alpha.0", path = "crates/test/tracing-subscriber" }
penumbra-sdk-tower-trace = { version = "1.0.0-alpha.0", path = "crates/util/tower-trace" }
penumbra-sdk-transaction = { default-features = false, version = "1.0.0-alpha.0", path = "crates/core/transaction" }
penumbra-sdk-txhash = { default-features = false, version = "1.0.0-alpha.0", path = "crates/core/txhash" }
penumbra-sdk-view = { version = "1.0.0-alpha.0", path = "crates/view" }
penumbra-sdk-wallet = { version = "1.0.0-alpha.0", path = "crates/wallet" }
pin-project = { version = "1.0.12" }
pin-project-lite = { version = "0.2.9" }
poseidon377 = { version = "1.2.0" }
Expand Down
29 changes: 17 additions & 12 deletions crates/bench/Cargo.toml
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
[package]
name = "penumbra-bench"
name = "penumbra-sdk-bench"
authors = {workspace = true}
repository = {workspace = true}
description = "Benchmarks for Penumbra crates"
version = {workspace = true}
license = {workspace = true}
edition = {workspace = true}
publish = false

[build-dependencies]
regex = { version = "1", optional = true }
Expand Down Expand Up @@ -74,9 +79,9 @@ bech32 = {workspace = true}
decaf377 = {workspace = true, features = ["r1cs"], default-features = true}
num-bigint = {workspace = true}
once_cell = {workspace = true}
penumbra-asset = {workspace = true, default-features = true}
penumbra-keys = {workspace = true, default-features = true}
penumbra-num = {workspace = true, default-features = true}
penumbra-sdk-asset = {workspace = true, default-features = true}
penumbra-sdk-keys = {workspace = true, default-features = true}
penumbra-sdk-num = {workspace = true, default-features = true}
rand = {workspace = true}
rand_core = {workspace = true, features = ["getrandom"]}
serde = {workspace = true, features = ["derive"]}
Expand All @@ -88,15 +93,15 @@ criterion = {workspace = true, features = ["html_reports"]}
decaf377-fmd = {workspace = true}
decaf377-ka = {workspace = true}
decaf377-rdsa = {workspace = true}
penumbra-dex = {workspace = true, default-features = true}
penumbra-fee = {workspace = true, default-features = true}
penumbra-governance = {workspace = true, default-features = true}
penumbra-sct = {workspace = true, default-features = true}
penumbra-shielded-pool = {workspace = true, default-features = true}
penumbra-stake = {workspace = true, default-features = true}
penumbra-tct = {workspace = true, features = ["r1cs"], default-features = true}
penumbra-sdk-dex = {workspace = true, default-features = true}
penumbra-sdk-fee = {workspace = true, default-features = true}
penumbra-sdk-governance = {workspace = true, default-features = true}
penumbra-sdk-sct = {workspace = true, default-features = true}
penumbra-sdk-shielded-pool = {workspace = true, default-features = true}
penumbra-sdk-stake = {workspace = true, default-features = true}
penumbra-sdk-tct = {workspace = true, features = ["r1cs"], default-features = true}

[dev-dependencies.penumbra-proof-params]
[dev-dependencies.penumbra-sdk-proof-params]
workspace = true
default-features = true
features = [
Expand Down
8 changes: 4 additions & 4 deletions crates/bench/benches/convert.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@ use ark_relations::r1cs::{
ConstraintSynthesizer, ConstraintSystem, OptimizationGoal, SynthesisMode,
};
use decaf377::{Fq, Fr};
use penumbra_asset::{Balance, Value, STAKING_TOKEN_ASSET_ID};
use penumbra_num::{fixpoint::U128x128, Amount};
use penumbra_proof_params::{DummyWitness, CONVERT_PROOF_PROVING_KEY};
use penumbra_shielded_pool::{
use penumbra_sdk_asset::{Balance, Value, STAKING_TOKEN_ASSET_ID};
use penumbra_sdk_num::{fixpoint::U128x128, Amount};
use penumbra_sdk_proof_params::{DummyWitness, CONVERT_PROOF_PROVING_KEY};
use penumbra_sdk_shielded_pool::{
ConvertCircuit, ConvertProof, ConvertProofPrivate, ConvertProofPublic,
};

Expand Down
14 changes: 7 additions & 7 deletions crates/bench/benches/delegator_vote.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,15 @@ use ark_relations::r1cs::{
};
use decaf377::{Fq, Fr};
use decaf377_rdsa::{SpendAuth, VerificationKey};
use penumbra_asset::Value;
use penumbra_governance::{
use penumbra_sdk_asset::Value;
use penumbra_sdk_governance::{
DelegatorVoteCircuit, DelegatorVoteProof, DelegatorVoteProofPrivate, DelegatorVoteProofPublic,
};
use penumbra_keys::keys::{Bip44Path, SeedPhrase, SpendKey};
use penumbra_proof_params::{DummyWitness, DELEGATOR_VOTE_PROOF_PROVING_KEY};
use penumbra_sct::Nullifier;
use penumbra_shielded_pool::Note;
use penumbra_tct as tct;
use penumbra_sdk_keys::keys::{Bip44Path, SeedPhrase, SpendKey};
use penumbra_sdk_proof_params::{DummyWitness, DELEGATOR_VOTE_PROOF_PROVING_KEY};
use penumbra_sdk_sct::Nullifier;
use penumbra_sdk_shielded_pool::Note;
use penumbra_sdk_tct as tct;

use criterion::{criterion_group, criterion_main, Criterion};
use rand_core::OsRng;
Expand Down
14 changes: 7 additions & 7 deletions crates/bench/benches/nullifier_derivation.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@ use ark_relations::r1cs::{
ConstraintSynthesizer, ConstraintSystem, OptimizationGoal, SynthesisMode,
};
use decaf377::Fq;
use penumbra_asset::Value;
use penumbra_keys::keys::{Bip44Path, SeedPhrase, SpendKey};
use penumbra_proof_params::{DummyWitness, NULLIFIER_DERIVATION_PROOF_PROVING_KEY};
use penumbra_sct::Nullifier;
use penumbra_shielded_pool::{
use penumbra_sdk_asset::Value;
use penumbra_sdk_keys::keys::{Bip44Path, SeedPhrase, SpendKey};
use penumbra_sdk_proof_params::{DummyWitness, NULLIFIER_DERIVATION_PROOF_PROVING_KEY};
use penumbra_sdk_sct::Nullifier;
use penumbra_sdk_shielded_pool::{
Note, NullifierDerivationProofPrivate, NullifierDerivationProofPublic, Rseed,
};
use penumbra_shielded_pool::{NullifierDerivationCircuit, NullifierDerivationProof};
use penumbra_tct as tct;
use penumbra_sdk_shielded_pool::{NullifierDerivationCircuit, NullifierDerivationProof};
use penumbra_sdk_tct as tct;

use criterion::{criterion_group, criterion_main, Criterion};
use rand_core::OsRng;
Expand Down
8 changes: 4 additions & 4 deletions crates/bench/benches/output.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ use ark_relations::r1cs::{
use decaf377::{Fq, Fr};
use decaf377_fmd as fmd;
use decaf377_ka as ka;
use penumbra_asset::{Balance, Value};
use penumbra_keys::{keys::Diversifier, Address};
use penumbra_proof_params::{DummyWitness, OUTPUT_PROOF_PROVING_KEY};
use penumbra_shielded_pool::{
use penumbra_sdk_asset::{Balance, Value};
use penumbra_sdk_keys::{keys::Diversifier, Address};
use penumbra_sdk_proof_params::{DummyWitness, OUTPUT_PROOF_PROVING_KEY};
use penumbra_sdk_shielded_pool::{
output::{OutputProofPrivate, OutputProofPublic},
Note, OutputCircuit, OutputProof, Rseed,
};
Expand Down
14 changes: 8 additions & 6 deletions crates/bench/benches/spend.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,14 @@ use ark_relations::r1cs::{
};
use decaf377::{Fq, Fr};
use decaf377_rdsa::{SpendAuth, VerificationKey};
use penumbra_asset::Value;
use penumbra_keys::keys::{Bip44Path, SeedPhrase, SpendKey};
use penumbra_proof_params::{DummyWitness, SPEND_PROOF_PROVING_KEY};
use penumbra_sct::Nullifier;
use penumbra_shielded_pool::{Note, SpendCircuit, SpendProof, SpendProofPrivate, SpendProofPublic};
use penumbra_tct as tct;
use penumbra_sdk_asset::Value;
use penumbra_sdk_keys::keys::{Bip44Path, SeedPhrase, SpendKey};
use penumbra_sdk_proof_params::{DummyWitness, SPEND_PROOF_PROVING_KEY};
use penumbra_sdk_sct::Nullifier;
use penumbra_sdk_shielded_pool::{
Note, SpendCircuit, SpendProof, SpendProofPrivate, SpendProofPublic,
};
use penumbra_sdk_tct as tct;

use criterion::{criterion_group, criterion_main, Criterion};
use rand_core::OsRng;
Expand Down
12 changes: 6 additions & 6 deletions crates/bench/benches/swap.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,16 @@ use ark_relations::r1cs::{
ConstraintSynthesizer, ConstraintSystem, OptimizationGoal, SynthesisMode,
};
use decaf377::{Fq, Fr};
use penumbra_asset::{asset, Balance, Value};
use penumbra_dex::{
use penumbra_sdk_asset::{asset, Balance, Value};
use penumbra_sdk_dex::{
swap::proof::{SwapCircuit, SwapProof, SwapProofPublic},
swap::{proof::SwapProofPrivate, SwapPlaintext},
TradingPair,
};
use penumbra_fee::Fee;
use penumbra_keys::keys::{Bip44Path, SeedPhrase, SpendKey};
use penumbra_num::Amount;
use penumbra_proof_params::{DummyWitness, SWAP_PROOF_PROVING_KEY};
use penumbra_sdk_fee::Fee;
use penumbra_sdk_keys::keys::{Bip44Path, SeedPhrase, SpendKey};
use penumbra_sdk_num::Amount;
use penumbra_sdk_proof_params::{DummyWitness, SWAP_PROOF_PROVING_KEY};

use criterion::{criterion_group, criterion_main, Criterion};
use rand_core::OsRng;
Expand Down
16 changes: 8 additions & 8 deletions crates/bench/benches/swap_claim.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,18 @@ use ark_relations::r1cs::{
ConstraintSynthesizer, ConstraintSystem, OptimizationGoal, SynthesisMode,
};
use decaf377::Fq;
use penumbra_asset::asset;
use penumbra_dex::{
use penumbra_sdk_asset::asset;
use penumbra_sdk_dex::{
swap::SwapPlaintext,
swap_claim::{SwapClaimCircuit, SwapClaimProof, SwapClaimProofPrivate, SwapClaimProofPublic},
BatchSwapOutputData, TradingPair,
};
use penumbra_fee::Fee;
use penumbra_keys::keys::{Bip44Path, SeedPhrase, SpendKey};
use penumbra_num::Amount;
use penumbra_proof_params::{DummyWitness, SWAPCLAIM_PROOF_PROVING_KEY};
use penumbra_sct::Nullifier;
use penumbra_tct as tct;
use penumbra_sdk_fee::Fee;
use penumbra_sdk_keys::keys::{Bip44Path, SeedPhrase, SpendKey};
use penumbra_sdk_num::Amount;
use penumbra_sdk_proof_params::{DummyWitness, SWAPCLAIM_PROOF_PROVING_KEY};
use penumbra_sdk_sct::Nullifier;
use penumbra_sdk_tct as tct;

use criterion::{criterion_group, criterion_main, Criterion};
use rand_core::OsRng;
Expand Down
Loading

0 comments on commit a7a0853

Please sign in to comment.