Use polkadot-sdk crates

[magecnion]

This PR builds upon @asiniscalchi previous work. Since Substrate does not plan to incorporate substrate-test-runtime, I have decided to include this crate at version stable2409 directly into the frontier repository.

[boundless-forest]

I suggest we create a dedicated branch for the crate.io version release for each version of polkadot-sdk to avoid breaking some downstream repos, as most of them are still using git dependencies, as far as I know. If you agree, I'll create a new branch called stable2409-crates, and then you can change your merge target to that. However, we may need extra cherry-pick work in the future. I don't have a better solution for this at the moment.

[magecnion]

we may need extra cherry-pick work in the future. I don't have a better solution for this at the moment.

Agree. Let me know when you create the branch and I'llchange the target.

[seunlanlege]

I suggest we create a dedicated branch for the crate.io version release for each version of polkadot-sdk to avoid breaking some downstream repos, as most of them are still using git dependencies, as far as I know

I think anyone who is using frontier would pin their dependency to specific commits so this isn't neccessary. Rather than creating a new branch for crates.io. We should instead create a new branch for git following the naming convention for polkadot-sdk stable releases.

Also i'd recommend versioning the crates using the "minor version" of polkadot-sdk releases as your major version. Eg stable2409 is actually polkadot-sdk-v1.16.0. So all crates in this repo should be versioned as v16.0.0.

I'm trying to set this as the standard for all polkadot-sdk crates so we can all escape versioning hell. This scheme allows you guys to make minor and patch releases.

[boundless-forest]

I think anyone who is using frontier would pin their dependency to specific commits so this isn't neccessary. Rather than creating a new branch for crates.io. We should instead create a new branch for git following the naming convention for polkadot-sdk stable releases.

I agree with this as more and more repos choose to rely on the crates version. If so, we are going to review and merge this PR, then create another branch for Git. What do you think? @crystalin @conr2d

I'm trying to set this as the standard for all polkadot-sdk crates so we can all escape versioning hell. This scheme allows you guys to make minor and patch releases.

I really like this idea; have you proposed it on the polkadot-sdk side? I would like to support this.

[seunlanlege]

Yeah i've created an issue here:

paritytech/polkadot-sdk#6987

[conr2d]

I agree with this as more and more repos choose to rely on the crates version. If so, we are going to review and merge this PR, then create another branch for Git.

I also think that, in the long term, it makes sense to update the master branch to use the Polkadot SDK crates published on crates.io. However, I’m not sure if it’s the right move at the moment. Especially if we’re considering plans to publish Frontier crates as well, we’ll need a clear versioning policy. Relying on different major versions of Polkadot SDK crates would introduce breaking changes, even if there are no changes on the Frontier side.

[drewstone]

What's blocking this from being merged? Would be helpful to get to versioned release of Frontier asap.

It currently prevents using Hyperbridge to bridge to solo chains w/ EVMs.

[boundless-forest]

What's blocking this from being merged? Would be helpful to get to versioned release of Frontier asap.

It currently prevents using Hyperbridge to bridge to solo chains w/ EVMs.

I'm not sure if it's a good practice to use the versioned crates from the polkadot-sdk right now. There's been no further feedback from the community. Honestly, I don't like the current versioning style of the published crates from polkadot, as I wrote in the related issue raised above. I prefer the git dependencies. However, to address your problem, would you consider submitting a PR to replace the substrate-test-runtime crate reference in the master branch with a copied version like this PR? So, you can generate a versioned frontier in your repo. I believe this can solve your problem while keeping the master branch aligned with the git dependencies, and we can see if there are improvements on the polkadot-sdk side.

[drewstone]

Ok learned it might now not block hyperbridge. Is there a working group that meets regularly to discuss this topic? It seems like it could have its future more well defined.

[conr2d]

Based on my experiences over the past few days, it seems premature to rely on the Polkadot SDK crates published on crates.io. The behavior of psvm with stable2412 is broken, and updating dependencies to stable2412 results in a downgrade of dependencies compared to stable2409, causing builds to fail.

Since the versioning history of the Polkadot SDK packages is currently messed up, I personally think it might be better to start from scratch. However, since that is an issue to be decided by upstream, we'll have to wait and see how they choose to resolve it.

[seunlanlege]

Polkadot-sdk core devs are holding a versioning poll. Let's all participate so we can clean up this mess.

paritytech/polkadot-sdk#6987 (comment)