Subxt v0.38.0 panics while parsing extrinsics

[f-squirrel]

Hi,

I am using subxt v0.38.0 compiled with metadata downloaded from a node as follows:

curl -X POST https://polkadot-rpc.publicnode.com \
    -H "Content-Type: application/json" \
    -d '{"jsonrpc": "2.0", "id": 1, "method": "state_getMetadata", "params": []}' \
      > scale

Later, we deserialize the data and set the runtime path in compile time;

#[subxt::subxt(runtime_metadata_path = "./polkadot.metadata.binary.scale.18-01-2025")]
pub mod polkadot {}

We receive the following error while parsing block 24413043:

thread 'main' panicked at 'slice index starts at 106 but ends at 0': ~/.cargo/registry/src/index.crates.io-6f17d22bba15001f/subxt-core-0.38.0/src/blocks/extrinsics.rs:196

Node URL: https://polkadot-rpc.publicnode.com/

[niklasad1]

hmm, weird looks like an integer overflow or something with the weird slice indexes....

let us try to reproduce it.

[f-squirrel]

@niklasad1 I am sorry for bothering you:)
Did you have a chance to look into the issue?

[niklasad1]

My bad, I'll take a look now

[niklasad1]

Hey again @f-squirrel

I can't reproduce this and subxt could decode the extrinsic for block 24413043 succesfully see this how I ran it here

I don't understand why you are not using $ subxt metadata --url wss://polkadot-rpc.publicnode.com > scale.metadata to generate the codegen instead of using curl and manual decoding it though....

[f-squirrel]

Hey @niklasad1,

Thank you for the investigation.
Indeed, your code works; it helped me to find out the difference:

The code crashes on the following:

        let is_root_ext = xt.as_root_extrinsic::<src_chain::Call>().is_ok();
        println!("is root extrinsic: {}", is_root_ext);

Please see the PR for your code example: niklasad1/subxt-decode-block-24413043#1.

Since the method returns Result, I think that the method shall not panic if it fails to decode.
Also, we use static call parsing, where the ability to decode to a root extrinsic is important, please see the example:

        let root = self
            .extrinsic
            .as_root_extrinsic::<polkadot::Call>()
            .expect("Cannot extract root ext, something is wring with the filter");

        match root {
            polkadot::Call::Balances(balance) => self.event_visitor.on_balance(&balance),
            polkadot::Call::Staking(staking) => self.event_visitor.on_staking(&staking),
            polkadot::Call::Utility(utility) => self.on_utility(&utility),
            other => {
                log::trace!("Ignored extrinsic: {other:?}");
                return Default::default();
            }
        }

[jsdw]

Also, we use static call parsing, where the ability to decode to a root extrinsic is important

As an aside, please note that if your metadata is out of date w.r.t the current runtime, there's a chance that you'll get back extrinsics that the generated code doesn't know about and so cannot decode, which would lead to a panic in your above example :)

[f-squirrel]

Hey @jsdw,

You are right; however, I prefer to panic if the protocol returns data that I do not expect.

However, this is not the case, the metadata was downloaded right before building the client.

I do not think that metadata is updated too often, and my code does not use any "new" features.

[niklasad1]

The metadata is actually "up-to-date" checked now and we should emit an error if it was incompat metadata not panic:ing as @f-squirrel said.

It's only Staking::chill that fails but I'll take a closer look in a moment.

The output:

Metadata IS COMPAT: YES
Timestamp::set => xt_hash=0x0043715c4bf9f8ce609e4d42006a5162472d94fe79e5035f9efaa04542db8184
pallet_idx 3
ext_bytes_len: 11
Extrinsic { compact_prefix_len: 1, version: 4, version_ty: Bare, byte_len: 11, signature: None, extensions: None, pallet_name: "Timestamp", pallet_index: 3, pallet_index_idx: 2, call_name: "set", call_index: 0, call_data: [NamedArg { name: "now", range: 4..11, ty: 11 }] }
call_data_data_range: 2..11
is root extrinsic: true
ParaInherent::enter => xt_hash=0xe4fab07c50bbf505654d2c1d53b871c2330c2e8e7872e923284eb3ecd3388ddd
pallet_idx 54
ext_bytes_len: 67724
Extrinsic { compact_prefix_len: 4, version: 4, version_ty: Bare, byte_len: 67724, signature: None, extensions: None, pallet_name: "ParaInherent", pallet_index: 54, pallet_index_idx: 5, call_name: "enter", call_index: 0, call_data: [NamedArg { name: "data", range: 7..67724, ty: 288 }] }
call_data_data_range: 5..67724
is root extrinsic: true
Balances::transfer_keep_alive => xt_hash=0x1122de534aedd9c9f162b680b5ed91e56bf7d31d63d2ad9cd8ec388ebb477e99
pallet_idx 5
ext_bytes_len: 148
Extrinsic { compact_prefix_len: 2, version: 4, version_ty: Signed, byte_len: 148, signature: Some(ExtrinsicSignature { address_start_idx: 3, address_end_idx: 36, signature_end_idx: 101, address_ty: 113, signature_ty: 341 }), extensions: Some(ExtrinsicExtensions { transaction_extensions_version: 0, transaction_extensions: [NamedArg { name: "CheckNonZeroSender", range: 101..101, ty: 843 }, NamedArg { name: "CheckSpecVersion", range: 101..101, ty: 844 }, NamedArg { name: "CheckTxVersion", range: 101..101, ty: 845 }, NamedArg { name: "CheckGenesis", range: 101..101, ty: 846 }, NamedArg { name: "CheckMortality", range: 101..103, ty: 847 }, NamedArg { name: "CheckNonce", range: 103..104, ty: 849 }, NamedArg { name: "CheckWeight", range: 104..104, ty: 850 }, NamedArg { name: "ChargeTransactionPayment", range: 104..105, ty: 851 }, NamedArg { name: "PrevalidateAttests", range: 105..105, ty: 852 }, NamedArg { name: "CheckMetadataHash", range: 105..106, ty: 853 }] }), pallet_name: "Balances", pallet_index: 5, pallet_index_idx: 106, call_name: "transfer_keep_alive", call_index: 3, call_data: [NamedArg { name: "dest", range: 108..141, ty: 113 }, NamedArg { name: "value", range: 141..148, ty: 63 }] }
call_data_data_range: 106..148
is root extrinsic: true
Staking::chill => xt_hash=0x749a0b0892f86714967385448389f492d5fb08e8aaa17a8ac1acfeeb6c556db2
pallet_idx 7
ext_bytes_len: 108
Extrinsic { compact_prefix_len: 2, version: 4, version_ty: Signed, byte_len: 108, signature: Some(ExtrinsicSignature { address_start_idx: 3, address_end_idx: 36, signature_end_idx: 101, address_ty: 113, signature_ty: 341 }), extensions: Some(ExtrinsicExtensions { transaction_extensions_version: 0, transaction_extensions: [NamedArg { name: "CheckNonZeroSender", range: 101..101, ty: 843 }, NamedArg { name: "CheckSpecVersion", range: 101..101, ty: 844 }, NamedArg { name: "CheckTxVersion", range: 101..101, ty: 845 }, NamedArg { name: "CheckGenesis", range: 101..101, ty: 846 }, NamedArg { name: "CheckMortality", range: 101..103, ty: 847 }, NamedArg { name: "CheckNonce", range: 103..104, ty: 849 }, NamedArg { name: "CheckWeight", range: 104..104, ty: 850 }, NamedArg { name: "ChargeTransactionPayment", range: 104..105, ty: 851 }, NamedArg { name: "PrevalidateAttests", range: 105..105, ty: 852 }, NamedArg { name: "CheckMetadataHash", range: 105..106, ty: 853 }] }), pallet_name: "Staking", pallet_index: 7, pallet_index_idx: 106, call_name: "chill", call_index: 6, call_data: [] }
call_data_data_range: 106..0
thread 'main' panicked at /home/niklasad1/Github/subxt/core/src/blocks/extrinsics.rs:196:22:
slice index starts at 106 but ends at 0

The issue comes from https://github.com/paritytech/frame-decode/blob/main/src/decoding/extrinsic_decoder.rs#L225 we shouldn't use unwrap_or(0) because it causes an invalid end idx if the call_bytes are empty.

[niklasad1]

This will be fixed by paritytech/frame-decode#30

[f-squirrel]

Thank you, @niklasad1, for your effort.

Chill is not a new functionality. Did it's format change in the recent Polkadot version?

[niklasad1]

No, we switched recently to the "frame_decode" crate for decoding extrinsics to support the new extrinsic format v5 in polkadot.

That switch occurred in subxt v0.38 IIRC so I expect it work fine if you downgrade to subxt v0.37 but we will fix it soon on v0.38 as well.

[niklasad1]

Fixed by paritytech/frame-decode#31 which is now released, cargo update -p frame-decode should fix it for you.

[f-squirrel]

@niklasad1 , it does not update, because the new version of frame-decode is not published to the crates.io.
Do you by any chance know when it is expected to happen?

[niklasad1]

https://crates.io/crates/frame-decode v0.6.1 is released which contains the fix, doesn't it work?

[f-squirrel]

It does not work. I think this is because subxt sets the exact version = 0.6.0.
I might miss something about "cargo update" logic.

[niklasad1]

It does not work. I think this is because subxt sets the exact version = 0.6.0.
I might miss something about "cargo update" logic.

That should work because frame-decode = 0.6.0 means that acceptable versions are in the range 0.6.0 to 0.6.x
I double-checked right now and subxt v0.38 sets frame-decode to 0.5 which is why it doesn't work

[niklasad1]

Alright, I released frame-decode 0.5.1 now, just do cargo update -p frame-decode and it should work.
(Also tested it in my re-production of this issue)

[f-squirrel]

@niklasad1, it indeed works now!
Thank you for the quick and elegant solution!