🔷 [Epic] Multichain

[itegulov]

Description

Multichain is a total overhaul of mpc-recovery service that will move us away from aggregated signatures towards TSS. Multichain network will store users' private keys in a non-custodial way by splitting keys into key shares and distributing the pieces among multiple independent parties.

Let P be the number of trusted parties holding the key shares.

Reasons for transition are:

• Fault tolerance. Current implementation requires all nodes to be online and ready to accept requests. If any party loses their keys then the whole network cannot operate anymore. Multichain, on the other hand, has a configurable (and dynamic) threshold of participants to be available in order to continue the network (currently presumed to be ~2/3 of P).
• Decentralization. We are getting rid of the leader node and introducing a proper consensus mechanism.
• Generalization. Multichain is all about being able to sign arbitrary payload on-chain, which enables other potential applications of multichain outside of FastAuth. This means we are decoupling from OIDC and FastAuth will become an application built on top of multichain.

Roughly, the new flow is going to look like this:

1. Developer D writes and deploys a smart contract C @ my_cool_app.near
2. User U calls function foo on C
3. foo makes a cross-contract call to multichain.near (a contract deployed by us) where they request to sign(payload), where payload is payload provided by C's internal logic
4. P participants index ingoing call to multichain.near/sign and see the transaction above
5. They follow the MPC cryptographic protocol and generate a signature S for the submitted payload
6. One of the participants submits S to multichain.near/response
7. D has an indexer that watches for multichain.near/response and once it sees the transaction above it registers S as the response to U's interaction with C

Resources

Overview of cryptography behind multichain: https://docs.google.com/document/d/1FKC9LvVyrEq6CiFYCnUFtQfvaDWddxzdtaEQb-_fq_s/edit#heading=h.we4ish11290u

This epic presumes that future work is happening on top of #313

Cryptography

Preview Give feedback

1. Implement Beaver triple generation mpc-recovery#328
   Emerging Tech Near BOS +2
   
2. Implement presignature generation mpc-recovery#341
   Emerging Tech Near BOS +2
   
3. Implement signature generation mpc-recovery#385
   Emerging Tech Near BOS +2
   
4. Use fair triple distribution mechanism mpc-recovery#384
   Emerging Tech Near BOS +2
   
5. Ensure we can have multiple signing state machines progressing at the same time
6. Stockpile Beaver triples and presignatures (persistently?) and clean them once we reshare (Almost necessary). Stockpiling exists, but it's naive and doesn't take bandwith or persistency into account mpc-recovery#456
   +0
   
7. (Optional). Reshare and run at the same time to ensure the liveliness of the protocol (i.e. there is no need to abort ongoing sign requests)
8. Additive key derivation for multichain mpc-recovery#352
   Emerging Tech Near BOS +2
9. Occasionally reshare the key even when the participant set does not change. Should be done once in 24 hours - 1 week according to Michel (Security, not necessary for March release IMO)
10. Serialize cryptographic primitives to the correct signature format used in NEAR ecosystem mpc-recovery#386
    Emerging Tech Near BOS +2
    
11. Persistent Beaver triples and presignatures. Right now on restart a node will lose all of the triples/presignatures rendering them useless for other nodes. (it can complicate the protocl, introdcuse many edge cases)
12. (Optional). Some mechanism on deciding who is messing with the protocol messages. It is impossible to tell whose message broke the protocol step in cait-sith, but in theory some karma system might help here (-1 for being a part of a set that failed to complete a protocol step). Note that there is no incentive to behave badly intentionally, so this is arguably very optional.
13. Handle offline nodes in beaver triple generation gracefully mpc-recovery#439
    Emerging Tech Near BOS +2
    

Network

Preview Give feedback

1. Secure p2p communication mpc-recovery#329
   Emerging Tech Near BOS +2
   
2. Separate key for message signing mpc-recovery#381
   Emerging Tech Near BOS +2
   
3. Message gets ignored if received before we sync with protocol state mpc-recovery#382
   Emerging Tech Near BOS +2
   
4. Create a full-connectivity mesh between the nodes mpc-recovery#405
   +0
   
5. Proper message queue for messages from various epochs and states. Might even make it persistent? Important that it does not leak memory
6. Versionized networking, use protobuf for keeping tack of compatibility, make sure that each two adjacent versions are backwards-compatible

Consensus

Preview Give feedback

1. Decentralize node id distribution mpc-recovery#330
   Emerging Tech Near BOS +2
   
2. Vote kick mechanism #353
   Emerging Tech Near BOS +2
3. Enforce minimum and maximum threshold (and hence the size of the set as it is 150% of the threshold presumably). Minimum is needed to prevent a small set overtaking the entire MPC. Maximum is needed to prevent performance issues.
4. (Optional). Consider a backup network (suggested by Michel, no elaboration)
5. (Optional). Decide if we need some sort of slashing mechanism for misbehaving nodes
6. Nodes should initialize their own information in the contract state mpc-recovery#389
   Emerging Tech Near BOS +2
   
7. Restart cait-sith protocols on errors mpc-recovery#424
   +0
   
8. (Opt). Recognize that someone else has gave up on the protocol and restarted it. This probably makes sense for only specific types of protocols - generating, resharing (maybe something else?). Major issue here is that there might not be an easy way to distinguish new protocol messages from the old protocol messages. This means some one of the nodes might get confused that the protocol is still alive due to race conditions. One way to battle is to attach random GenerationId and ReshareId to all protocols.
9. (Optional). Allow rolling back from Resharing to Running if something fatal happened during the resharing phase (e.g. one of the nodes is not participating and blocking us from making progress). Then the set of new joiners is wiped and everyone new have to sign up from scratch.

API (no yield/resume option)

Preview Give feedback

1. Index request transaction for multichain mpc-recovery#346
   Emerging Tech Near BOS +2
2. (Optional). Migrate to standalone independent indexer for each multichain node. See Index request transaction for multichain #346 for
3. Finish with Bowen's self-call improvement (see adding experiment with self calls #401). It is a direct improvement over what we have right now. You are not forced to pay gas for self calls, you can always just index respond txs as it was with the current approach. But with it you get an option to pay up to 300TGas to potentially get a sequential response. This also simplifies integration tests as you don't have to run indexer there.

API (yield/resume option)

Preview Give feedback

1. Wait until Support for yielded execution NEPs#519 is done
2. Rewrite MPC contract to use yield/resume

Infra

Preview Give feedback

1. Add dev terraform infrastructure for multichain mpc-recovery#383
   Emerging Tech Near BOS +2
   
2. Add testnet terraform infrastructure for multichain mpc-recovery#426
   +0
   
3. Add mainnet terraform infrastructure for multichain mpc-recovery#427
   +0
   
4. Make sure that it's hard to lose a node keys mpc-recovery#435
   +0
   
5. Port opentelemetry code to multichain codebase mpc-recovery#428
   +0
   
6. Pick a persistent storage solution (let's **not** use Google Datastore this time though) mpc-recovery#430
   +0
   
7. Enforce a tracing style that we should use uniformly, we can base it off of [this](https://github.com/near/nearcore/blob/master/docs/practices/style.md#tracing)
8. Consider tokenomics of multichain, design a reward system mpc-recovery#327
   +0
   
9. Add the ability to start local test env (simmilar to the one we had in old design)
10. Make dev env index starting from a recent block height
11. Keep track of the last block height processed by the indexer
12. Stablize the consensus on node restart mpc-recovery#425
    Emerging Tech Near BOS +2
13. Github workflow for multichain mpc-recovery#419
    Emerging Tech Near BOS +2
14. Multichain move away from cloud run mpc-recovery#420
    Emerging Tech Near BOS +2
    
15. Save last indexed block in a persistent storage so that the indexer does not have to start from the very beginning every time mpc-recovery#431
    Emerging Tech Near BOS +2
    

Low priority bugs/performance improvements

Preview Give feedback

1. clear messages of already generated signatures from the message queue mpc-recovery#433
   Emerging Tech Near BOS +2

[volovyks]

stale