CRDT System

[gonpombo8]

decentraland/ecs#16

The idea is to use CRDT systems to synchronize entities through the network between peers.

Which entities will be synced ?

We are creating a component called SyncComponent, and every entity that has that component will send the entityId with all the components through the network to the peers.
To handle this, we are creating a system that iterates all the entities that has a synced component on it, and it's dirty.

CRDT its only for the network between peers ?

No, the idea is tu used it also in the communication between the scene and the renderer. So the CRDT will live at the ECS leven but also at the renderer.
We still need to discuss a lot of things here. We need to create a crdt lib for unity? Or how we are going to implement it ?
Will be the same state that the network/peers one ?
Where the state store will live ?

How the messages will be sent ?

We are going to use an RPC protocol to send the binary messages through the wire. We thought about protocol buffers with a structure similar to this.

message ComponentOperation {
    int32 message_type = 1;     // PUT_COMPONENT | DELETE_COMPONENT
    int64 entity_id = 2;        // Entity number
    int64 component_number = 3; // ClassIdentifier number for the component kind
 
    // We use lamport timestamps to identify components and to track in which order
    // a client created them. The key for the lamport number is key=(entity_id,component_number)
    int64 timestamp = 4;
 
    // @optional
    // missing data means the component was deleted
    bytes data = 5;
}

CRDT Renderer state.

Store the state so we can retrieve the store and send it to the scene for initialization ?

For more reference see

• Metaverse Runtime
• New Ecs Path

[gonpombo8]

Broadcast CRDT message problem. [Transactor ?]

Given clients A,B,C,D
Client D has the same state as A
Client B and C has the latest state for key A ( lamport: 5 ).
Client A creates a message for key A thats outdated ( lamport: 2 )
Client A sends outdated message to client B and C.
Client B receives the message, see that's outdated and re-send it through the wire (to all the clients. ws.broadcast)
Same for Client C. He receives the message, see that's outdated and re-send it through the wire.

This way client A and D recieves the new message with lamport: 5

Now imagine this with client N, with N > 1000. we are generating (N - 1) messages for each invalid message.
If we only send the message to the client who creates the outdated message, how client D will update his state ? He still thinks that the Lamport 2 of client A is valid.

Maybe here comes the transactor which is the one who broadcasts the messages, so we dont have to send tons of repeated messages ?

[menduz]

https://adr.decentraland.org/adr/ADR-117