replication: define requirement

[maxux]

Replication is a subject scheduled since the beginning but was never fully implemented. Let's talk about how we should implement that correctly.

My first decision was to keep 0-db server as standalone server and not act like a client (to connect to another server). I think this will need to change to make things easier on runtime.

I have some early question about what 0-db should suport:

• What kind of replication should 0-db support ? (Master/slave, master/master, other ?)
• Who copy data to whom ?
• How replication can affect performance ?

I expected something like: a 0-db server could connect to another running server, after negotiating a master key, the new server become a slave of the master. It first request list of namespaces, get all namespaces descriptors. Slave compute hash of all existing data (index and data), master does the same and transfert each files where hash differs. All of this should happen without affecting clients (not freezing server for the whole process). When servers are in-sync, master send a copy of all commands to slave.

This lead to some issues aswell, just copying commands (like db-mirror did previously) can lead to different timestamp, protocol was updated to support an extra timestamp to use, but other field could differs. And if anything bad happen, db will be out-of-sync and all offset etc. will be wrong.

Maybe another solution would be to sync db periodically and not on realtime ? Depend if slave needs to be inline with master in realtime ?

I'm open to any suggestion.