design new update_index transaction mechanism

[miketang84]

Optimize the performance of id-hash pair index updating

By introducing a new algorithm in the transaction pool of Substrate, we can promote the performance of the entire blockchain network on the consensus for the results submitted from the off-chain wasm worker.A brief explanation on this algorithm:

Usually, the way we process the oracle data is to collect the data from all nodes' off-chain workers, and then vote on the temporary list in the next round, this will postpone the finalization of this piece of data some blocks (depending on the GRANDPA). For the token price oracle, this is acceptable. But for an application oriented to internet users, it's unacceptable. Let's do some basic calculations. The whole process to get consensus will conduct 2 x N^2 (N is the number of nodes) network traffic. And delay about 6 x M (M>=2) seconds. Another stuff matters too. If there are too many index updates contained in one block, the on-chain votings would conduct a heavy computing burdens and cost a long time, which maybe exceed a block inteval.

So we conceive a new idea to optimize it. A rough idea is to mark the index updaing transation as a special transaction with attribution of non-propagation, just store it in local transaction pool, and will check it in the next block generation and validation. There are some details. If find the local value in self's node mismatched with the one in the coming block, this node will retrieve this transaction with a tag from all nodes' transaction pool, to check what is the major value of all, if local value is wrong, discard the local one; if the one in the block is wrong, reject this block. By this means, we can make the model indexes available within one block interval, after the next block is executed and the state space is updated. For most internet applications, 6 seconds on post is acceptable for most of users. The details of this algorithm have to be designed carefully, and maybe it's a hard work. It's an efficiency improving method.