Code Walkthrough

Following a block from the network socket to the hard drive

Note: All links to the code are based on a fixed revision, so we can link line numbers. Code might have changed by now, nonetheless use these revisions when doing updates to this document.

Preconditons

• the app was configured and started
• all services registered
• the discovery protocol discovered some nodes
• the peermanager successfully connected a node
• established an encrypted multiplexed session
• and created a ETHProtocol instance for this peer
• we are waiting for ingress data in the peer connection

Receiving and decoding network data

Wait for data on the socket: https://github.com/ethereum/pydevp2p/blob/6577016a6f50ce220e20f66c2c8c1eb6dc271b9c/devp2p/peer.py#L208

add the (partial) message to the MultiplexedSession: https://github.com/ethereum/pydevp2p/blob/6577016a6f50ce220e20f66c2c8c1eb6dc271b9c/devp2p/peer.py#L223

try to decode packets (objects representing the equivalent to to a rpc cmd): https://github.com/ethereum/pydevp2p/blob/6577016a6f50ce220e20f66c2c8c1eb6dc271b9c/devp2p/muxsession.py#L66

decode if there is enough data in the buffer https://github.com/ethereum/pydevp2p/blob/6577016a6f50ce220e20f66c2c8c1eb6dc271b9c/devp2p/multiplexer.py#L520 https://github.com/ethereum/pydevp2p/blob/6577016a6f50ce220e20f66c2c8c1eb6dc271b9c/devp2p/multiplexer.py#L426

decrypt https://github.com/ethereum/pydevp2p/blob/6577016a6f50ce220e20f66c2c8c1eb6dc271b9c/devp2p/multiplexer.py#L442

using the cipher https://github.com/ethereum/pydevp2p/blob/6577016a6f50ce220e20f66c2c8c1eb6dc271b9c/devp2p/muxsession.py#L17

which we have in the RLPx session https://github.com/ethereum/pydevp2p/blob/6577016a6f50ce220e20f66c2c8c1eb6dc271b9c/devp2p/rlpxcipher.py#L41

decoded packets are put in a queue which are processed by a greenlet https://github.com/ethereum/pydevp2p/blob/6577016a6f50ce220e20f66c2c8c1eb6dc271b9c/devp2p/peer.py#L197

which was spawned by https://github.com/ethereum/pydevp2p/blob/6577016a6f50ce220e20f66c2c8c1eb6dc271b9c/devp2p/peer.py#L203

the Peer instance which is another greenlet https://github.com/ethereum/pydevp2p/blob/6577016a6f50ce220e20f66c2c8c1eb6dc271b9c/devp2p/peer.py#L22

the protocol for handling the packet is resolved https://github.com/ethereum/pydevp2p/blob/6577016a6f50ce220e20f66c2c8c1eb6dc271b9c/devp2p/peer.py#L164

and given the packet https://github.com/ethereum/pydevp2p/blob/6577016a6f50ce220e20f66c2c8c1eb6dc271b9c/devp2p/peer.py#L174 https://github.com/ethereum/pydevp2p/blob/6577016a6f50ce220e20f66c2c8c1eb6dc271b9c/devp2p/protocol.py#L160

in our case we assume the packet is for the "eth" protocol https://github.com/ethereum/pyethapp/blob/e4ce60854785ffec4fd2b9490fd58a015287c7c0/pyethapp/eth_protocol.py#L15

Handling the message

The ETHProtocol instance was linked to the ChainService instance on creation, which registered callbacks https://github.com/ethereum/pyethapp/blob/e4ce60854785ffec4fd2b9490fd58a015287c7c0/pyethapp/eth_service.py#L334

which are called https://github.com/ethereum/pydevp2p/blob/6577016a6f50ce220e20f66c2c8c1eb6dc271b9c/devp2p/protocol.py#L68

after the Packet.cmd_id is resolved to a handling command class https://github.com/ethereum/pydevp2p/blob/6577016a6f50ce220e20f66c2c8c1eb6dc271b9c/devp2p/protocol.py#L160 https://github.com/ethereum/pydevp2p/blob/6577016a6f50ce220e20f66c2c8c1eb6dc271b9c/devp2p/protocol.py#L44

in our case eth_protocol.NewBlock: https://github.com/ethereum/pyethapp/blob/e4ce60854785ffec4fd2b9490fd58a015287c7c0/pyethapp/eth_protocol.py#L143

which decodes the payload: https://github.com/ethereum/pyethapp/blob/e4ce60854785ffec4fd2b9490fd58a015287c7c0/pyethapp/eth_protocol.py#L157

the registered callback function is called and handles the message: https://github.com/ethereum/pyethapp/blob/e4ce60854785ffec4fd2b9490fd58a015287c7c0/pyethapp/eth_service.py#L461

new blocks are handled by the synchronizer which knows the state of the synchronization: https://github.com/ethereum/pyethapp/blob/e4ce60854785ffec4fd2b9490fd58a015287c7c0/pyethapp/synchronizer.py#L283

if the block is unknown https://github.com/ethereum/pyethapp/blob/e4ce60854785ffec4fd2b9490fd58a015287c7c0/pyethapp/synchronizer.py#L294

and pow is valid https://github.com/ethereum/pyethapp/blob/e4ce60854785ffec4fd2b9490fd58a015287c7c0/pyethapp/synchronizer.py#L299

and chain difficulty sufficient https://github.com/ethereum/pyethapp/blob/e4ce60854785ffec4fd2b9490fd58a015287c7c0/pyethapp/synchronizer.py#L304

and we have a parent for the block https://github.com/ethereum/pyethapp/blob/e4ce60854785ffec4fd2b9490fd58a015287c7c0/pyethapp/synchronizer.py#L304

the block is added to a queue https://github.com/ethereum/pyethapp/blob/e4ce60854785ffec4fd2b9490fd58a015287c7c0/pyethapp/synchronizer.py#L304 https://github.com/ethereum/pyethapp/blob/e4ce60854785ffec4fd2b9490fd58a015287c7c0/pyethapp/eth_service.py#L202

which is processed by a greenlet https://github.com/ethereum/pyethapp/blob/e4ce60854785ffec4fd2b9490fd58a015287c7c0/pyethapp/eth_service.py#L207

which pop blocks from the queue and tries to add them to the chain: https://github.com/ethereum/pyethapp/blob/e4ce60854785ffec4fd2b9490fd58a015287c7c0/pyethapp/eth_service.py#L228

entering the block chain

so far we had a transient block (t_block) which was not yet connected to the database, i.e. it could not access it's ancestors or access state.

it is now deserialized to a block.Block https://github.com/ethereum/pyethapp/blob/e4ce60854785ffec4fd2b9490fd58a015287c7c0/pyethapp/eth_service.py#L252 https://github.com/ethereum/pyethapp/blob/e4ce60854785ffec4fd2b9490fd58a015287c7c0/pyethapp/eth_protocol.py#L220

blocks.Block and blocks.BlockHeader are subclasses of rlp.Serializable https://github.com/ethereum/pyethereum/blob/efb08554ee95aa18b08d8f5d21d76097631d2c4a/ethereum/blocks.py#L378 https://github.com/ethereum/pyrlp/blob/45160b64cc13c2c4b688965aec5028bef2e67f94/rlp/sedes/lists.py#L89

on blocks.Block init https://github.com/ethereum/pyethereum/blob/efb08554ee95aa18b08d8f5d21d76097631d2c4a/ethereum/blocks.py#L404