Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add EIP-7805 (FOCIL) specs #4003

Open
wants to merge 45 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
3a16a0e
Add focil containers
terencechain Sep 11, 2024
fa27c8f
Add process inclusion list aggregate
terencechain Sep 12, 2024
3c86963
Add inclusion list gossip conditions
terencechain Sep 14, 2024
8a1e940
Add execution api
terencechain Sep 15, 2024
833be94
Add validator spec
terencechain Sep 16, 2024
a2b68a7
Review feedback
terencechain Sep 17, 2024
fb4a3c3
Update forkchoice
terencechain Sep 17, 2024
c7a056b
Update based on latest discussion. No more IL aggregate
terencechain Sep 21, 2024
f84556c
Update fork-choice.md
terencechain Sep 26, 2024
2787b0d
Update per https://ethresear.ch/t/focil-cl-el-workflow/20526
terencechain Sep 30, 2024
0b09f96
Address feedback
terencechain Oct 2, 2024
6056b69
Clarify forkchoice and p2p
terencechain Oct 12, 2024
a8aec7c
Remove local
terencechain Oct 17, 2024
cbe6129
suggested changes
fradamt Oct 22, 2024
0efcf13
Merge pull request #5 from fradamt/focil-patch
terencechain Oct 22, 2024
f151c7c
replace parent_root with il_committee_root
fradamt Oct 22, 2024
b181655
add timing logic (deaadlines) to IL processing
fradamt Oct 23, 2024
0681f8c
missing :
fradamt Oct 25, 2024
2861684
Merge pull request #6 from fradamt/focil-patch
terencechain Oct 29, 2024
e678deb
Use eip7805
terencechain Nov 4, 2024
e9fdfc6
@jtraglia's suggestions
terencechain Nov 8, 2024
6ce0077
Update new inclusion list comments
terencechain Nov 19, 2024
be471b8
P2p spec feedbacks
terencechain Nov 19, 2024
c443db0
Add get attester head
terencechain Nov 19, 2024
c8b6296
Fix typo and make CI happy
hwwhww Dec 2, 2024
7fa44f4
fix byte limit check
fradamt Dec 16, 2024
84acfbb
ensure valid committee with <16 validators
nerolation Dec 30, 2024
702b9e5
Merge pull request #7 from nerolation/patch-1
terencechain Jan 2, 2025
561aed7
Merge branch 'dev' into focil-consensus
jtraglia Jan 14, 2025
c762a33
Add eip7805 to list of executable specs
jtraglia Jan 14, 2025
20b0c50
Fix simple linter warnings
jtraglia Jan 14, 2025
aca38ea
Fix some more linter issues
jtraglia Jan 14, 2025
1c6761a
Add execution_requests_list to notify_new_payload
jtraglia Jan 14, 2025
d63042e
Fix things regarding engine API
jtraglia Jan 14, 2025
86c86d0
Update eth2spec before linting
jtraglia Jan 14, 2025
7d8e6fb
Expand acronym in DOMAIN_IL_COMMITTEE and IL_COMMITTEE_SIZE
jtraglia Jan 14, 2025
19b50a6
Rename il_transactions to inclusion_list_transactions
jtraglia Jan 14, 2025
f25efbf
Change engine API param type to Sequence[Transaction]
jtraglia Jan 14, 2025
5a37c91
Fix various nits
jtraglia Jan 14, 2025
c411d45
Replace "since" with "up to"
jtraglia Jan 14, 2025
dcd31d6
Pull in electra's changes to verify_and_notify_new_payload
jtraglia Jan 14, 2025
3652904
Change inclusion_list_transactions to a Sequence
jtraglia Jan 15, 2025
2781459
Make some improvements to fork-choice doc
jtraglia Jan 15, 2025
7a2cb7a
Make time params work with minimal
jtraglia Jan 24, 2025
91054e6
Fix networking toc
jtraglia Jan 24, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
183 changes: 183 additions & 0 deletions specs/_features/eip7805/beacon-chain.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
# EIP-7805 -- The Beacon Chain

## Table of contents

<!-- TOC -->
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->

- [Introduction](#introduction)
- [Preset](#preset)
- [Domain types](#domain-types)
- [Inclusion List Committee](#inclusion-list-committee)
- [Execution](#execution)
- [Containers](#containers)
- [New containers](#new-containers)
- [`InclusionList`](#inclusionlist)
- [`SignedInclusionList`](#signedinclusionlist)
- [Predicates](#predicates)
- [New `is_valid_inclusion_list_signature`](#new-is_valid_inclusion_list_signature)
- [Beacon State accessors](#beacon-state-accessors)
- [`get_inclusion_list_committee`](#get_inclusion_list_committee)
- [Beacon chain state transition function](#beacon-chain-state-transition-function)
- [Execution engine](#execution-engine)
- [Request data](#request-data)
- [Modified `NewPayloadRequest`](#modified-newpayloadrequest)
- [Engine APIs](#engine-apis)
- [Modified `notify_new_payload`](#modified-notify_new_payload)
- [Modified `verify_and_notify_new_payload`](#modified-verify_and_notify_new_payload)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->
<!-- /TOC -->

## Introduction

This is the beacon chain specification to add EIP-7805 / fork-choice enforced, committee-based inclusion list (FOCIL) mechanism to allow forced transaction inclusion. Refers to the following posts:
- [Fork-Choice enforced Inclusion Lists (FOCIL): A simple committee-based inclusion list proposal](https://ethresear.ch/t/fork-choice-enforced-inclusion-lists-focil-a-simple-committee-based-inclusion-list-proposal/19870/1)
- [FOCIL CL & EL workflow](https://ethresear.ch/t/focil-cl-el-workflow/20526)
*Note:* This specification is built upon [Electra](../../electra/beacon_chain.md) and is under active development.

## Preset

### Domain types

| Name | Value |
| - | - |
| `DOMAIN_IL_COMMITTEE` | `DomainType('0x0C000000')` # (New in EIP-7805)|
terencechain marked this conversation as resolved.
Show resolved Hide resolved

### Inclusion List Committee

| Name | Value |
| - | - |
| `IL_COMMITTEE_SIZE` | `uint64(2**4)` (=16) # (New in EIP-7805) |
jtraglia marked this conversation as resolved.
Show resolved Hide resolved

### Execution

| Name | Value |
| - | - |
| `MAX_TRANSACTIONS_PER_INCLUSION_LIST` | `uint64(1)` # (New in EIP-7805) TODO: Placeholder |
terencechain marked this conversation as resolved.
Show resolved Hide resolved

## Containers

### New containers

#### `InclusionList`

```python
class InclusionList(Container):
slot: Slot
validator_index: ValidatorIndex
inclusion_list_committee_root: Root
transactions: List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST]
```

#### `SignedInclusionList`

```python
class SignedInclusionList(Container):
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason why we introduce a SignedInclusionList container instead of just adding the signature field directly onto InclusionList?

Copy link
Member

@nflaig nflaig Jan 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When the beacon node initially retrieves the IL transactions from the execution client wouldn't it create the (unsigned) InclusionList and pass it through the beacon-api to the validator client to sign over it? only after that the validator client would submit the SignedInclusionList to be broadcast

message: InclusionList
signature: BLSSignature
```

### Predicates

#### New `is_valid_inclusion_list_signature`

```python
def is_valid_inclusion_list_signature(
state: BeaconState,
signed_inclusion_list: SignedInclusionList) -> bool:
"""
Check if ``signed_inclusion_list`` has a valid signature
terencechain marked this conversation as resolved.
Show resolved Hide resolved
"""
message = signed_inclusion_list.message
index = message.validator_index
pubkey = state.validators[index].pubkey
domain = get_domain(state, DOMAIN_IL_COMMITTEE, compute_epoch_at_slot(message.slot))
signing_root = compute_signing_root(message, domain)
return bls.FastAggregateVerify(pubkey, signing_root, signed_inclusion_list.signature)
terencechain marked this conversation as resolved.
Show resolved Hide resolved
```

### Beacon State accessors

#### `get_inclusion_list_committee`

```python
def get_inclusion_list_committee(state: BeaconState, slot: Slot) -> Vector[ValidatorIndex, IL_COMMITTEE_SIZE]:
epoch = compute_epoch_at_slot(slot)
seed = get_seed(state, epoch, DOMAIN_IL_COMMITTEE)
indices = get_active_validator_indices(state, epoch)
start = (slot % SLOTS_PER_EPOCH) * IL_COMMITTEE_SIZE
end = start + IL_COMMITTEE_SIZE
return [indices[compute_shuffled_index(uint64(i), uint64(len(indices)), seed)] for i in range(start, end)]
```

## Beacon chain state transition function

### Execution engine

#### Request data

##### Modified `NewPayloadRequest`

```python
@dataclass
class NewPayloadRequest(object):
execution_payload: ExecutionPayload
versioned_hashes: Sequence[VersionedHash]
parent_beacon_block_root: Root
execution_requests: ExecutionRequests
il_transactions: List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST] # [New in EIP-7805]
```

#### Engine APIs

##### Modified `notify_new_payload`

*Note*: The function `notify_new_payload` is modified to include the additional `il_transactions` parameter in EIP-7805.

```python
def notify_new_payload(self: ExecutionEngine,
execution_payload: ExecutionPayload,
execution_requests: ExecutionRequests,
parent_beacon_block_root: Root,
il_transactions: List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST] ) -> bool:
"""
Return ``True`` if and only if ``execution_payload`` and ``execution_requests``
are valid with respect to ``self.execution_state``.
"""
...
```

##### Modified `verify_and_notify_new_payload`

*Note*: The function `verify_and_notify_new_payload` is modified to pass the additional parameter `il_transactions`
when calling `notify_new_payload` in EIP-7805.

```python
def verify_and_notify_new_payload(self: ExecutionEngine,
new_payload_request: NewPayloadRequest) -> bool:
"""
Return ``True`` if and only if ``new_payload_request`` is valid with respect to ``self.execution_state``.
"""
execution_payload = new_payload_request.execution_payload
execution_requests = new_payload_request.execution_requests
parent_beacon_block_root = new_payload_request.parent_beacon_block_root
il_transactions = new_payload_request.il_transactions # [New in EIP-7805]

if not self.is_valid_block_hash(execution_payload, parent_beacon_block_root):
return False

if not self.is_valid_versioned_hashes(new_payload_request):
return False

# [Modified in EIP-7805]
if not self.notify_new_payload(
execution_payload,
execution_requests,
parent_beacon_block_root,
il_transactions):
return False

return True
```
109 changes: 109 additions & 0 deletions specs/_features/eip7805/fork-choice.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# EIP-7805 -- Fork Choice

## Table of contents
<!-- TOC -->
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->

- [Introduction](#introduction)
- [Helpers](#helpers)
- [New `validate_inclusion_lists`](#new-validate_inclusion_lists)
- [Modified `Store`](#modified-store)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->
<!-- /TOC -->

## Introduction

This is the modification of the fork choice accompanying the EIP-7805 upgrade.

## Configuration

| Name | Value | Unit | Duration |
| - | - | :-: | :-: |
| `VIEW_FREEZE_DEADLINE` | `uint64(9)` | seconds | 9 seconds | # [New in EIP-7805]
terencechain marked this conversation as resolved.
Show resolved Hide resolved

## Helpers

### New `validate_inclusion_lists`

```python
def validate_inclusion_lists(store: Store, inclusion_list_transactions: List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST * IL_COMMITTEE_SIZE], execution_payload: ExecutionPayload) -> bool:
"""
Return ``True`` if and only if the input ``inclusion_list_transactions`` satifies validation, that to verify if the `execution_payload` satisfies `inclusion_list_transactions` validity conditions either when all transactions are present in payload or when any missing transactions are found to be invalid when appended to the end of the payload unless the block is full.
jtraglia marked this conversation as resolved.
Show resolved Hide resolved
"""
...
```

### Modified `Store`
**Note:** `Store` is modified to track the seen inclusion lists.
jtraglia marked this conversation as resolved.
Show resolved Hide resolved

```python
@dataclass
class Store(object):
time: uint64
genesis_time: uint64
justified_checkpoint: Checkpoint
finalized_checkpoint: Checkpoint
unrealized_justified_checkpoint: Checkpoint
unrealized_finalized_checkpoint: Checkpoint
proposer_boost_root: Root
equivocating_indices: Set[ValidatorIndex]
blocks: Dict[Root, BeaconBlock] = field(default_factory=dict)
block_states: Dict[Root, BeaconState] = field(default_factory=dict)
block_timeliness: Dict[Root, boolean] = field(default_factory=dict)
checkpoint_states: Dict[Checkpoint, BeaconState] = field(default_factory=dict)
latest_messages: Dict[ValidatorIndex, LatestMessage] = field(default_factory=dict)
unrealized_justifications: Dict[Root, Checkpoint] = field(default_factory=dict)
inclusion_lists: Dict[Tuple[Slot, Root], List[InclusionList]] = field(default_factory=dict) # [New in EIP-7805]
inclusion_list_equivocators: Dict[Tuple[Slot, Root], Set[ValidatorIndex]] = field(default_factory=dict) # [New in EIP-7805]
```

### New `on_inclusion_list`

`on_inclusion_list` is called to import `signed_inclusion_list` to the fork choice store.
```python
jtraglia marked this conversation as resolved.
Show resolved Hide resolved
def on_inclusion_list(
store: Store,
signed_inclusion_list: SignedInclusionList,
inclusion_list_committee: Vector[ValidatorIndex, IL_COMMITTEE_SIZE]]) -> None:
"""
``on_inclusion_list`` verify the inclusion list before importing it to fork choice store.
terencechain marked this conversation as resolved.
Show resolved Hide resolved
If there exists more than 1 inclusion list in store with the same slot and validator index, remove the original one.
Copy link
Member

@jtraglia jtraglia Nov 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is an out-of-date comment. Where is "remove the original one" happening exactly? It appears that it adds the validator to inclusion_list_equivocators instead now.

Copy link
Contributor

@ensi321 ensi321 Nov 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is an out-of-date comment. Where is "remove the original one" happening exactly? It appears that it adds the validator to inclusion_list_equivocators instead now.

I thought it's the other way around?

p2p only accepts an IL if it's the first or second IL broadcasted by the peer.

The responsibility of picking out the equivocators should be on the p2p side. Beacon node just replace the first IL with the second IL if that happens

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comments are outdated, and different client implementations may handle this differently. Essentially, we want to:

  • Cache at least one inclusion list
  • Re-broadcast up to two inclusion lists over P2P
  • Track which inclusion lists have been equivocated (e.g., identify the specific validator)
  • When calling the EL to validate inclusion lists, ignore those that have been equivocated (discard them from being sent to the EL)
  • Regarding whether to prioritize the first or second inclusion list, I'm leaning towards the first as it seems simpler

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment is outdated, yeah. It doesn't really matter which one we keep, we could also not keep either since anyway we ignore them once we know that it is an equivocation. For spec simplicity it seemed easier to just not do anything instead of removing, and also perhaps it makes sense to keep the first, just because if you're the builder you would probably rather use it even if there's an equivocation, just in case, and you'd rather use the first since you saw it earlier and from your perspective it is more likely to be enforced by someone

"""
message = signed_inclusion_list.message
# Verify inclusion list slot is either from the current or previous slot
assert get_current_slot(store) in [message.slot, message.slot + 1]

time_into_slot = (store.time - store.genesis_time) % SECONDS_PER_SLOT
is_before_attesting_interval = time_into_slot < SECONDS_PER_SLOT // INTERVALS_PER_SLOT
# If the inclusion list is from the previous slot, ignore it if already past the attestation deadline
if get_current_slot(store) == message.slot + 1:
assert is_before_attesting_interval

# Sanity check that the given `inclusion_list_committee` matches the root in the inclusion list
root = message.inclusion_list_committee_root
assert hash_tree_root(inclusion_list_committee) == root

# Verify inclusion list validator is part of the committee
validator_index = message.validator_index
assert validator_index in inclusion_list_committee

# Verify inclusion list signature
assert is_valid_inclusion_list_signature(state, signed_inclusion_list)

is_before_freeze_deadline = get_current_slot(store) == message.slot and time_into_slot < VIEW_FREEZE_DEADLINE

# Do not process inclusion lists from known equivocators
if validator_index not in inclusion_list_equivocators[(message.slot, root)]:
if validator_index in [il.validator_index for il in inclusion_lists[(message.slot, root)]]:
il = [il for il in inclusion_lists[(message.slot, root)] if il.validator_index == validator_index][0]
if not il == message:
terencechain marked this conversation as resolved.
Show resolved Hide resolved
# We have equivocation evidence for `validator_index`, record it as equivocator
inclusion_list_equivocators[(message.slot, root)].add(validator_index)
# This inclusion list is not an equivocation. Store it if prior to the view freeze deadline
elif is_before_freeze_deadline:
inclusion_lists[(message.slot, root)].append(message)
```


87 changes: 87 additions & 0 deletions specs/_features/eip7805/p2p-interface.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# EIP-7805 -- Networking

This document contains the consensus-layer networking specification for EIP-7805.

<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->

- [Time parameters](#time-parameters)
- [Configuration](#configuration)
- [The gossip domain: gossipsub](#the-gossip-domain-gossipsub)
- [Topics and messages](#topics-and-messages)
- [Global topics](#global-topics)
- [`inclusion_list`](#inclusion_list)
- [The Req/Resp domain](#the-reqresp-domain)
- [Messages](#messages)
- [InclusionListByCommitteeIndices v1](#inclusionlistbycommitteeindices-v1)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->

### Time parameters

| Name | Value | Unit | Duration |
| - | - | :-: | :-: |
| `attestation_deadline` | `uint64(4)` | seconds | 4 seconds |

### Configuration

| `MAX_REQUEST_INCLUSION_LIST` | `2**4` (= 16) | Maximum number of inclusion list in a single request |

### The gossip domain: gossipsub

#### Topics and messages

The new topics along with the type of the `data` field of a gossipsub message are given in this table:

| Name | Message Type |
|-------------------------------|------------------------------------------------------|
| `inclusion_list` | `SignedInclusionList` [New in EIP-7805] |

##### Global topics

EIP-7805 introduces a new global topic for inclusion lists.

###### `inclusion_list`

This topic is used to propagate signed inclusion list as `SignedInclusionList`.
The following validations MUST pass before forwarding the `inclusion_list` on the network, assuming the alias `message = signed_inclusion_list.message`:

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aren't we also limiting IL size by MAX_BYTES_PER_INCLUSION_LIST = 8192 bytes as indicated in the EIP?

- _[REJECT]_ The slot `message.slot` is equal to the previous or current slot.
- _[IGNORE]_ The slot `message.slot` is equal to the current slot, or it is equal to the previous slot and the current time is less than `attestation_deadline` seconds into the slot.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- _[IGNORE]_ The slot `message.slot` is equal to the current slot, or it is equal to the previous slot and the current time is less than `attestation_deadline` seconds into the slot.
- _[IGNORE]_ The slot `message.slot` is equal to the previous slot and the current time is less than `attestation_deadline` seconds into the slot.

- _[IGNORE]_ The `inclusion_list_committee` for slot `message.slot` on the current branch corresponds to `message.inclusion_list_committee_root`, as determined by `hash_tree_root(inclusion_list_committee) == message.inclusion_list_committee_root`.
- _[REJECT]_ The validator index `message.validator_index` is within the `inclusion_list_committee` corresponding to `message.inclusion_list_committee_root`.
- _[REJECT]_ The transactions `message.transactions` length is within upperbound `MAX_TRANSACTIONS_PER_INCLUSION_LIST`.
- _[IGNORE]_ The `message` is either the first or second valid message received from the validator with index `message.validator_index`.
- _[REJECT]_ The signature of `inclusion_list.signature` is valid with respect to the validator index.


### The Req/Resp domain
jtraglia marked this conversation as resolved.
Show resolved Hide resolved

#### Messages

##### InclusionListByCommitteeIndices v1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need another endpoint like InclusionListByRange since recently joined nodes need to request past ILs to sync blocks since the last checkpoint?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do not. Inclusion lists only matter for the current slot. When a node is syncing blocks from past slots, the inclusion list check can be skipped


**Protocol ID:** `/eth2/beacon_chain/req/inclusion_list_by_committee_indices/1/`

The `<context-bytes>` field is calculated as `context = compute_fork_digest(fork_version, genesis_validators_root)`:

[1]: # (eth2spec: skip)

| `fork_version` | Chunk SSZ type |
|------------------------|------------------------------------------|
| `EIP-7805_FORK_VERSION` | `EIP-7805.SignedInclusionList` |
jtraglia marked this conversation as resolved.
Show resolved Hide resolved

Request Content:
```
(
slot: Slot
committee_indices: Bitvector[IL_COMMITTEE_SIZE]
)
```

Response Content:
```
(
List[SignedInclusionList, MAX_REQUEST_INCLUSION_LIST]
)
```
Loading
Loading