Executable Spec instead of Core EIPs

Discussion link

https://ethereum-magicians.org/t/core-eips-in-an-executable-spec-world/8640

Contents

• Executable Spec instead of Core EIPs
  • Discussion link
  • Contents
  • Context
  • Constraints
  • High Level Proposals
    • 1. Move Core EIPs directly in EL/CL specs
    • 2. Use Core EIPs alongside EL/CL specs
  • Implementation Details
    • EIP Location & Format
    • Core EIPs Template
    • Executable Specifications
    • EIP Process
  • Keeping Core EIPs vs. Specs-Only
    • EIPs Repository
    • Specs Only

Context

Today, Core EIPs are used to specify changes affecting the execution layer (EL) which need to be activated through coordinated network upgrades (a.k.a. soft/hard forks).

Core EIPs currently serve two purposes: explaining the motivation/rationale/security implications of the change and formally specifying it. While the former is very useful, and helps the Ethereum community understand and debate changes, the latter isn’t ideal for a few reasons:

1. The EIP format is very different than the “full specification” for the EL, the Yellow Paper. The YP often lags behind network upgrades for several months before reflecting the current state of the network. This means the actual specification is often “Yellow Paper + EIPs A, B, C”
2. Complex EIPs which touch several parts of Ethereum, e.g. EIP-1559, can require authors re-write a “skeleton spec” for Ethereum to highlight the changes they introduce.
3. The consensus layer (CL) uses a different process: an executable specification on which changes are directly proposed as a diff. Changes’ motivation/rationale/security implications are not generally detailled in EIPs.
4. Changes spanning both the consensus and execution layer are awkward to specify and now require “proto EIPs” to act as a meta-spec, e.g. for withdrawals, or many crosslinks, e.g. for 4844

We now (almost!) have an executable specification for the EL as well (link). Here is a proposal for how we could harmonize the process across EL + CL.

Constraints

A few things we are trying to optimize for here:

1. EIP Editors bandwidth: EIP Editors are part time volunteers with incredibly high opportunity cost. We already have less than we would like, and we need to make sure we don’t burn out or drive away the ones we do have.
   • Links: One particularly challenging topic for EIP editors is external links. Currently, they aren’t allowed. Because links vary in quality and degrade over time, making a judgement call about them is something that adds a ton of overhead to EIP editors.
2. Coherent process across EL + CL: while it’s OK if there are small differences between the EL & CL processes, post-merge, there is just one Ethereum. We need to make sure that the overall process is coherent across both the EL + CL layers, and legible to the broader Ethereum community.
3. Varying Quality of EIPs: EIPs are open for anyone to submit, which, while good, leads to a lot of stale, poor quality, duplicate, etc. proposals. We need to make sure that these can be easily triaged from potential proposals for an upgrade and from EIPs which have been included in an upgrade.
4. EIP Champions bandwidth: It’s hard to get an EIP implemented in a network update, we want to make sure we’re not creating unnecessary work for the people who endeavour to try.
5. Little external dependencies: Changes to the Ethereum protocol should ideally use processes which are tool-agnostic. Whenever possible, for example, we should use code or markdown rather than, say, Github PRs or Issues.

High Level Proposals

1. Move Core EIPs directly in EL/CL specs

Separate Core EIPs from other EIP categories, and store them directly in the executable and consensus spec repositories. The technical “Specification” section would instead be represented by the accompanying changes to the specification itself, while the other text-based sections would move into reStructuredText files alongside the code changes in the same repository.

2. Use Core EIPs alongside EL/CL specs

Similar to (1), but keep using Core EIPs as a way to describe changes, provide rationale, etc. and link to the EL and/or CL specs for actual implementation details.

Implementation Details

EIP Location & Format

Currently all EIPs live in the ethereum/EIPs repository.

Under this proposal, the specification for changes would instead live in ethereum/execution-specs and ethereum/consensus-specs under a subdirectory.

If Core EIPs were moved to the specifications repository as well, they would be formatted with reStructuredText for consistency with the inline documentation.

Core EIPs Template

Make the following changes to the template:

Specification

If Core EIPs remain in the EIP repository, allow links to ethereum/execution-specs and ethereum/consensus-specs in this section.

Test Cases

TBD.

Reference Implementation

Make this section optional, for cases where things are easier to express using a different notation, but the implementation(s) in the specs repositories should be considered canonical.

Notes:

• This will require maintaining two templates: one for Core EIPs and one for everything else. While this isn’t the end of the world, we should be mindful of it.

Executable Specifications

Use branches to highlight EIPs in various stages, specifically:

• master: what is currently live on the Ethereum mainnet
• forks/$FORK_NAME: PR target branch for EIPs slated for inclusion in $FORK_NAME
• eips/$EIP_NUMBER: a specific EIP’s specification

When an EIP goes from being proposed to being Considered for Inclusion (CFI) for a fork, merge eips/$EIP_NUMBER into forks/$FORK_NAME. When $FORK_NAME is deployed on mainnet, merge forks/$FORK_NAME into master.

Note: if a CFI’d EIP is then removed from the fork specification, the merging of it into $FORK_NAME will have to be reverted.

This means that specifications for historical EIPs will forever live as branches in the executable spec repository. If the EIP wants to be proposed for another fork, an author must update its branch by rebasing it.

EIP Process

Currently, Core EIPs roughly follow this process:

Note that EIPs tend to be made CFI when they are either in Draft or Review.

Instead if we, as an example, combine this with the Executable Specifications flow using a single repository, we get:

If we were to keep Core EIPs alongside Executable Specifications, you would have the following flow:

Note: this would apply across both the EL and CL, implying that CL changes would also require an EIP for specification.

Keeping Core EIPs vs. Specs-Only

One contentious aspect of moving to executable specifications is whether to keep using Core EIPs in theeips repository to describe non-code aspects of the change, or whether to exclusively use exectuable specifications for everything.

EIPs Repository

Pros

• Well known within the broader Ethereum community, legitimacy around the name/process/etc.
• Basic infrastructure exists (eips.ethereum.org, EIP bot)
  • Will require changes
• Clear identifiers for changes (“EIP-X”, everyone knows what e.g. EIP-1559, ERC-20, ERC-721, etc. refers to).
  • Counterpoint: people may get used to things like “ELIP-X” and “CLIP-Y”
• Can use the same number to describe EL + CL changes (e.g. EIP-4844) rather than having a different PR/branch number to specify a change across the EL & CL (e.g. CL “withdrawals” and EIP-4895)

Cons

• Awkward to have to wait for an EIP draft to be merged before you can have a branch number on the EL/CL spec.
  • Counterpoint: may reduce the number of branches on the specs?
• Need to maintain 2-3 resources (EIP + EL/CL specs) rather than 1-2 and ensure they are in alignment with each other.
• Core EIP template will likely diverge from general template.

Specs Only

Pros

• Everything is self-contained within the specs repo, no dependency on other repositories
  • Counterpoint: features which span EL + CL
    • Countercounterpoint: EL + CL spec could end up in the same repository
• Can have a different template to describe changes, adapted to core protocol changes, without affecting EIPs

Cons

• No unified naming scheme / less community recognition
• When EIPs get bundled and become a fork, it may be harder to untangle the different changes which went into a fork.
  • Counterpoint: will still have English descriptions somewhere in specs repos, and this will still be true to an extent if we use the executable specs at all.