Proposal: Multi-Fork Architecture for leanSpec

# Proposal: Multi-Fork Architecture for leanSpec ## Motivation leanSpec currently operates as a single-fork codebase — one `State`, one `Block`, one set of processing functions. Each devnet is tracked as a git commit hash in `VERSIONS.md`, meaning the only way to distinguish devnet0 from devnet1 is by checking out different commits. There is no multi-fork coexistence mechanism in the spec code itself. As leanSpec progresses through devnets, this approach will create the same pain points that both existing Ethereum spec repositories already suffer from. This proposal introduces a protocol-and-fork-class architecture that avoids those pitfalls while leveraging Python's native type system and leanSpec's existing `subspecs/` structure. ### At a Glance: Three Approaches Compared | | **consensus-specs** | **execution-specs (EELS)** | **leanSpec (proposed)** | |---|---|---|---| | **Fork mechanism** | Markdown code-gen with name-based override | Full directory copy per fork | Python class inheritance | | **Code duplication** | Medium — full SSZ container redefinition per fork | Very high — entire spec copied | Low — only overrides, inherited methods carry forward | | **Fork chain definition** | `PREVIOUS_FORK_OF` dict maintained in 2+ places | Implicit (directory ordering) | Python class hierarchy: `class Devnet1(Devnet0)` — defined once | | **Adding a new fork** | Touch 10+ files (markdown, pysetup, presets, tests, configs) | Copy entire directory, modify | One new package, inherit from parent, override what changes | | **Reading a complete fork** | Mentally merge N fork-diff documents | One self-contained directory ✓ | IDE "go to definition" traces inheritance; single class is entry point | | **Cross-fork code sharing** | Via build system merge — fragile | None — zero cross-fork imports | Automatic via inheritance | | **Bug fix propagation** | Fix once in earliest fork, rebuild | Must patch every fork directory | Fix once, all inheriting forks get it | | **Non-linear fork branching** | Not supported — linear chain only | Requires duplicating from any fork | `class Experiment(Devnet0)` — branch from any point | | **Type safety** | Runtime only (name-based text override) | Runtime only (no cross-fork types) | Static — type checkers verify Protocol conformance at CI | | **Sub-spec composability** | No formal mechanism | No formal mechanism | Capability Protocols (`PQCapable`, `NetworkCapable`) | | **Test targeting by fork** | Handwritten `@with_X_and_later` decorators per fork | Per-fork test directories | `@requires(PQCapable)` — derived from capability Protocols | | **Experimentation speed** | Slow — full build pipeline per feature | Slow — full copy + manual diff | Fast — new class with overrides, immediately runnable | | **Formal verification alignment** | `f(state, block)` ✓ | `f(state, block)` ✓ | Methods on `State` today; migration path to standalone (see [Appendix A](#appendix-a-migrating-processing-logic-off-state)) | --- ## How the Code Will Look After This Refactoring The core idea: define an abstract **`ForkProtocol`** interface that every devnet must implement, then create **concrete fork classes** (`Devnet0`, `Devnet1`, …) that implement it. Each fork provides its own state and container types via Pydantic model inheritance, and only overrides the processing logic that actually changes. An **era-aware runner** dispatches to the correct fork at runtime. ### Directory Structure ``` src/lean_spec/ ├── core/ # NEW — protocol definitions │ ├── __init__.py │ ├── protocol.py # ForkProtocol abstract base class │ ├── capabilities.py # Sub-spec Protocol interfaces (PQCapable, etc.) │ └── runner.py # SpecRunner — dispatches to correct fork ├── types/ # UNCHANGED — SSZ primitives, shared by all forks ├── subspecs/ # UNCHANGED — cryptographic & networking implementations │ ├── containers/ # Shared container definitions (base types) │ ├── poseidon2/ │ ├── xmss/ │ ├── forkchoice/ │ ├── networking/ │ └── ssz/ ├── forks/ # NEW — one package per fork │ ├── __init__.py │ ├── registry.py # Fork registry mapping │ ├── devnet0/ │ │ ├── __init__.py │ │ ├── spec.py # Devnet0(ForkProtocol) — wraps current code │ │ ├── state.py # Devnet0State = current State (alias) │ │ └── containers.py # Fork-specific container overrides (if any) │ ├── devnet1/ │ │ ├── __init__.py │ │ ├── spec.py # Devnet1(Devnet0) — inherits, overrides only changes │ │ ├── state.py # Devnet1State(Devnet0State) — adds new fields │ │ └── containers.py │ └── experimental/ # Non-linear experimental branches │ └── rainbow_staking/ │ ├── spec.py # RainbowExperiment(Devnet0) — branches from any fork │ └── state.py └── snappy/ # UNCHANGED ``` ### Architecture Overview ```mermaid graph TB subgraph "core/ — Interfaces" FP[ForkProtocol Abstract Base Class] PQ[PQCapable Protocol] FC[ForkChoiceCapable Protocol] NC[NetworkCapable Protocol] SR[SpecRunner Era Combinator] end subgraph "forks/ — Implementations" D0[Devnet0] D1[Devnet1] D2[Devnet2] EXP[RainbowExperiment] end subgraph "subspecs/ — Shared Libraries" XMSS[xmss/] NET[networking/] FCH[forkchoice/] SSZ[ssz/] end FP --> D0 D0 --> D1 D1 --> D2 D0 --> EXP PQ -.->|"implements"| D1 PQ -.->|"implements"| D2 NC -.->|"implements"| D2 D1 -->|"delegates to"| XMSS D2 -->|"delegates to"| NET D0 -->|"delegates to"| FCH D0 -->|"delegates to"| SSZ SR -->|"dispatches"| D0 SR -->|"dispatches"| D1 SR -->|"dispatches"| D2 style FP fill:#4a90d9,color:#fff style PQ fill:#6ab04c,color:#fff style FC fill:#6ab04c,color:#fff style NC fill:#6ab04c,color:#fff style SR fill:#e17055,color:#fff ``` ### What the Code Looks Like **The protocol interface** — every fork must satisfy this contract: ```python # src/lean_spec/core/protocol.py from abc import ABC, abstractmethod from typing import ClassVar from lean_spec.types import Container class ForkProtocol(ABC): """Abstract fork — every devnet must implement this interface.""" StateType: ClassVar[type[Container]] BlockType: ClassVar[type[Container]] @abstractmethod def process_block(self, state: Container, block: Container) -> Container: ... @abstractmethod def state_transition(self, state: Container, block: Container) -> Container: ... @abstractmethod def process_attestations(self, state: Container, attestations) -> Container: ... def upgrade_state(self, state: Container) -> Container: """Migrate state from the previous fork. Default: identity.""" return state ``` **Devnet0** — a thin wrapper around today's code, zero disruption: ```python # src/lean_spec/forks/devnet0/spec.py from lean_spec.core.protocol import ForkProtocol from lean_spec.subspecs.containers import Block, State, Attestation class Devnet0(ForkProtocol): StateType = State BlockType = Block def process_block(self, state: State, block: Block) -> State: return state.process_block(block) def state_transition(self, state: State, block: Block) -> State: return state.state_transition(block) def process_attestations(self, state, attestations): return state.process_attestations(attestations) ``` **Devnet1** — inherits everything, overrides only what changes: ```python # src/lean_spec/forks/devnet1/state.py from lean_spec.forks.devnet0.state import Devnet0State from lean_spec.types import Bytes32 class Devnet1State(Devnet0State): pq_aggregation_root: Bytes32 # New field # src/lean_spec/forks/devnet1/spec.py from lean_spec.forks.devnet0.spec import Devnet0 from lean_spec.core.capabilities import PQCapable class Devnet1(Devnet0, PQCapable): StateType = Devnet1State def verify_pq_signature(self, sig, pubkey, message) -> bool: ... # Delegates to subspecs/xmss/ # process_block, state_transition, process_attestations # all INHERITED from Devnet0 — no copy-paste needed def upgrade_state(self, state) -> Devnet1State: return Devnet1State( **state.model_dump(), pq_aggregation_root=Bytes32.zero(), ) ``` **Sub-spec capability interfaces** — the existing `subspecs/` structure maps directly: ```python # src/lean_spec/core/capabilities.py from typing import Protocol, runtime_checkable @runtime_checkable class PQCapable(Protocol): """Fork supports post-quantum signatures (XMSS).""" def verify_pq_signature(self, sig, pubkey, message) -> bool: ... @runtime_checkable class ForkChoiceCapable(Protocol): """Fork has a fork-choice mechanism.""" def get_head(self, store) -> Bytes32: ... def on_block(self, store, block) -> object: ... @runtime_checkable class NetworkCapable(Protocol): """Fork supports P2P networking.""" def gossipsub_topics(self) -> list[str]: ... def reqresp_methods(self) -> list[str]: ... ``` **The era-aware runner** — dispatches to the correct fork: ```python # src/lean_spec/core/runner.py class SpecRunner: def __init__(self, forks: dict[str, ForkProtocol]): self._forks = forks def get_fork(self, name: str) -> ForkProtocol: return self._forks[name] def process_block(self, state, block, fork_name: str): return self._forks[fork_name].process_block(state, block) def upgrade(self, state, from_fork: str, to_fork: str): return self._forks[to_fork].upgrade_state(state) ``` --- ## Why This Approach — Compared to the Existing Ethereum Specs ### Problems with the consensus-specs' code-generation inheritance The `ethereum/consensus-specs` uses Markdown files with embedded Python code blocks, compiled by a build system that merges definitions by name across forks. A `PREVIOUS_FORK_OF` dictionary defines the fork chain, and later forks override earlier definitions by redeclaring functions and SSZ containers with the same name. ```mermaid graph LR subgraph "consensus-specs build pipeline" MD1[phase0/ beacon-chain.md] --> MERGE[combine_spec_objects name-based override] MD2[altair/ beacon-chain.md] --> MERGE MD3[bellatrix/ beacon-chain.md] --> MERGE MERGE --> SB[SpecBuilder per-fork] SB --> PY[altair.py complete module] end style MERGE fill:#e17055,color:#fff ``` **Key pain points this architecture avoids:** - **Fragile name-based overrides.** In the consensus-specs, if you rename a function, nothing breaks at the textual level — the old name simply persists from the earlier fork, and you now have two definitions doing different things. The proposed approach uses Python class inheritance, where renaming or removing a method produces an immediate error from the type checker. - **No single complete document.** Understanding Capella in the consensus-specs requires mentally merging four separate fork-diff documents (Phase0 → Altair → Bellatrix → Capella). With fork classes, `Devnet1` is a single class you can inspect. IDE "go to definition" traces any inherited method back to its source. - **Fork chain maintained in multiple places.** The `PREVIOUS_FORK_OF` mapping must be kept in sync between `pysetup` and test helpers — a known source of bugs. Here, the fork chain is expressed once through Python's class hierarchy: `class Devnet1(Devnet0)`. - **10+ files to update for a new feature.** Adding a feature to the consensus-specs requires touching markdown files, pysetup configs, test generators, preset files, and more. Here, a new fork is one package directory with a spec class that inherits from the previous fork. - **SSZ container bloat.** Each fork in the consensus-specs completely redefines `BeaconState` with all fields copied verbatim plus additions. Pydantic model inheritance (`Devnet1State(Devnet0State)`) adds only new fields — inherited fields carry forward automatically. ### Problems with the execution-specs' total isolation The `ethereum/execution-specs` (EELS) takes the opposite extreme: each fork is a complete, self-contained Python package with zero cross-fork imports. Creating a new fork means copying the entire previous fork directory. ```mermaid graph LR subgraph "EELS fork structure" F1[frontier/ complete snapshot] F2[homestead/ complete snapshot] F3[london/ complete snapshot] F4[paris/ complete snapshot] F5[cancun/ complete snapshot] F6[prague/ complete snapshot] end F1 -.-|"copy + modify"| F2 F2 -.-|"..."| F3 F3 -.-|"copy + modify"| F4 F4 -.-|"copy + modify"| F5 F5 -.-|"copy + modify"| F6 style F1 fill:#dfe6e9 style F2 fill:#dfe6e9 style F3 fill:#dfe6e9 style F4 fill:#dfe6e9 style F5 fill:#dfe6e9 style F6 fill:#dfe6e9 ``` **Key pain points this architecture avoids:** - **Massive code duplication.** Fifteen fork directories with thousands of duplicated lines. A bug fix in shared logic (trie operations, bloom filters) must be applied to every fork individually. With fork class inheritance, unchanged logic is inherited — you fix it once. - **Linear repository growth.** Every new fork adds a full copy of the entire spec. With the proposed approach, each new fork adds only a small package containing its overrides. - **No sharing of unchanged logic.** Even when two adjacent forks share 95% of their code, EELS duplicates 100%. Fork class inheritance shares unchanged methods automatically. ### What this architecture offers that neither does ```mermaid graph TD subgraph "Proposed Architecture" PROTO[ForkProtocol Interface] CAP1[PQCapable] CAP2[NetworkCapable] CAP3[ForkChoiceCapable] D0[Devnet0] D1[Devnet1] EXP1[ExperimentA branches from Devnet0] EXP2[ExperimentB branches from Devnet1] end PROTO --> D0 D0 --> D1 D0 --> EXP1 D1 --> EXP2 CAP1 -.-> D1 CAP2 -.-> EXP2 CAP3 -.-> D0 style PROTO fill:#4a90d9,color:#fff style CAP1 fill:#6ab04c,color:#fff style CAP2 fill:#6ab04c,color:#fff style CAP3 fill:#6ab04c,color:#fff ``` - **Non-linear fork inheritance.** An experimental fork can branch from any point — `RainbowExperiment(Devnet0)` doesn't need to include devnet1's changes. Neither existing spec repository supports this cleanly. - **Sub-spec conformance as typed contracts.** The `subspecs/` directory structure (xmss, networking, forkchoice) maps 1:1 onto Protocol interfaces. When devnet1 adds PQ signatures, it declares `PQCapable`. When devnet2 adds the networking stack, it also declares `NetworkCapable`. A type checker enforces these contracts at CI time. This validates the existing subspecs architecture and makes feature composition explicit. - **Type-checked fork consistency.** Python type checkers (ty, mypy, pyright) can verify that every fork satisfies its protocol contract. The consensus-specs rely on runtime testing; EELS relies on complete duplication. This catches errors earlier. - **Composable experimentation.** Creating a new experimental fork is `class MyExperiment(Devnet0)` with a few method overrides. No code generation, no directory duplication, no build system changes. --- ## How Sub-Specs Map Onto This Architecture leanSpec's modular `subspecs/` structure is one of its key architectural strengths. This proposal makes that structure load-bearing: | Sub-spec directory | Capability Protocol | First fork to implement | |---|---|---| | `subspecs/xmss/` | `PQCapable` | devnet1 (planned) | | `subspecs/forkchoice/` | `ForkChoiceCapable` | devnet0 | | `subspecs/networking/` | `NetworkCapable` | devnet2 (planned) | | `subspecs/ssz/` | (always available — fork-agnostic) | all forks | | `subspecs/poseidon2/` | (always available — fork-agnostic) | all forks | Each capability Protocol defines the methods a fork must implement to claim support. The sub-spec directories provide the implementations that fork classes delegate to. This keeps the sub-specs as reusable libraries while making their integration points explicit and verifiable. ### Conformance Testing Protocol conformance tests run against all registered forks: ```python import pytest from lean_spec.core.capabilities import PQCapable, ForkChoiceCapable from lean_spec.forks.devnet0.spec import Devnet0 from lean_spec.forks.devnet1.spec import Devnet1 def test_devnet0_is_not_pq_capable(): assert not isinstance(Devnet0(), PQCapable) def test_devnet1_is_pq_capable(): assert isinstance(Devnet1(), PQCapable) @pytest.mark.parametrize("fork_cls", [Devnet0, Devnet1]) def test_all_forks_implement_base_protocol(fork_cls): fork = fork_cls() assert hasattr(fork, 'process_block') assert hasattr(fork, 'state_transition') ``` --- ## Enabling EIP Experimentation Across Mainline and leanSpec The multi-fork architecture turns leanSpec into a **low-friction proving ground for EIPs** that are too ambitious for near-term mainline inclusion but critical to Ethereum's long-term direction. Where the consensus-specs' `_features/` directory requires touching 10+ files and coordinating with the full build pipeline to prototype a single feature, leanSpec's fork classes let a researcher create `class MyExperiment(Devnet0)` with a few overrides and immediately have a runnable, testable spec. This matters now because leanSpec and mainline are converging on the same problems from opposite directions. ### Mainline features that leanSpec forks can prototype **ePBS (EIP-7732).** Glamsterdam's headline — enshrined proposer-builder separation — restructures block production by splitting it into a proposer phase and a builder phase. An `ePBSExperiment(Devnet1)` fork class can override `process_block` to implement the two-phase commit, add an `EPBSCapable` Protocol interface, and generate test vectors against it — all without disrupting the PQ devnet progression. The mainline `epbs-devnet-0` took months to stand up; a leanSpec fork can iterate in days. **3-slot finality.** leanSpec's pq-devnets already run 3SF-mini. The multi-fork architecture lets us maintain the current 3SF implementation in `Devnet0` while experimenting with variants — different committee sizes, Orbit-style sampling, alternative justification thresholds — as parallel fork classes that share 90% of the code. As 3SF matures toward a mainline EIP, these variants produce the comparative data researchers need. **Rainbow staking.** Still conceptual in mainline (no EIP number), but a natural fit for an experimental fork that branches from any devnet and overrides the validator set management logic. The `RainbowExperiment(Devnet0)` pattern shown in the directory structure is exactly this use case — exploring 1 ETH minimum stake with heavy/light validator separation while the main devnet chain stays focused on PQ. **Protocol simplification.** Vitalik's "Simplifying the L1" vision — replacing the epoch/slot/committee system with 3SF, standardizing on SSZ, targeting ~200 lines for consensus — is what leanSpec embodies. Each simplification can be expressed as a fork that *removes* complexity from a parent, and the multi-fork architecture makes it easy to measure what breaks when you simplify. ### How features flow between the two tracks ```mermaid graph LR subgraph "leanSpec forks/" D0[pq-devnet-0] --> D1[pq-devnet-1] D1 --> D2[pq-devnet-2] D0 --> EXP1[ePBS experiment] D1 --> EXP2[rainbow staking] end subgraph "Mainline consensus-specs" F1[specs/_features/eipNNNN/] F2[specs/gloas/] F3[specs/heze/] end EXP1 -->|"test vectors + findings"| F1 D2 -->|"PQ spec + operational data"| F3 F2 -->|"EIPs to validate"| EXP1 style EXP1 fill:#f39c12,color:#fff style EXP2 fill:#f39c12,color:#fff style F1 fill:#95a5a6,color:#fff ``` The flow is bidirectional. Mainline EIPs at the Considered-for-Inclusion stage (like FOCIL, which was explicitly deferred from Glamsterdam) can be prototyped as leanSpec experimental forks, generating multi-client operational data faster than the mainline devnet pipeline. In the other direction, leanSpec features that mature through the pq-devnet progression — PQ signatures, signature aggregation, 3SF — can be extracted as `_features/` contributions to the consensus-specs, complete with test vectors from `uv run fill`. The key enabler is **non-linear fork inheritance**. Mainline's `_features/` must always extend the latest scheduled fork. leanSpec's experimental forks can branch from *any* point — testing ePBS on top of devnet0 without PQ signatures, or rainbow staking on top of devnet1 with them. This lets researchers isolate variables that the linear mainline pipeline cannot. --- ## Integration with the Existing Test Framework The testing package already has a `BaseFork` metaclass with comparison operators (`Devnet > Phase0`) and a `ForkRegistry`. The connection is direct: ```python # packages/testing/src/consensus_testing/forks/forks.py from framework.forks import BaseFork from lean_spec.forks.devnet0.spec import Devnet0 as Devnet0Spec from lean_spec.forks.devnet1.spec import Devnet1 as Devnet1Spec class Devnet0(BaseFork): spec = Devnet0Spec() @classmethod def name(cls) -> str: return "Devnet0" class Devnet1(Devnet0): spec = Devnet1Spec() @classmethod def name(cls) -> str: return "Devnet1" ``` The test vector filler (`uv run fill --clean --fork=devnet`) can dispatch to the correct fork class based on the `--fork` argument, using the `SpecRunner` or the test framework's own `ForkRegistry`. --- ## Fork-Scoped Test Targeting Not every test applies to every fork. A PQ signature verification test is meaningless against devnet0. An ePBS block processing test only makes sense for forks that implement ePBS. The consensus-specs solve this with handwritten decorators per fork — `@with_capella_and_later`, `@with_deneb_and_later` — each manually maintained and requiring updates when new forks are added. The proposed architecture can do better by deriving test applicability from two things that already exist: the **fork class hierarchy** and the **capability Protocol interfaces**. ### Three targeting mechanisms **Capability-based targeting** — the most powerful. A test declares what the fork must support, and the framework resolves which forks qualify: ```python from lean_spec.testing import requires, for_all_forks from lean_spec.core.capabilities import PQCapable, NetworkCapable @requires(PQCapable) def test_pq_signature_verification(fork): """Runs against Devnet1, Devnet2, ... — any fork implementing PQCapable.""" sig = create_test_pq_signature() assert fork.verify_pq_signature(sig, pubkey, message) @requires(PQCapable, NetworkCapable) def test_pq_attestation_gossip(fork): """Runs only against forks implementing BOTH capabilities.""" ... ``` The `@requires` decorator introspects the fork registry at collection time and parametrizes the test across all forks satisfying `isinstance(fork, PQCapable)`. When devnet2 adds `NetworkCapable`, every `@requires(NetworkCapable)` test automatically starts running against it — no manual decorator updates needed. **Fork-range targeting** — for tests tied to the linear devnet progression: ```python from lean_spec.testing import from_fork, until_fork, only_fork @from_fork("devnet1") def test_aggregation_root_in_state(fork): """Runs against devnet1 and all subsequent forks.""" state = fork.StateType.genesis() assert hasattr(state, 'pq_aggregation_root') @until_fork("devnet0") def test_bls_only_signatures(fork): """Runs against devnet0 only — superseded by PQ in later forks.""" ... @only_fork("devnet1") def test_devnet0_to_devnet1_upgrade(fork): """State migration test — only meaningful for the specific transition.""" ... ``` This uses the `BaseFork` metaclass comparison operators (`Devnet1 > Devnet0`) that the test framework already provides. Unlike the consensus-specs' approach, `@from_fork` doesn't need a new decorator per fork — it's parameterized. **Explicit fork list** — for tests that apply to a non-contiguous set: ```python from lean_spec.testing import for_forks @for_forks("devnet0", "epbs_experiment") def test_non_pq_block_processing(fork): """Runs against specific forks that share a characteristic not captured by a Protocol.""" ... ``` ### Implementation All three mechanisms produce standard pytest parametrization under the hood. The `@requires` decorator is the key piece: ```python # lean_spec/testing/decorators.py import pytest from lean_spec.forks.registry import FORK_REGISTRY def requires(*capabilities): """Parametrize test across all forks implementing the given capability Protocols.""" matching_forks = [ (name, fork) for name, fork in FORK_REGISTRY.items() if all(isinstance(fork, cap) for cap in capabilities) ] return pytest.mark.parametrize( "fork", [f for _, f in matching_forks], ids=[n for n, _ in matching_forks], ) def from_fork(earliest: str): """Parametrize test across all forks >= earliest in the fork ordering.""" forks = [ (name, fork) for name, fork in FORK_REGISTRY.items() if fork >= FORK_REGISTRY[earliest] ] return pytest.mark.parametrize( "fork", [f for _, f in forks], ids=[n for n, _ in forks], ) ``` ### Why this matters for cross-mainline experimentation When an experimental fork like `ePBSExperiment(Devnet1)` is added to the registry and declares `EPBSCapable`, every test decorated with `@requires(EPBSCapable)` immediately runs against it. The test suite doesn't need to know about the fork in advance. This means: - A researcher adds an experimental fork class → existing capability tests validate it automatically - A new capability Protocol is defined → tests written against it run against current and future forks that implement it - Test vectors generated via `uv run fill --fork=epbs_experiment` include exactly the tests that apply to that fork, with no manual curation --- ## Migration Path This refactoring is designed to be incremental — each phase adds value without requiring a rewrite. ```mermaid gantt title Migration Timeline dateFormat YYYY-MM axisFormat %b %Y section Phase 1 Create core/protocol.py + capabilities.py :p1a, 2025-07, 2w Wrap current code in Devnet0 fork class :p1b, after p1a, 1w Verify all existing tests pass unchanged :p1c, after p1b, 1w section Phase 2 Create Devnet1 with real overrides :p2a, after p1c, 2w Wire into test ForkRegistry :p2b, after p2a, 1w section Phase 3 Extract PQCapable, NetworkCapable Protocols :p3a, after p2b, 2w Add conformance tests across all forks :p3b, after p3a, 1w section Phase 4 Add SpecRunner for multi-fork processing :p4a, after p3b, 2w ``` **Phase 1 (immediate):** Create `core/protocol.py` with the `ForkProtocol` ABC. Create `forks/devnet0/` as a thin wrapper around the current code. All existing tests continue to pass without modification. **Phase 2 (at next devnet):** When devnet1 needs new behavior, create `forks/devnet1/` inheriting from devnet0. Override only what changes. Wire into the test framework's `ForkRegistry`. **Phase 3 (as sub-specs mature):** Extract `PQCapable`, `NetworkCapable`, `ForkChoiceCapable` Protocol interfaces from the sub-specs. Add conformance tests that run against all registered forks. **Phase 4 (optional):** Add the `SpecRunner` era combinator for multi-fork processing in integration tests or node implementations. --- ## What Changes vs. What Stays the Same ### Unchanged (zero refactoring) - **`types/`** — `Container`, `Uint64`, `Bytes32`, all SSZ primitives. Fork-agnostic. - **`subspecs/ssz/`** — Merkleization, hashing. Fork-agnostic. - **`subspecs/poseidon2/`**, **`subspecs/koalabear/`** — Cryptographic primitives. Fork-agnostic. - **`subspecs/xmss/`** — XMSS implementation stays as-is. Forks reference it via `PQCapable`. - **`subspecs/networking/`** — gossipsub, reqresp, discovery. Forks reference via `NetworkCapable`. - **`snappy/`** — Compression. Fork-agnostic. - **Test infrastructure** — `BaseFork`, `ForkRegistry`, `StateTransitionTestFiller`. ### New additions | What | Effort | |------|--------| | `core/protocol.py` — `ForkProtocol` ABC | Small — one new file | | `core/capabilities.py` — sub-spec Protocols | Small — one new file | | `core/runner.py` — `SpecRunner` | Small — one new file | | `forks/devnet0/spec.py` — thin wrapper | Small — delegates to existing `State` methods | | `forks/devnet1/` (and beyond) | Incremental — only override what changes | --- ## Risks and Mitigations **Python's `Protocol` + `ABC` interaction.** Using both on the same interface requires care. Mitigation: use `ABC` for `ForkProtocol` (which has default implementations) and `Protocol` for capabilities (which are structural contracts). **Associated types aren't compiler-enforced.** `StateType = Devnet0State` is a class variable, not a type parameter in the Rust/Haskell sense. Mitigation: conformance tests verify the types match at CI time. **Coarse override granularity.** If `State.process_attestations()` is a 100-line method and devnet2 needs to change one threshold, you override the entire method. Mitigation: see [Appendix A](#appendix-a-migrating-processing-logic-off-state) for a phased approach to decomposing large methods. --- ## Appendix A: Migrating Processing Logic Off `State` ### What leanSpec currently places processing functions (`process_block`, `process_attestations`, `state_transition`, `build_block`) as **methods on the `State` container**. This proposal suggests eventually migrating these to standalone functions on the **fork class** instead, with `State` becoming a pure data container (Pydantic model with fields, validation, and serialization — but no processing logic). ### Why In the context of a multi-fork architecture, behavior on `State` creates several friction points: **Two parallel inheritance chains.** With behavior on `State`, you must keep two class hierarchies in sync: `Devnet0 → Devnet1` for fork classes and `Devnet0State → Devnet1State` for state classes. This mirrors the exact problem the consensus-specs have with maintaining `PREVIOUS_FORK_OF` in multiple places. ```mermaid graph LR subgraph "Behavior on State — two chains" D0F[Devnet0 fork class] --> D1F[Devnet1 fork class] D0S[Devnet0State data + behavior] --> D1S[Devnet1State data + behavior] end subgraph "Standalone functions — one chain" D0[Devnet0 fork class + behavior] --> D1[Devnet1 fork class + behavior] S0[Devnet0State pure data] --> S1[Devnet1State pure data] end D0F -.->|"delegates to"| D0S D1F -.->|"delegates to"| D1S style D0S fill:#e17055,color:#fff style D1S fill:#e17055,color:#fff ``` **Coarse override granularity.** When processing logic lives on `State`, overriding a small piece of behavior (e.g., the justification threshold) requires overriding the entire method and copying unchanged logic. With standalone methods on the fork class, you can factor them into small composable pieces: ```python class Devnet0(ForkProtocol): def process_attestations(self, state, attestations): state = self._check_attestation_signatures(state, attestations) state = self._update_justification(state) state = self._apply_rewards(state) return state class Devnet1(Devnet0): # Override ONLY the signature check — everything else inherited def _check_attestation_signatures(self, state, attestations): # PQ signature verification instead of BLS ... ``` **Sub-spec Protocols fit awkwardly on State.** `PQCapable` is a behavioral property of the fork's rules, not of the state data. Placing it on `State` creates a confusing split where some capabilities live on the fork class and others on the state class. **Ecosystem alignment.** The consensus-specs' `process_block(state, block)` signature is deeply ingrained across client teams, auditors, and EIP authors. Standalone functions match this shared vocabulary. The `σ' = f(σ, B)` pattern also maps directly onto formal verification proof styles. ### How — Phased Migration **Phase 1 (now):** Keep behavior on `State`. The `Devnet0` fork class delegates to `state.process_block(block)`. Zero disruption. **Phase 2 (at devnet1):** When real multi-fork divergence begins and specific processing steps need overriding, extract those specific methods from `State` onto the fork class. Leave `State` as the data container for the methods that move. **Phase 3 (as the spec stabilizes):** Complete the migration — all processing logic on fork classes, state classes as pure Pydantic data models. ```python # Phase 1 — today class Devnet0(ForkProtocol): def process_block(self, state, block): return state.process_block(block) # delegates to State # Phase 3 — eventual target class Devnet0(ForkProtocol): def process_block(self, state, block): state = self._process_header(state, block) state = self._process_attestations(state, block.body.attestations) state = self._process_deposits(state, block.body.deposits) return state ``` This avoids a premature rewrite while ensuring the architecture supports fine-grained overrides once they're actually needed. The key principle: **don't extract methods until you need to override them**.