# State of the spec (from Danny's perspective) _last updated 2018/12/12_ _Warning_: These are Danny Ryan's personal notes and thoughts on the spec and do not necessarily reflect the thoughts and opinions of the other members of the EF Research team (although I expect we have some overlap :) . ## Phase 0 In the words of Justin Drake, "It's a hard life for the people implementing sharding". We have continued to implement heavy revisions of the phase 0 beacon chain spec since devcon in early November. My intention is to have the phase 0 spec near its final form before the end of this year. To this goal, we have attempted to tackle all of the known big changes sooner rather than later. _[Note: By "near final form", I mean data structures stable and in place, features stable and in place, and general flow stable and in place. I expect bug fixes, minor changes, etc to continue through testnets.]_ I imagine it has been difficult to keep up with the changes as a spec reader and even worse as an implementer. That said, I hope that if you have been following the changes that you agree that things are really starting to shape up. I am confident that we have made significant forward progress and that much of the drastic changes are out of the way. ### Stability, general comments, and "as an implementer" recommendations These are my (Danny's) personal impressions of stability and comments on the phase 0 spec along with some "as an implementer" recommendations. * Constants * Names and uses -- Stable * Values -- Unstable * Expect some additions but I don't expect any major reworkings here * Data structures * This has been one of the more painful points for implementers in the past month. Almost every data structure has received major refactorings both in structure as well as naming. * Names -- Pretty stable but I'd expect some continual nitpicks * Content and inter-relationships -- Relatively stable * **Beacon chain operations** * The removal of "specials" and unifying with "attestations" as block operations is here to stay. * The concept of a block having a body of "operations" or "transactions" of various types is here to stay. * The five operation types currently in the spec are here to stay, and the content of each type is relatively stable. * _As an implementer_, I would first start with a fixed validator set and only process attestations per block. This would warrant building out `Deposit` for constructing the initial state, and then only processing `Attestation`s in the block bodies. Save slashing, ongoing deposits, and exits for later. You could even build the initial validator set without `Deposit` for that matter and only build `Attestation` operation type. * **Beacon chain blocks** * Much of the rework here had to do with moving operations into the `BeaconBlockBody`. Although there is a chance some of this header structure will change, the block vs body relation is here to stay. I expect general stability here after we merge in [#269](https://github.com/ethereum/eth2.0-specs/pull/269) which simplifies block header with `parent_root` rather than `ancestor_hashes` * _As an implementer_, I would build out this new block structure and begin using it immediately. Even preempt the merging of #269 moving to just having a `parent_root` in the block instead of `ancestor_hashes` * **Beacon chain state** * The state data structures underwent heavy naming revisions but only moderate structure/feature revisions. One of the biggest changes had to do with moving to per-slot state transitions but this barely affected the State data structures. * For `BeaconState`, the content that is there I would deem general stable. I would expect some additions (such as [block accumulator](#https://github.com/ethereum/eth2.0-specs/pull/269))and maybe some nitpick naming changes. * `ValidatorRecord` is stable * `CrosslinkRecord` is stable * `ShardCommittee` is stable * `CandidatePoWReceiptRootRecord` is stable but might get renamed * `PendingAttestationRecord` is stable but might get renamed * `ForkData` is stable * Ethereum 1.0 deposit contract * I think we are generally approaching feature complete on this contract, but we have yet to really test it. * _As an implementer_, I would not target processing receipts from PoW chain in the next month or so. You can easily simulate incoming `Deposit` records in block bodies without tieing sync to a pow chain for the time being. * We need to pull this contract out into a separate repo and start building out testing. Until we do that, I would hold off on processing directly from pow receipts. * State transition * The state transition function underwent three major substantive revisions along with a ton of minor revisions in naming and general clarity. This sum totalled to what feels like a full rewrite to the reader. The three major substantive revisions were: * _Revert to epoched Casper from epoch-less Casper_. This is here to stay. The epoch-less Casper added significant complexity with only marginal benefits in time to finality. Epoched Casper also plays nicely with LMD GHOST which appears it will be the fork choice. * _Per-slot state transition_. This is here to stay. The per-slot state transition greatly simplified some of the messier parts of the spec such as the `while` loop that would ensure enough epoch transitions occurred if parent to child block skipped a ton of slots. * _Remove specials in favor of block operations_. This is here to stay as mentioned above. This removed the ad-hoc typing of specials and allows us to treat all beacon operations/transactions in a similar manner. * **Fork choice** * We switched to the much simpler LMD GHOST from IMD. A "stickiness" condition was added to prevent degenerate scenarios in which a validator of small weight could cause the chain to "flip-flop" due to newly justified blocks. This modified LMD GHOST is very likely to be the fork choice rule. As of yet, we have no other candidates with advantages over LMD. * _As an implementer_, I would initially target a very simple fork choice such as "highest slot" or "longest chain" (both would be fine as stubs). Put these behind a clean interface and add in the LMD fork choice when you have more components of the protocol built and the spec has had more time to stabilize. * **on_startup** * There's nothing crazy going on in `on_startup`. I consider it generally stable. * _As an implementer_, I would get this together immediately as it (or something like it) is needed to seed any test chain you run. Because `on_startup` uses `process_deposit` to bring in all initial validators, I would also probably build this out. That said, to start you could just loop through and super simply create `N` validators without thinking about the minutia of `process_deposit`. * **updating validator status** * This is all getting more stable, but I suspect some bugs and minor reworks soon. * _As an implementer_, I would just avoid any changes to validator status at the start. Fixed validator set FTW. ### Phase 0 state transition in a nutshell Phase 0 processing of beacon chain blocks largely looks like the following pseudocode. What's really nice about this is that the phase 0 table of contents is starting to logically reflect the following code structure. ```python # Throws or returns False if any of the # verifications or processings fail def slot_state_transition(state, block): state = deep_copy(state) # per-slot processing state.slot += 1 update_proposer_randao_skips(state) update_recent_block_hashes(state) # per-block processing if block: # handle block header verify_signature(state, block) verify_and_update_randao(state, block) tally_pow_receipt_root_vote(state, block) # handle block body operations process_proposer_slashings(state, block) process_casper_slashings(state, block) process_attestations(state, block) process_deposits(state, block) process_exits(state, block) # handle any forced ejections process_low_balance_ejections(state) # per-epoch processing if state.slot % EPOCH_LENGTH == 0: if state.slot % POW_RECEIPT_ROOT_VOTING_PERIOD == 0: process_candidate_receipt_roots(state) update_justification(state) update_finalization(state) update_crosslinks(state) process_casper_reward_penalties(state) process_crosslink_reward_penalties(state) update_validator_registry(state) update_shard_shufflings(state) final_book_keeping(state) return state def process_block(block): if not block_pre_processing_conditions(block): return False # process skipped slots while (state.slot < block.slot - 1): state = slot_state_transition(state, block=None) # process slot with block state = slot_state_transition(state, block) # check state root if block.state_root == hash(state): return state else: return False # or throw or whatever ``` _As an implementer_, I would initially stub out a lot of the modules from `slot_state_transition` and make it look more like the code below. This is both in an effort to get the simplest version of the protocol running and also in an effort to not address the potentially more unstable portions of the spec. What is left is essentially a static validator set that creates and attests to beacon chain blocks. (I left randao in there because it's stable, but you could even pull that out to start). ```python def slot_state_transition(state, block): state = deep_copy(state) # per-slot processing state.slot += 1 update_proposer_randao_skips(state) update_recent_block_hashes(state) # per-block processing if block: # handle block header verify_signature(state, block) verify_and_update_randao(state, block) # handle block body operations process_attestations(state, block) # per-epoch processing if state.slot % EPOCH_LENGTH == 0: update_justification(state) update_finalization(state) update_crosslinks(state) final_book_keeping(state) return state ``` ## Phase 1 Although we have been working to ensure that the phase 0 beacon chain data structures support the features that will be in phase 1, the minutia of technical details of phase 1 are still largely in flux. I don't currently recommend building anything in the phase 1 doc unless you are just curious or bored. I expect much of phase 1 doc to be under heavy revision through at least February. ## Phase 2 Nothing to build here (unless you are the eWASM team doing state prototypes!). Right now if you want to contribute to phase 2, you'll have to go more toward the research route. To start, I recommend taking a look at two recent ethresearch posts by Vitalik -- both of which he is actively seeking feedback on. * [A minimal state execution proposal](https://ethresear.ch/t/a-minimal-state-execution-proposal/4445/3) * [A layer 2 computing model using optimistic state roots](https://ethresear.ch/t/a-layer-2-computing-model-using-optimistic-state-roots/4481) ## SSZ and tree hashing For consensus serialization/hashing, I consider this stable. There are minor things to debate such as big vs little endian, and field sort order. Other than that, I don't expect major reworks. The tree hashing algorithm might undergo some minor revisions or clarifications regarding padding and other minutia. _As an implementer_, you can always use a simple `hash` on data structures for the time being if you don’t have the tree hashing algorithm implemented. The tree hashing alg is to be able to cache portions of the state that are unchanged since last hashing and to allow for serving sub-sections of the state proved against the root. ## BLS Verify This just underwent some minor refactors for clarity but not substantive changes. I am confident on naming and the general interface to these functions _pending academic review_. _As an implementer_, I would go ahead with utilizing the spec as is. ## libp2p Confident/stable. _As an implementer_, I would either begin building out the components contained in [this list](https://github.com/ethresearch/p2p/issues/4#issuecomment-436702674), or I would begin interfacing with the libp2p-daemon. ## Discovery protocol Leaning toward [discv5](https://github.com/fjl/p2p-drafts/blob/master/discv5-eip.md). From my understanding there is still work to do here to get the spec up to snuff. We are pushing on this to ensure that it is ready and meets our needs. _As an implementer_, if I am to the point of running a network, I would probably use a "bootstrap list" to start. From there if there is not yet clarity on discv5, I might begin using one of the discovery protocols found [here](https://github.com/ethresearch/p2p/issues/7) and just run everything on the beacon chain subnet.