Loading ...
Settings
1
1
2
1
2
1
1
1
2
2

New sharding design with tight beacon and shard block integration

Previous data shard construction: Add n=64 data shards to the beacon chain. Each slot, n proposers independently propose their shard blobs, which are then confirmed by committees. Once a shard blob is confirmed (this can take several slots), it can be referenced by the execution chain.

Here is an alternative proposal: Add a new type of transaction that can contain additional sharded data as calldata. The block is then constructed by a single block builder (which can be different from the proposer using proposer builder separation), and includes normal transactions as well as transactions with sharded calldata. This is efficient because it means that tight integrations between rollups and L1 become possible, and it is expected that this “super-block-builder” strategy will emerge in practice anyway in order to maximize MEV extraction.

Advantages:

Disadvantages

Suggested Implementation

MAX_SHARDS = 2**8 # 256
SHARD_SIZE_FIELD_ELEMENTS = 2**12 # 2**12*31 = 126976 bytes
MAX_BLOCK_SIZE = SHARD_SIZE_FIELD_ELEMENTS * MAX_SHARDS # 2**20 field elements / 32,505,856 bytes
TARGET_BLOCK_SIZE = MAX_BLOCK_SIZE // 2 # EIP1559 for data gas

class ShardedBeaconBlockCommitment(Container):
    sharded_commitments: List[KZGCommitment, 2 * MAX_SHARDS]
    beacon_block_commitments: uint64      # Number of commitments occupied by Beacon block + Execution Payload

    # Aggregate degree proof for all sharded_commitments
    degree_proof: KZGCommitment

class ExecutionPayload(Container):
    # Execution block header fields
    parent_hash: Hash32
    fee_recipient: ExecutionAddress  # 'beneficiary' in the yellow paper
    state_root: Bytes32
    receipt_root: Bytes32  # 'receipts root' in the yellow paper
    logs_bloom: ByteVector[BYTES_PER_LOGS_BLOOM]
    random: Bytes32  # 'difficulty' in the yellow paper
    block_number: uint64  # 'number' in the yellow paper
    gas_limit: uint64
    gas_used: uint64
    timestamp: uint64
    extra_data: ByteList[MAX_EXTRA_DATA_BYTES]
    base_fee_per_gas: uint256

    # Extra payload fields
    block_hash: Hash32  # Hash of execution block
    transactions: List[Transaction, MAX_TRANSACTIONS_PER_PAYLOAD]
    
    # NEW fields for data gas market
    data_gas_limit: uint64
    data_gas_used: uint64
    data_base_fee_per_gas: uint256

@dataclass
class TransactionKZGCalldataPayload:
	chain_id: int = 0
	signer_nonce: int = 0
	max_priority_fee_per_gas: int = 0
	max_fee_per_gas: int = 0

    # New data gas
    max_priority_fee_per_data_gas: int = 0
	max_fee_per_data_gas: int = 0

    gas_limit: int = 0
	destination: int = 0
	amount: int = 0
	payload: bytes = bytes()
    
    # KZG Calldata
    kzg_calldata_commitments: List[KZGCommitment, MAX_SHARDS]
    
	access_list: List[Tuple[int, List[int]]] = field(default_factory=list)
	signature_y_parity: bool = False
	signature_r: int = 0
	signature_s: int = 0

Sampling

Block Builder Costs

Block builders get two new tasks. One is computing the KZG witnesses for the samples, and the other one is seeding the samples in the P2P network.

Sharded mempool

Rationale

The current mempool that most transactions go through on Ethereum does not scale to data transactions. Every node processes the full mempool, but with sharding the capacity of the chain increases to 1.3 MB/s, which is also the lower bound for the expected mempool bandwidth if all transactions go through it.

Clearly it makes no sense to shard data transactions on chain while requiring nodes to process the full mempool. We instead need to do data availability sampling on the mempool as well when data transactions are concerned.

Is a mempool still required?

An alternative to sending transactions to a public mempool is to send them to a block builder, which is effectively already happening now with Flashbots. The advantage is that block builders can promise not to front run transactions. If they do, then users can simply switch to another block builder.

Solana, as an example, already implements this design and by default only sends transactions to the next proposer.

It is therefore quite likely that the mempool will be much smaller in size and not even be required for the chain at all. However, we want to keep a mempool in the design for two reasons:

  1. The new censorship resistance design by Francesco requires a public mempool
    This does not contradict the earlier point that most transactions will be sent directly to a builder first. Basically users will mostly send their transactions to builders, and only if they get censored they will escalate to the mempool; the mempool becomes a “censored pool”
  2. An Ethereum user with a full node should have a way of sending transactions without configuring a central provider.
    I find this argument less convincing as the future will largely consist of rollups on L1, and end users using L2s. So end users should very rarely have to use the L1.

Construction

All transactions are broadcastet via a libp2p pubsub channel. For transaction type 3, the kzg_calldata_commitments are included, but not the actual underlying data.

We construct a second set of 512 pubsub channels that are only for propagating samples of shard data transactions. When someone wants to send a type 3 transaction to the mempool, they send the 512 samples of each kzg_calldata_commitment to these sample channels.

Last changed by  
 
dankrad
47763

Read more

Let's reduce the churn limit to give us more time to deal with the economic and technical consequences of liquid staking

The Ethereum validator set is expanding at a rapid pace. Ever since withdrawals were introduced in April, the validator activation queue has been constantly full. We expected growth in the validator set when withdrawals were implemented, as this reduced staking risks. However, it's fair to say that we may not have fully understood the broad implications of liquid staking at that time. As of now, Lido dominates the liquid staking space, holding nearly one-third of all staked Ether. They've shown no willingness to self-limit, posing a risk to Ethereum's decentralization. A governance attack on Lido could create significant problems for the Ethereum network. Economically, liquid staking has obvious benefits, provided users trust their staking provider. A solo staker is sensitive to staking returns for several reasons: The financial burden of maintaining staking operations, both initial and ongoing. The opportunity cost associated with tying up capital.

Apr 12, 2024
RSA Bounties

We want to encourage research in bounties related to RSA, as some Ethereum-related projects (VDF, DARK using RSA) make use of relatively recent assumptions that we believe are as difficult as the RSA problem and IFP, but of course this is unproven Assumptions to be checked RSA Adaptive root problem: Computing random prime roots of chosen elements in RSA groups with unknown factorization is hard [Necessary for Wesolowski]; implies Low order assumption RSA Low order problem: Finding an element with low order in an RSA group with unknown factorization is hard [Necessary for Pietrzak] Strong RSA assumption [RSA Accumulators] Suggested bounties It is very unlikely that the assumptions will be outright broken. Also, a lot of research has been done on IFP (Integer Factorization Problem), so even if any really cool ideas came about on say, the adaptive root problem, it might not initially compete with factorization on pure computational terms.

Jun 19, 2023
Data availability encoding

Reverse Bit Order Reverse bit order is the name for the order of a sequence of $n$ integers, that comes from reversing the bit sequence of each individual integer and then ordering by the resulting integer. It is an idempotent operation. For example, applying this to the integers $0,1,2,3,4,5,6,7$, results in the reverse bit order sequence $0,4,2,6,1,5,3,7$. Now say we have a root of unity $\omega^8=0$. Then the sequence $\omega^0,\omega^4,\omega^2,\omega^6,\omega^1,\omega^5,\omega^3,\omega^7$ is a permutation of the roots of unity of order 8 with the following nice properties: Partitioning it into the first four and the second four elements, we get a subgroup of order 4 ($\omega^0,\omega^4,\omega^2$) and a coset of order 4 ($\omega^6,\omega^1,\omega^5,\omega^3,\omega^7$) Partitioning it into four sets of size 2, we get cosets of the order 2 subgroup: ($\omega^0,\omega^4$) (which is also the subgroup), ($\omega^2,\omega^6$), ($\omega^1,\omega^5$) and ($\omega^3,\omega^7$) This property of the reverse bit order extends to higher powers: If you partition any list of length $n=2^k$ into powers of two, you always get cosets in each partition. This is useful for Fourier transforms.

Feb 23, 2023
Sample reconstruction estimate

Assume $n \cdot n$ samples ($n$ per row and column), $v$ validators, and each validator being able to repopulate $s$ samples (because their rows and columns intersect at $s$ points). What we want: Very high probability that at least 50% of the samples on each row and column can be repopulated from orthogonal rows/columns. Probability computation For each given sample, the probability that one of the validators can make it available as one of their reconstructed samples is $$ p = 1 - (1 - q) ^ {s v} $$ where $q = \frac{1}{n^2}$.

May 19, 2022
Read more from dankrad
Published on HackMD