### 🦾 STEEL Workshop (starting 14:15) 🦾
<div style="text-alight: center">
<a href=https://github.com/ethereum/execution-specs>📜 EELS - ethereum/execution-specs</a>
<br/>
<a href="https://github.com/ethereum/execution-spec-tests">🧪 EEST - ethereum/execution-spec-tests</a>
<br/>
</div>
<style>
.reveal .slides > section > section {
text-align:left;
}
p {
text-align: left;
}
.top-left-label {
position: fixed;
top: -70px;
left: 30px;
font-size: 30px;
font-weight: bold;
color: #555;
}
.team-grid {
display: grid;
grid-template-columns: repeat(3, 1fr);
gap: 5px;
justify-items: center;
margin-top: 20px;
}
.team-member {
text-align: center;
font-size: 12px;
}
.team-member img {
border-radius: 50%;
width: 100px;
height: 100px;
}
</style>
---
### STEEL Who?
- Specifications and Testing for the Ethereum Execution Layer
---
<div class="top-left-label">📜 EELS (Specs) Team </div>
<div class="team-grid">
<div class="team-member">
<img src="https://github.com/SamWilsn.png" alt="Sam's profile picture" width="100">
<p><b>Sam: <code>@SamWilsn</code></b></p>
</div>
<div class="team-member">
<img src="https://github.com/petertdavies.png" alt="Peter's profile picture" width="100">
<p><b>Peter: <code>@petertdavies</code></b></p>
</div>
<div class="team-member">
<img src="https://github.com/gurukamath.png" alt="Guru's profile picture" width="100">
<p><b>Guru: <code>@gurukamath</code></b></p>
</div>
</div>
---
<div class="top-left-label">🧪 EEST (Testing) Team</div>
<div class="team-grid">
<div class="team-member">
<img src="https://github.com/marioevz.png" alt="Mario's profile picture" width="100">
<p><b>Mario: <code>@marioevz</code></b></p>
</div>
<div class="team-member">
<img src="https://github.com/danceratopz.png" alt="Dan's profile picture" width="100">
<p><b>Dan: <code>@danceratopz</code></b></p>
</div>
<div class="team-member">
<img src="https://github.com/winsvega.png" alt="Dimitry's profile picture" width="100">
<p><b>Dimitry: <code>@winsvega</code></b></p>
</div>
<div class="team-member">
<img src="https://github.com/spencer-tb.png" alt="Spencer's profile picture" width="100">
<p><b>spencer: <code>@spencer-tb</code></b></p>
</div>
</div>
---
### Testing Team / EEST Aims
<div class="top-left-label">🧪 EEST</div>
- Provide a source of truth for implementations in the form of consensus spec tests.
- Mainnet and development forks:
- Prague, EOF & Verkle etc
- Improve the EL dev. cycle for all clients - unlock tdd.
- Create and maintain external testing tooling.
---
### Testing Landscape Last Year
<div class="top-left-label">🧪 EEST</div>
- Internal client CIs:
- EEST releases within your CI for development.
- `ethereum/tests` within your CI for stable/mainnet.
---
### Testing Landscape Last Year
<div class="top-left-label">🧪 EEST</div>
- Hive simulators, viewing with hiveview:
- consensus - load clients up with block rlps.
- pyspec - send blocks as engine new payloads.
- engine - sync, invalid payload, re-orgs, block building.
- Multi-client, using a CL mocker to send engine calls.
---
### Our Latest Framework Updates
<div class="top-left-label">🧪 EEST</div>
- `fill` - generates fixtures from python spec tests.
- 🔜 `gentest` - generates a python test from a transaction hash.
- 🆕`consume` - runs generated fixtures against clients.
- 🆕`execute` - sends existing python spec test transactions to devnets/live networks, fuzzable?
---
<div style="text-alight: center">
<b>☢️ FILL-ING != PASSING TESTS ☢️</b>
</div>
---
<div class="top-left-label">🧪 EEST - <code>consume direct</code> </div>
<div style="text-alight: center">
<img src="https://storage.googleapis.com/ethereum-hackmd/upload_e96b0e9bcbc8f9fb278c618f08ab0bbd.png" width="700"style="display: block; margin: 0 auto;">
</div>
---
<div class="top-left-label">🧪 EEST - <code>consume rlp</code> </div>
<div style="text-alight: center">
<img src="https://storage.googleapis.com/ethereum-hackmd/upload_4252307e2012cb026b572bce154a88a2.png" width="750"style="display: block; margin: 0 auto;">
</div>
---
<div class="top-left-label">🧪 EEST - <code>consume engine</code> </div>
<div style="text-alight: center">
<img src="https://storage.googleapis.com/ethereum-hackmd/upload_b8c123a59c4fe47e90e6ab86062b1b46.png" width="750"style="display: block; margin: 0 auto;">
</div>
---
<div>
<img src="https://storage.googleapis.com/ethereum-hackmd/upload_d73b04789c8d945aaac93055d486cac9.jpg" width="400" alt="">
</div>
---
<div style="text-alight: center">
<b>☢️ NO MORE HIVE PYSPEC ☢️</b>
<br>
<img src="https://storage.googleapis.com/ethereum-hackmd/upload_ac4e6591528761885e68d10e2b55bdcb.png" width="750"style="display: block; margin: 0 auto;">
</div>
---
<div style="text-alight: center">
<b>✅ CONSUME ENGINE FROM NOW ON ✅</b>
<br>
<img src="https://storage.googleapis.com/ethereum-hackmd/upload_f80a18010692017e2ad61efbe356b18d.png" width="750"style="display: block; margin: 0 auto;">
</div>
---
### Devnet Entry Requirements?
<div class="top-left-label">🧪 EEST</div>
- Major/minor particapation based on passing tests?
- Should 3/6 ELs be passing all of `consume engine` before starting a devnet.
- Remaining clients must do the same before joining.
- Less devnet bugs? Faster devnet cycles?
---
### Fast Consume Direct Engine
<div class="top-left-label">🧪 EEST</div>
- If we assume `consume engine` is required to pass for devnet entry, how can we make it run faster?
- Direct EL client interface to for blockchain engine tests?
- Single shot test fixtures? All blocks within a single test?
---
### Fast Consume Direct Engine
<div class="top-left-label">🧪 EEST</div>
<img src="https://storage.googleapis.com/ethereum-hackmd/upload_cf3822d8f5e73a8228c88da5af56b7a8.png" width="350" style="display: block; margin: 0 auto;">
---
### Current Testing Landscape
<div class="top-left-label">🧪 EEST</div>
- Internal client CIs (`consume direct`):
- EEST releases within your CI for development.
- `ethereum/tests` within your CI for stable/mainnet.
---
### Current Testing Landscape
<div class="top-left-label">🧪 EEST</div>
- https://hiveprague.ethdevops.io/
- consume rlp/engine using EEST.
- https://hivetests.ethdevops.io/
- consensus simulator using `ethereum/tests`.
---
### 📍 Min Testing Aim for Prague
<div class="top-left-label">🧪 EEST</div>
- https://hiveprague.ethdevops.io/
- consume direct/rlp/engine using EEST.
- execute within hive using EEST txs.
- hive engine simulator (only Prague).
- https://hivetests.ethdevops.io/
- consume rlp `ethereum/tests`.
- hive engine simulator (up to Cancun).
---
### What's happening with `ethereum/tests`?
<div class="top-left-label">🧪 EEST</div>
- Plan is to port all of `ethereum/tests` -> EEST for EOF.
- 2 types of releases for clients in EEST:
- stable fixtures (mainnet fork)
- develop fixtures
---
### yOu LeFt `HiVe` BeHiNd :(
<div class="top-left-label">🧪 EEST</div>
<div>
<img src="https://storage.googleapis.com/ethereum-hackmd/upload_6bc051f09df3581979aa7cf36f0dc3e5.png" width="600" alt="">
</div>
---
### Hive Improvements
- Documentation website!
- Hiveview tweaks:
- Commit history, visually inspect what commit added fails.
- Stop running it on a continous loop...
- Benchmarks? Metric for test execution time across all clients?
- Hivegrid:
- Gridserver for running tests on hive extremely fast!
---
### EELS Team Aims
<div class="top-left-label">📜 EELS</div>
- EELS should be the authoritative specification for the EL
- Every client should be required to do exactly the same thing as EELS
- EELS should be the primary test filler
- The point of testing is to make sure clients do the same thing as EELS
- Client development can be test-driven, STEEL can ship tests before client devs write code
---
### EELS EIP process proposals
<div class="top-left-label">📜 EELS</div>
- In order to move past Draft, EL EIPS must:
- Have a version
- Have a linked EELS implementation (specific commit)
- The version in EELS must match the version in the EL
- Writing tests will be expected for serious proposals, but not a procedual requirement
---
### EELS: Where are we at?
<div style="text-alight: center">
<img src="https://storage.googleapis.com/ethereum-hackmd/upload_9b95d2e4b5fc7d9e21d15cf765fb1a8d.png" width="500"style="display: block; margin: 0 auto;">
</div>
---
### EELS: Where are we at?
- EELS can fill Prague (subject to 1 PR)
- Devnet tests will be filled by EELS
- EIP versioning process hasn't been implemented
- Upcoming features like EOF and Verkle are still in development
---
### 🦾 STEEL Workshop 🦾
<div style="text-alight: center">
<a href=https://github.com/ethereum/execution-specs>📜 EELS - ethereum/execution-specs</a>
<br/>
<a href="https://github.com/ethereum/execution-spec-tests">🧪 EEST - ethereum/execution-spec-tests</a>
<br/>
</div>
{"title":"EEST & EELS Devcon SEA Workshop","tags":"Talk, Ethereum, Testing, Spec","slideOptions":{"theme":"white","transition":"slide"}}