### Execution Spec Tests x Immunefi
<div style="text-alight: center">
<a href="https://github.com/ethereum/execution-spec-tests">🧪 EEST - ethereum/execution-spec-tests</a>
<br/>
</div>
<style>
.reveal .slides > section > section {
text-align:left;
}
p {
text-align: left;
}
.top-left-label {
position: fixed;
top: -70px;
left: 30px;
font-size: 30px;
font-weight: bold;
color: #555;
}
.team-grid {
display: grid;
grid-template-columns: repeat(2, 1fr);
gap: 5px;
justify-items: center;
margin-top: 20px;
}
.team-member {
text-align: center;
font-size: 12px;
}
.team-member img {
border-radius: 50%;
width: 100px;
height: 100px;
}
</style>
---
### Disclaimer
- Do not report bugs in our repo with Immunefi!
- Use our tools to find or consolidate bugs found within Execution Layer clients.
- Everything discussed is for testing the EL.
---
### Testing Team Who?
<div class="team-grid">
<div class="team-member">
<img src="https://github.com/marioevz.png" alt="Mario's profile picture" width="100">
<p><b>Mario: <code>@marioevz</code></b></p>
</div>
<div class="team-member">
<img src="https://github.com/danceratopz.png" alt="Dan's profile picture" width="100">
<p><b>Dan: <code>@danceratopz</code></b></p>
</div>
<div class="team-member">
<img src="https://github.com/winsvega.png" alt="Dimitry's profile picture" width="100">
<p><b>Dimitry: <code>@winsvega</code></b></p>
</div>
<div class="team-member">
<img src="https://github.com/spencer-tb.png" alt="Spencer's profile picture" width="100">
<p><b>spencer: <code>@spencer-tb</code></b></p>
</div>
</div>
---
### What is EEST?
- Custom python framework used to:
- Write EL layer consensus tests in a readable python format
- Generate test fixtures that can be consumed by EL clients
- Run the generated test fixtures against EL clients
---
### EEST Framework Aims
- Provide a source of truth for implementations in the form of consensus spec tests.
- Mainnet, Prague, EOF, Verkle...
- Improve the EL client development cycle.
- Create and maintain external testing tooling.
- Replace `ethereum/tests` by EOY 2025.
---
### EEST x Immunefi Aims
- Write tests that can validate a bug you find.
- Can be added to a bug report.
- Write complex tests that can catch consensus bugs.
- Test multiple clients for the same bug.
---
### Framework Overview
#### Covered Today
- `fill` - generates fixtures from python spec tests.
- `consume` - runs generated fixtures against clients.
---
### EEST Framework Overview
#### Not Covered Today
- 🆕 `execute` - sends existing python spec test transactions to devnets/live networks.
- 🔜 `gentest` - generates a python test from a transaction hash.
---
<div class="top-left-label"> <code>fill</code> </div>
<div style="text-alight: center">
<img src="https://storage.googleapis.com/ethereum-hackmd/upload_bbabd570f204bd1a08d8b3bf55d193ba.png" width="500"style="display: block; margin: 0 auto;">
</div>
---
<div class="top-left-label"> <code>consume direct</code> </div>
<div style="text-alight: center">
<img src="https://storage.googleapis.com/ethereum-hackmd/upload_e96b0e9bcbc8f9fb278c618f08ab0bbd.png" width="700"style="display: block; margin: 0 auto;">
</div>
---
<div class="top-left-label"> <code>consume rlp</code> </div>
<div style="text-alight: center">
<img src="https://storage.googleapis.com/ethereum-hackmd/upload_4252307e2012cb026b572bce154a88a2.png" width="750"style="display: block; margin: 0 auto;">
</div>
---
### What is hive?
- `ethereum/hive` - an end-to-end test harness:
- Test simulators are spun up in a docker container.
- Fully instantiated clients are created in docker container.
- Hive controls the interaction between the simulators/clients.
---
### What is hive?
<div style="text-alight: center">
<img src="https://storage.googleapis.com/ethereum-hackmd/upload_76572fdd07fea14e1703226393063229.png" width="800"style="display: block; margin: 0 auto;">
</div>
---
<div class="top-left-label"> <code>consume engine</code> </div>
<div style="text-alight: center">
<img src="https://storage.googleapis.com/ethereum-hackmd/upload_b8c123a59c4fe47e90e6ab86062b1b46.png" width="750"style="display: block; margin: 0 auto;">
</div>
---
### Additional Resources
- Documentation
- ethereum.github.io/execution-spec-tests/main
- [`ethereum/hive`](https://github.com/ethereum/hive/blob/master/docs/overview.md)
---
### Additional Resources
- Devcon SEA
- [Protec and Attac: Programmatic Execution Layer Consensus Tests](https://www.youtube.com/watch?v=4OF-TJ9nB4I)
- [Hunt the Bug, Save the Chain: Uncovering Bugs in EIP Implementations](https://www.youtube.com/watch?v=K0pQ7bRuJOk)
---
### Execution Spec Tests | Immunefi
<div style="text-alight: center">
<a href="https://github.com/ethereum/execution-spec-tests">🧪 EEST - ethereum/execution-spec-tests</a>
<br/>
</div>
{"title":"Execution Spec Tests | Immunefi","tags":"Talk, Ethereum, Testing, Spec","slideOptions":{"theme":"white","transition":"slide"}}