### ethereum/execution-spec-tests
<style>
.reveal .slides > section > section {
text-align:left;
}
p {
text-align: left;
}
</style>
##### 🐍 python test cases & framework to generate tests for execution clients 🧪
<div style="text-alight: center">
<img src="https://github.com/ethereum/execution-spec-tests/blob/main/docs/img/execution_spec_tests.png?raw=true" height=150 alignment="center">
</div>
<div style="text-alight: center">
<a href="https://github.com/danceratopz/"> danceratopz</a>, <a href=https://github.com/spencer-tb>spencertb</a> & <a href="https://github.com/marioevz">marioevz</a>
<br/>
<a href="https://github.com/ethereum/pm/issues/808">acde 164</a>, 2023-06-22
</div>
---
### Testing Team Aims
- Ensure that implementations meet spec.
- Provide a source of truth for client implementations.
- Improve the development cycle for all client teams.
➡️➡️➡️ Get in touch with test ideas, feedback,...
</section>
---
### execution-spec-tests background
- 📦 A framework & library that enable the implementation of test cases in Python.
- 🐣 Q4 2021: - Initial developement.
- 🚀 Q3 2022--Q1 2023: Test case development started in earnest.
- 🔧 NOW: Framework Improvements, Online Doc, 4844 Test Cases
---
### What's different about it?
1. Test cases are implemented as Python code, not YAML.
2. The repo also contains the tooling to generate fixtures from transition `t8n` tools.
3. It avoids a two-step approach:
```graphviz
digraph {
compound=true
rankdir=LR
graph [ fontname="Source Sans Pro", fontsize=20];
node [ fontname="Source Sans Pro", fontsize=18];
edge [ fontname="Source Sans Pro", fontsize=12];
subgraph cluster1 {
concentrate=true
a [label="Source code\n(available?)"] [shape=box]
b [label="YAML\n\"Spec\""] [shape=box]
c [label="JSON\n Fixtures"] [shape=box]
a -> b -> c
label="Fixture Generation"
}
}
```
<!-- - Code (available?) -> YAML -> JSON -->
---
### Why do we like it?
👉 Test cases as code
<div style="text-alight: center">
<img src="https://storage.googleapis.com/ethereum-hackmd/upload_dc278962bd4bde6e1ec00d734811cfec.png" height=400 alignment="center">
</div>
---
### Test Fixture Generation with execution-spec-tests
---
### Test Fixture Generation with execution-spec-tests
---
#### Types Of Tests
- <b> State Tests </b> - Validate opcodes, gas usage & smart contract interactions.
- <img src="https://storage.googleapis.com/ethereum-hackmd/upload_c1a56198f98caec44adb7d435b1482ca.png" alt="State Test Diagram" style="border:none; box-shadow: none; width: 500px;">
- <b> Blockchain Tests </b> - Confirm block interactions, rejections & fork transitions.
- <img src="https://storage.googleapis.com/ethereum-hackmd/upload_5ef2d83dd07bcb46a3b17eeaca803b3a.png" alt="State Test Diagram" style="border:none; box-shadow: none; width: 500px">
---
<section data-background-iframe="https://ethereum.github.io/execution-spec-tests/tests/homestead/yul/test_yul_example/"
data-background-interactive>
</section>
---
<section data-background-iframe="https://ethereum.github.io/execution-spec-tests/tests/cancun/eip4844_blobs/test_point_evaluation_precompile/#tests.cancun.eip4844_blobs.test_point_evaluation_precompile.test_invalid_precompile_calls"
data-background-interactive>
</section>
---
### EIP-4844 Testing Using Execution-Spec-Tests
- Comprises all Execution Layer updates:
- Blob Type (3) Transactions (Minimal version)
- BLOBHASH opcode
- ExcessDataGas, DataGasUsed header fields
- Point Evaluation Precompile
---
- Python allows efficient parametrization and automatic generation of tests.
- E.g. on spec changes, such as Target/Max blob changes, test combinations are automatically updated:
| 2/4 Blobs | 3/6 Blobs |
| -- | -- |
|  |  |
---
<section data-background-iframe="https://ethereum.github.io/execution-spec-tests/tests/cancun/eip4844_blobs/test_blob_txs/index/test_cases/"
data-background-interactive>
<div style="position: absolute; width: 40%; right: 0; box-shadow: 0 1px 4px rgba(0,0,0,0.5), 0 5px 25px rgba(0,0,0,0.2); background-color: rgba(0, 0, 0, 0.9); color: #fff; padding: 20px; font-size: 20px; text-align: left;">
<p>Full list of test cases available in the documentation.</p>
</div>
</section>
---
### Links
Repo:
<p>
<a href="https://github.com/ethereum/execution-spec-tests">https://github.com/ethereum/execution-spec-tests</a>
</p>
<br/>
Docs:
<p>
<a href="https://ethereum.github.io/execution-spec-tests/">https://ethereum.github.io/execution-spec-tests/</a>
</p>
---
{"title":"ethereum/execution-spec-tests","tags":"Talk, Ethereum, Testing, ACDE","description":"python test cases & framework to generate tests for execution clients","slideOptions":{"theme":"white","transition":"slide"}}