# Execution Layer Testing Workshop This page has all the links and instructions required to follow along, so if you run into any issues or fall behind you can always catch up. ### ☢️ Link to the slides are below ☢️ - https://docs.google.com/presentation/d/117F-s4Jnf3r7cRIQqAwsYqwIGULHx4JTcdJjW64wZag/edit?usp=sharing (sorry for not making it clear) ### 🤨 Are you stuck? Please raise your hand! There will be a few of us at the side to come and help you out! ### ⭐️ Fast Links - https://github.com/ethereum/execution-spec-tests - http://eest-devnet.ethpandaops.io/ ### ⭐️ Workshop Wifi - Name: **Classroom** - Password: **noclientisolation** ## What will we be doing today? 1. Get to know the testing repo and learn how to set it up. 2. Implement a basic test case and fill it. 3. Find edge-cases in a new EIP and write test for them. 4. Run the implemented tests against a live network. 5. Break the chain 💥 ## What tools will we use today? (pre-requisites) ### Git 1. The `git` command should already be installed in Linux and Mac 2. Verify its installation by running `git --version` 3. Otherwise it can be installed in the following links: - Linux/MacOS - https://git-scm.com/ - Windows - https://gitforwindows.org/ ### VS Code 1. If you already have an IDE installed on your computer that can be used to develop python you can use that. 2. However we **highly recommend** you to use VS Code for this workshop, which can be found in the following link (or just google VS Code): - https://code.visualstudio.com/ ### Extensions 1. If the IDE you are using today is not python native, we recommend to install a python extension to make coding easier. 2. If using VS Code, we *recommend* installing the official Python extension, which can be found in the following link: - https://marketplace.visualstudio.com/items?itemName=ms-python.python ### Ethereum Execution Spec Tests (EEST) 1. This is the main framework to develop tests for the Ethereum protocol 2. Bookmark this link since it will be used extensively! - https://github.com/ethereum/execution-spec-tests ### Custom Ethereum Devnet 1. We will be running our tests in a live ethereum network deployed only for the purposes this workshop, as we need the tools to explore and monitor what happens in the chain. 2. Bookmark this link since it will also be used extensively! - http://eest-devnet.ethpandaops.io/ 3. **Adding metamask to the devnet:** - It’s possible to add this network to metamask so we can use it to send transactions to the network. - Metamask download link: - https://metamask.io/download - These are the configuration options you'll want to add: - Network Name: `eest-devnet` - New RPC URLs (pick one): - `http://eest-devnet.ethpandaops.io:32002/` - `http://eest-devnet.ethpandaops.io:32007/` - Chain ID:`3151908` - Currency symbol: `EESTH` - Documentation for adding custom networks: - https://support.metamask.io/networks-and-sidechains/managing-networks/how-to-add-a-custom-network-rpc/ ## Execution Spec Tests - EVM Tests Repository 1. Python source code 2. Powered by pytest and provides simple to complex parametrization 3. Requires execution-specs (EELS) to fill tests 4. Capable of filling and verifying tests, and running tests against clients or even live networks. ## Parts of a Test 1. Setup (pre-state) - Smart contracts with code to execute - Pre-funded accounts to send transactions 2. Action (transaction) - Transaction(s) that execute an action over the pre-state - Can be a single transaction or many 3. Verification (post-state) - Checks that the virtual machine state mutated into the expected form - Usually by checking balances and storage of accounts ## Initial Setup 1. Install `uv` on your machine by running the following command: - `curl -LsSf https://astral.sh/uv/install.sh | sh` 2. Astral's `uv` documentation for any issues: - https://docs.astral.sh/uv/ 3. Fetch the execution-spec-tests repository: - `git clone --depth 1 https://github.com/ethereum/execution-spec-tests` 4. Move into the execution-spec-tests folder: - `cd execution-spec-tests` 5. Create the python virtual environment: - `uv venv --python 3.12` 6. Perform the initial uv sync: - `uv sync --all-extras` 7. Verify you can run EEST commands: - `uv run fill --collect-only` 8. (Optional) Open the created folder within VS Code: - `code .` ## The Simplest Test We will use this simple test for our initial test execution: - https://gist.github.com/marioevz/fa297d5da592b52d7890a2266e539e73 - Please add it to this subfolder: `tests/cancun` ## Filling A Test - To fill a test, the fill command is used. - Fill the test using the following command: - `uv run fill --fork=Cancun ./tests/cancun/test_simplest.py` ## Executing the simple test on a live network ### Generate a private key - First generate a test account to be able to send transactions. - Use the code here and run the script: - https://gist.github.com/marioevz/2fd79e986befbd711d4a7da7913d42af - The first printed hexadecimal number is the private key, and second one is the address. ### Funding the test address - The execute command uses the private key to generate the transactions that run on the devnet, but it needs funds to be able to do so - Use the faucet to obtain funds at this link: - http://eest-devnet.ethpandaops.io:8080/ - If not possible give us your address and we will send funds to it. - You can check the balance of your account using the devnet block explorer: - http://eest-devnet.ethpandaops.io:36001/ ### Execute the test - Run the execute command to send the transactions that comprise the test to the live testnet. - `uv run execute remote --rpc-seed-key <PRIVATE_KEY> --rpc-endpoint http://eest-devnet.ethpandaops.io:32002 --seed-account-sweep-amount "10 eth" --rpc-chain-id 3151908 --fork Cancun ./tests/cancun/test_simplest.py` - The rpc-endpoint can be either of the 2 below. It should match the one you added to metamask: - http://eest-devnet.ethpandaops.io:32002 - http://eest-devnet.ethpandaops.io:32007 ### Modifying the simple test - We can do some modifications to the simple test: - Change the smart contract - Add smart contracts - Add parametrization - Change the expected outcomes ## Testing a new Ethereum feature ### EIP-5920: PAY Opcode - The EIPs are the updates to the Ethereum protocol that are included in each fork - They can be found in the following repository: - https://github.com/ethereum/EIPs - Today we are focusing on a change that is not yet included in any fork: **EIP-5920** - https://github.com/ethereum/EIPs/blob/master/EIPS/eip-5920.md - Includes a new opcode that sends transfers Ether to another account without executing any of its code - Normally we use CALL or DELEGATECALL opcodes to send value, but this also triggers code execution. ### Writing tests for a new fork: Osaka - When testing a new fork, it needs to be added to src/ethereum_test_forks/forks/forks.py in the EEST repository - In the case of Osaka, the fork already exists. - If an EIP adds a new opcode, we need to create it in src/ethereum_test_vm/opcode.py - For EIP-5920 the opcode PAY has to be added with binary value of 0xf9: - https://gist.github.com/marioevz/8a6ba200a220a89ddaae0379347f5754 - Basic test for EIP-5920: - https://gist.github.com/marioevz/53454ebd39c4fdcec624eef180245edf ### Executing the new test for EIP-5920 - Using the execute command to send the transactions that comprise the test to the live testnet: - `uv run execute remote --rpc-seed-key <PRIVATE_KEY> --rpc-endpoint http://eest-devnet.ethpandaops.io:32002 --seed-account-sweep-amount "10 eth" --rpc-chain-id 3151908 --fork Osaka ./tests/osaka/eip5920_pay/test_pay.py::test_pay` - Again, the rpc-endpoint can be either of the 2 below. It should match the one you added to metamask: - http://eest-devnet.ethpandaops.io:32002 - http://eest-devnet.ethpandaops.io:32007 ### Expanding the EIP-5920 test - We can expand testing of the EIP-5920 with the following tests: - Test sending more balance than what the account has - Test sending balance to a precompile - Test running without enough gas - Test sending balance to self - Try doing an opcode stack underflow ### Finding the EIP-5920 potential consensus issues - What should the error be when the opcode is called without balance?
Last changed by  
 
[email protected]
194

Read more

Execution Spec Tests | Immunefi

Execution Spec Tests x Immunefi 🧪 EEST - ethereum/execution-spec-tests Disclaimer Do not report bugs in our repo with Immunefi! Use our tools to find or consolidate bugs found within Execution Layer clients. Everything discussed is for testing the EL.

11/26/2024
EEST & EELS Devcon SEA Workshop

🦾 STEEL Workshop (starting 14:15) 🦾 📜 EELS - ethereum/execution-specs 🧪 EEST - ethereum/execution-spec-tests STEEL Who? Specifications and Testing for the Ethereum Execution Layer

11/12/2024
EEST & EELS (STEEL 🦾)

Date: Monday 11 Nov, 14:00-15:30, Room 2 Summary: Our aim is to facilitate the discussion around EL client consensus testing using EEST (execution-spec-tests) during fork development, alongside a future EIP spec implementation and testing process utilizing EELS (execution-specs). By the end of this workshop, all EL client teams will have a unified understanding of the past, present & future testing landscape, and have an idea of the future EIP spec implementation process within EELS. Facilitator: Spencer (EEST) & Peter (EELS)

11/10/2024
Hivegrid Spec

What is Hive? Before diving into hivegrid we will go over some preliminary hive information! Hive is an end-to-end test harness used for integration tests against ethereum clients. It generally works by spinning up 3 types of docker containers working alongside the hive controller: Simulators - within hive, tests are grouped by test suites which are grouped by simulators. For example the tests that are specific to the engine api for cancun are grouped within the engine-cancun test suite, which is held within the engine simulator. Tests are ran within their corresponding simulator container and interact with clients using RPC or p2p. Containers must be rebuilt upon changes to the simulator or tests within the simulator. Simulator runtime is usually longer than the build time if there is more than 50 tests.

11/23/2023
Read more from [email protected]
Published on HackMD