# Goerli issue tracker - Nimbus being offline - Lodestar partially being offline - Why the Besu re-org was so large? Why was it a 30 slot reorg instead of like 2-3? - Why ergion stalled for so long before fixing themselves? - Why nethermind was offline for a while after ttd was hit? - Rocketpool node producing invalid blocks? ### General debug info: - Paul's fork choice analysis: https://hackmd.io/uexPVG0fQwu96FYm_7dNmA - Teku/Besu reorg: https://hackmd.io/fjvxrKzbRdCBf5y-hwqnRg ### Nimbus offline - The nimbus nodes were using an older version of geth on Sepolia that still allowed for the engine API to be exposed without a JWT token. They reused the setup, but with the latest version of geth that doesn't allow engine API communication without a JWT token. This broke their setup. - The nimbus nodes should slowly come back online once the ports and JWT configuration is fixed. **Status**: Paritally resolved, some nodes still need the update. Participation rates climbed above 80% as a result of the fixes. Some nodes are still attesting sporadically, the setup would definiltey need to be monitored closely. ## Lodestar partially offline - Some lodestar nodes didn't have an EL configured and synced. - The issue affected a subset of nodes and not the whole fleet **Status**: Patched and deployed. Config issue, no other info needed. ### Besu reorg - Adrian and the Consensus teams have worked on a comprehensive doc found [here](https://hackmd.io/fjvxrKzbRdCBf5y-hwqnRg). - TLDR: Besu incorrectly invalidated the canonical chain transition block and the Teku node proposed on its own head (as expected). A Teku restart re-issued all its requests and this time Besu responded correctly, allowing Teku to re-org onto the correct chain. **Status**: Unresolved and unpatched, afaik no remaining questions or data needed ## Nethermind offline post TTD - Nethermind processes pre-merge and post-merge blocks differently. Post-merge they process everything from a CL to return valid/invalid. Nethermind can only process one new block for every payload. The transition is a tricky case because the CL could ask nethermind to process a block with a TD higher than what nethermind sees. To solve this, there is a force processing logic that has been added. However, In force processing, they should process till they find the parent state or main chain. - TLDR: Multiple terminal blocks are hard to deal with. The issue can be seen in failing hive tests. **Status**: Fix is known, it will be released and checked with the hive tests. ## Erigon stall before self-healing - If the transition block is built on top of side fork terminal block, then the nodes with this state running erigon will get stuck for 128 slots until they switch to this branch optimistically. **Status**: Likely a client issue, Needs discussion for a fix. ## Rocketpool producing invalid blocks - Potentially related to a transaction index bug in Erigon `v2022.07.04`, yet to be confirmed. **Status**: Potentially updating to latest `devel` commit.