# Spadina rehearsal launch retrospective notes
- Spadina launch call was running well 👍
- Beaconcha.in's genesis countdown feature is awesome
- Prysm responded quickly on the release process issue below. Spadina is getting recovery. (~63.50% participant rate at epoch 40)
### [Client] Release process failure
- Prysm/Terence: (for [peer connection issue](https://github.com/prysmaticlabs/prysm/issues/7372)) This is more on our release process. All the well known issues were discovered in alpha-v26 and fixed in latest master. We missed releasing alpha-v27 last night which would have contained all the Spadina updates. We now are helping the stakers in our discord to get alpha-v26 working with Spadina, It involves using 2 additional CLI flags. (See Nishant’s announcement)
- The detailed report: [No finality at Spadina testnet launch (Incident #11)
### [Client] New release announcement communication
- Dankrad/Carl: Many teku users didn't know there was [a critical bugfix within 24 hours before the launch](https://github.com/PegaSysEng/teku/pull/2863).
### [Client] Issues of running nodes for `>=2` testnets on the same machine
- Justin: to get Lighthouse working I needed to manually specify a datadir (separate from the Medalla one)
- Prysm/Terence: Some of them are having issues with docker where running both Medalla and Spadina via docker compose
### [Infrastucture] Beaconcha.in
- spadina.beaconcha.in was down for a while (at ~1PM UTC). But they fixed it soon.
### [Infrastucture] Launchpad
- [Prysm's instruction link is pointing to Medalla instruction](https://github.com/ethereum/eth2.0-deposit/issues/179#issuecomment-700679181) so the users wrongly used Medalla config when they wanted to run on Spadina
### The users forgot/ignored the launch date?
- No incentives on testnets
## Improvement suggestions
- Let's do another dress rehearsal
- To notify users in time, clients can maintain opt in email lists for critical updates
- Set up eth2stats next time so that we can have better visibility of peer connections.
- Improve communication. Use EF blog to announce the client release versions 3 days before the launch.
- Concur [Pico on ethstaker discord](https://discordapp.com/channels/694822223575384095/738442543477555280/760488951030218832), we can have a separate staker checklist on the launchpad so that the user can come back to review it after the client software is ready.
- Ask client teams to provide mainnet landing page URL/the release announcement center URL for launchpad to link to.
- Carl: Add a static "preparing for genesis" page on launchpad
- Terence: I think `GENESIS_DELAY` at 2 days is too short. Say we discover at bug at genesis, there's 2 days to fix, test and release. Ideally we will like more time to test and verify it doesnt break existing testnet (e.g. medella)
- Afri: Let's talk about release management/release strategies!
- Client teams should maintain stable release branches that:
- only recieve bug fixes
- don't recieve new features
- don't get massively refactored
- For new features/refactoring/etc. you should use beta/alpha/nightly releases.
- End users in most cases want _stable_ releases, especially on mainnet or pre-launch testnets.
- We had similar issues prior to Medalla launch, the time to introduce stable releases is now.
- I'm happy to consult on different release strategies.