# [Cancelled] Network Crawler RFP
## Important Update
This RFP is now **cancelled**. The scope of the project has changed since the RFP was launched and we will be publishing a new RFP shortly. **Please do not submit a proposal based on this, as we will not be selecting any of them.** We will update this document with a link to the new RFP when it is live.
## Original RFP
The [Ethereum Foundation](https://ethereum.foundation/) (EF) would like to announce a Request for Proposal (RFP) for an open source Ethereum Network Crawler which can provide statistics about the Ethereum network.
The motivation for this crawler is to enable community users to monitor the Ethereum network more easily and contribute functionality back to the project. The EF would likely run a publicly accessible instance of the crawler.
## Crawler Requirements
These requirements are split between "Must Haves", which should be delivered as part of the initial crawler, and "Nice to Haves", which could be delivered at a future date. The EF is also open to suggestions/improvements on these.
### Must Haves
* The crawler should run as a standalone piece of software (i.e. does not require running a separate Ethereum client) which gathers data about nodes it sees on the Ethereum network. Specifically, it should gather:
* Total number of peers on the network (over time, so history can be built), and for each peer:
* Client type (go-ethereum, open ethereum, nethermind, besu, etc.);
* Public key
* Node software (software, version and build)
* Network ID
* Fork ID
* Highest block height, difficulty and hash
* IP address and port
* First seen timestamp
* Last seen timestamp
* Connection type
* The crawler should store this data in a way that makes it easy to build dynamic dashboards, such as:
* Number of nodes on the network over time;
* Average "age" of peers over time;
* Percentage of peers who have upgrade to a client that is compatible with a specific network upgrade;
* Percentage of peers who support a specific `eth` protocol version;
* The crawler should be released under a permissive open source license (preferably Apache 2.0 or MIT, GPLv3 if stricly necessary).
### Nice to Haves
* Prometheus/Influx DB support to export data;
* A UI/hosted version of the crawler which displays the dashboards mentioned above;
* Documentation about how to contribute to the crawler.
An initial version of the crawler should be delivered by **July 1, 2021**.
The EF will keep this RFP open for submissions until April 23rd, and then take one week to deliberate and finalize the agreement with the selected party.
Work on the crawler could therefore begin around May 1, which leaves ~2 months for an initial implementation.
Further grants could be provided to continue work on the crawler past the initial version.
## Fee Structure
Based on the timeline of this audit, the chosen vendor will be expected to submit 2 invoices:
* a first invoice of 40% of the total engagement fee at the start of the engagement
* a second invoice of 60% of the total engagement fee at the end of the engagement, following the delivery of the summary report and a 5 day review period.
### Payment method
The vendor will be given the option to be paid in Fiat money (via bank transfer) or ETH.
If the vendor chooses to be paid in ETH, the value of ETH described under the agreement will be the value in USD at NYSE closing time (4 PM EST) on the day prior to the due date (as described at https://www.coingecko.com).
## Selection criteria
The selected vendor will have significant expertise in the areas necessary to meet the needs and requirements set forth in this RFP. Particularly:
* Familiarity with operating Ethereum nodes;
* Familiarity with extracting network-level information from the Ethereum network;
* Proven track record in delivering high-quality open source software.
## Bidding instructions
Upon reception of this request, interested vendors are expected to confirm receipt and intention to bid on the engagement.
In this confirmation, they should explain what they need from us in order to get started. As well as bring attention to areas in which they feel they do not have significant expertise. We will use this information to try to provide appropriate entrance material for the engagement.
Proposals must be submitted before April 23, 2021. We expect to take 1 week to deliberate and respond.
Please send initial confirmations, and proposals to the following address: [email protected]
Feel free to send us any questions you may have: [email protected]