# How to run a checkpoint sync endpoint This document covers the high level ideas behind setting up a Beacon chain Checkpoint Sync endpoint. It **does not** go over the low level details like which exact commands to run. Instructions on how to expose an endpoint to the internet (i.e. https, dns) are out of scope for this guide. [Check out this document for an idea on how end-users will use your endpoint](https://notes.ethereum.org/@launchpad/checkpoint-sync). ## Overview Checkpoint syncing is the process where a fresh beacon node jumps to a finalized checkpoint that is relatively close to the head of the chain. Fetching the state/blocks for that finalized checkpoint from another beacon node, node operators are able to spin up a beacon node and start performing validating duties within a matter of minutes. Each client has implemented this process slightly differently. For more details on the routes required by each client [check out this doc](https://notes.ethereum.org/5uV7UE85RFqzGtRY2GHO5A?view). ### Checkpointz This guide recommends running [checkpointz](https://github.com/samcm/checkpointz), but the alternative of directly exposing your beacon node is completely viable and is often the simplest to setup for `state` providers. The main benefits of `checkpointz` are that it ensures no requests are directy routed to your upstream beacon node (shielding it from a direct DOS attack), while also providing a frontend for users to compare state/block roots if they get their checkpoint data from somewhere else (i.e. providing `verification`). :::warning :warning: `checkpointz` is still being actively developed. Use with caution. ::: ## Running an endpoint The checkpoint sync process is broken down in to 2 individual steps. 1. Downloading the `state`. The beacon node downloads the state from the endpoint and uses it to jump to the finalized checkpoint returned by the endpoint. i.e. providing `state` 2. The operator verifies that the sync went as expected by cross checking their beacon node against a source they trust. i.e. providing `verification` **To run a checkpoint sync endpoint you need to decide if you'll be providing `state`, `verification` or both.** ### Options #### Option A - Beacon Node via Nginx > Provides: `state` > Best suited for: Entities/organizations with bandwidth available to serve large payloads Exposing your beacon node to the internet using nginx as a reverse proxy to only allow the required routes. This option is the simplest to setup, but does not provide users with an easy way to use your endpoint as a way to verify their checkpoint sync worked as expected. #### Option B - Checkpointz in 'full' mode > Provides: `state`, `verification` > Best suited for: Entities/organizations with bandwidth available to serve large payloads Exposing `checkpointz`in `full` mode to the internet. This mode provides data that users can checkpoint sync from, and also provides a frontend for cross referencing that the sync went as expected. #### Option C - Checkpointz in 'light' mode > Provides: `verification` > Best suited for: Anyone with a synced beacon node. i.e. individuals, home stakers Exposing `checkpointz` in `light` mode to the internet. Allows users to get their state from somewhere else and then use your Checkpointz instance to verify their sync. #### Comparison | | A: Beacon Node via Nginx | B: Checkpointz (full) | C: Checkpointz (light) | |:--------------------------------------------------------------------------:|:------------------------:|:---------------------:|:----------------------:| | Users can use endpoint to checkpoint sync from (i.e. provides `state`) | ✅ | ✅ | ❌ | | Users can use endpoint to verify their sync against (i.e. provides `verification`) | ❌ ^(1)^ | ✅ | ✅ | | Load on upstream beacon nodes | Very High | Low | Very low | | Bandwidth requirements | High | Medium | Very low | | Only serves required beacon APIs | ✅ | ✅ | ✅ | | Only serves state requests via SSZ | ❌ | ✅ | ❌ | | Caches requests to ensure no request is sent directly to beacon node | ❌ | ✅ | ✅ | | Adds HTTP cache headers based on content | ❌ | ✅ | ✅ | | Provides a frontend for users to compare state/block roots against | ❌ | ✅ | ✅ | | Provides additional Prometheus metrics & Grafana dashboard | ❌ | ✅ | ✅ | 1: Users will need to query the API. ### Requirements - A fully synced beacon node for the network you're running the endpoint for. - **Strongly recommended**: This beacon node should only be tasked with serving checkpoint data (i.e. don't validate on this instance) - Teku requires a fresh sync from genesis with the `--data-storage-mode archive` flag set to be used with Checkpointz in `full` mode. - An instance to run `nginx` (and/or `checkpointz`). - Note: Can be the same instance as the beacon node, but this is not recommended if your beacon node is doing anything other than serving checkpoint sync data. - Hardware requirements: - Nginx: ``` 256m cpu (1/4th of a vCPU) 512mb memory ``` - Checkpointz in `full` mode: ``` 512m cpu (1/4th of a vCPU) 2gb memory ``` - Checkpointz in `light` mode: ``` 50m cpu (1/20th of a vCPU) 50mb memory ``` These requirements are a rough estimate. They will increase or decrease depending on load. - Any CDN if using `checkpointz` - `checkpointz` will add HTTP cache control headers to most `/eth/` responses. If you add a CDN in front of your endpoint some of these requests can be returned at the edge - reducing the load on your instances and reducing the amount of bandwidth required. `checkpointz` is aware of the content being served, and will change the amount of time that a response can be cached for accordingly. I.e. the `genesis` beacon state can be cached for hours, but the `finalized` beacon state can only be cached for a few minutes. - **All clients will fetch a beacon state from your endpoint if you are providing `state`. This response, at time of writing on Mainnet, is 60MB when encoded as SSZ.** ## Instructions ### Option A :::warning :warning: Option A is STRONGLY not recommended if you are validating using this beacon node. ::: #### Configuring NGINX Note: This NGINX config is the bare minimum required for exposing a beacon node for checkpoint syncing. Additional configuration may be required depending on your stack. ``` events { worker_connections 1024; } http { upstream beacon { server 192.168.0.138:5052; # Your beacon node address } server { listen 8080; location /eth/v1/beacon/genesis { proxy_pass http://beacon; } location /eth/v1/beacon/states { proxy_pass http://beacon; } location /eth/v1/beacon/blocks { proxy_pass http://beacon; } location /eth/v1/debug/beacon/states { proxy_pass http://beacon; } location /eth/v2/beacon/blocks { proxy_pass http://beacon; } location /eth/v2/debug/beacon/states { proxy_pass http://beacon; } } } ``` ### Option B #### Running Checkpointz in full mode Check out the `Getting Started` section of the checkpointz [README](https://github.com/samcm/checkpointz#getting-started) for different options. Officially supported: - Via a plain binary - Via Docker - Via Kubernetes w/ Helm Chart #### Configuring Check out the [README](https://github.com/samcm/checkpointz#configuration) for a full reference on how to configure your instance. Example config: ``` beacon: upstreams: - name: remote address: http://localhost:5052 # Your beacon node address dataProvider: true checkpointz: mode: full ``` #### Exposing Checkpointz It's recommended to run a basic webserver like NGINX to reverse proxy to checkpointz. This instance should negotiate TLS etc. Point it at the `global.listenAddr` value of Checkpointz. #### Collecting metrics Metrics can be collected on the `global.metricsAddr` address of your checkpointz instance. If you're running on Kubernetes via the helm chart, setting `serviceMonitor.enabled=true` should set everything up (assuming your have the Prometheus Operator installed on your cluster). #### Grafana dashboard Install the Checkpointz dashboard here: https://grafana.com/grafana/dashboards/16814. Configure it with your Prometheus datasource and you're all set. ### Option C #### Running Checkpointz in light mode Follow the instructions for Option B, but set the config option `checkpointz.mode=light`.
Last changed by  
 
[email protected]
1139

Read more

Slashings

Comparing MigaLabs data vs Xatu. https://gist.githubusercontent.com/samcm/6534158d275a8ee9ac615d07683a39e4/raw/f2b7c9c73b78c62456b1997deb4311e48c8fc7d5/xatu.csv WITH -- CTE for attester slashings with intersection of attesting indices attester_slashings AS ( SELECT arrayIntersect(attestation_1_attesting_indices, attestation_2_attesting_indices) AS intersected_indices, meta_network_name

9/18/2024
Xatu CL Mimicry Ideas

This document contains ideas for achieving Xatu consensus layer mimicry, which is a thin client that can stay connected to the beacon p2p network with minimal resources with the goal of monitoring and exporting data on the network for later analysis. Goals 1. Exports beacon p2p events for analysis :::warning Severity: Required ::: Notes

2/14/2024
Untitled

Ethereum Prometheus alerts Nearly all of our Ethereum network alerts are based on a % of the nodes we run for a network hitting a condition. We generally don't alert on individual instances having issues through our Ethereum channels. Alerting on a % of nodes seeing a condition generally leads to a higher signal-to-noise ratio (i.e. when there is an actual issue), but some of these can still be pretty annoying so might need individual tweaking. Finalized Epoch Stalled on % of nodes we run on a network Description: Only alerts if more than 35% of our nodes on a network have had a stalled finalized epoch for more than 20mins. Alert config:

4/6/2023
EIP4844-Devnet 4

Links Testnet Homepage Instances Running Prysm Lighthouse Lodestar Teku Nimbus

2/7/2023
Read more from [email protected]
Published on HackMD