# Revisiting Proto's DHT -- the "Peer DHT" - Nodes use discv5 to find other nodes "in some range" - They then create an overlay table that covers the key space, so that they have someone to ask for each part of the key space - The nodes in the overlay table are scored according to how they reply to one's request - This means we can see this as a second type of "light" peer, only used for samples # Special properties of our problem compared to generic DHT - The data is equally distributed in key space by design - There is a fixed amount of data (spamming the table with data is impossible) - We know what data a node should have. We can downscore nodes that are not able/willing to provide data that they should have # The "T" Peer network Say we also want to use the same DHT table structure in order to distribute the samples. Then we should additionally connect with peers with nearby keys so that we can share samples with them:  Each node thus maintains two sets of light peers: - Global peers that evenly cover the keyspace and are used for querying the DHT - Global peers are scored on how well they reply to sample requests - Local peers that are used to disseminate samples - Local peers are scored based on how early they will first send relevant samples # Total load thought experiment Assumption: We need at least 10 nodes in each bucket, so we need to make the probability of having less than 10 extremely small. 5000 nodes (~ current network size): - 100 buckets @ 50 nodes average - Each node responsible for 1% of samples - Dissemination load: - 7 * 1% * 64 MB = 4.48 MB/block (!!!) - Storage load: - 1% * 64 MB * (30 days / 12 s) = 138 GB (!!!) 100,000 nodes: - 1000 buckets @ 100 nodes average - Each node responsible for .1% of samples - Dissemination load: - 7 * .1% * 64 MB = 0.45 MB/block - Storage load: - .1% * 64 MB * (30 days / 12 s) = 13.8 GB We can probably design dissemination more efficiently? - Reduce replication factor because "local peer" network is tiny - places larger responsibility on initial distribution (Remark: With an indivitual validator being currently responsible for 4 lines out of 1024 or ca. 0.4%, the DHT load for sample dissemination is high :( ) # Open Questions - How to allow "light nodes" that only query but cannot provide data - If we do this, how can these light nodes reconstruct data in case of data censorship? - How to handle imbalances in the node table - Nodes need to have a high tolerance for peer table coverage - What if an adversary spams some part of the key space with "good" nodes? Other nodes in the same key space would then find it difficult to find peers - How to handle discv5 spamming -- can we somehow "gate" it to only allow good nodes in? - How to automatically detect a good radius? # Key space ideas - We probably don't need to hash samples at all to derive a key but simply use a function of sample (row, column) as the key. This has the advantage that one node always has the same coordinates in sample space - We can use this property to allow "subscriptions" to a node. For example a validator can simply subscribe to a number of nodes to always get one row or column - This obviates the need for any subnets for sample dissemination - We can also design the key space to be "compatible" with the 2d structure - Make nodes responsible for samples in the same row or same column - This also makes row/column subscriptions cheaper -- you only need to ask very few light peers to cover a whole row/column - Allow nodes to choose their ID rather than deriving it by hash; allow multiple IDs - The adversary has this ability, so why not make normal nodes use it to the advantage of the network? - Use case 1: During discovery, you find no good peers in some keyspace area (or only very few). Or all peers turn out to be unreliable. Enter that keyspace yourself - Use case 2: You expect to download much more than other nodes; e.g. you run many validators. Due to reciprocity, other nodes will decrease your score if you try to download all this data from them as they get too many requests from you. So you enter with many different IDs rather than just one,