____                      
|  _ \  __ ___   _____  
| | | |/ _` \ \ / / _ \ 
| |_| | (_| |\ V /  __/
|____/ \__,_| \_/ \___|

Dave is a distributed hash table. It may also be described as an anonymised packet-sharing peer-to-peer network protocol. This is a thin layer over UDP, with many possible decentralised applications.

Packets of data up to 1424 bytes in length are pushed and pulled via a set of operations continuously executed at random. We call these packets "dats".

Storage is prioritised according to the age and difficulty of the proof-of-work. We call this "mass".

A remote peer earns trust when a dat not already in the local hash table is received. The amount of trust earned is equal to the mass of the dat. Trust is used to modify the probability of a peer being randomly selected for a small subset of operations, such as seeding and pushes. Trust values are currently not gossiped, as this creates a complex attack vector.

Below is an outline of this implementation, although different implementations could be interoperable. There are many possible optimisations that don't necessitate changing the message format. The project is still in an early enough phase that we can also modify the message format easily.



1.      Configurable Settings

UdpLstnAddr     *net.UDPAddr        Listening address:port
Edges           []netip.AddrPort    Bootstrap peers
Epoch           time.Duration       Base cycle period, lower runs faster, using more bandwidth
Prune           int                 Interval between refreshing dat & peer maps
ShardCap        int                 Shard capacity (number of dats per shard/difficulty level)
Log             chan<- []byte       Log message output
LogLvl          LogLvl              Log level
BackupFname     string              Dat table backup filename



2.      Message Operation Codes

GETPEER     Packet containing only the op-code. Remote should reply with NPEER random peer descriptors.
PEER        Packet containing NPEER peer descriptors.
DAT         Packet containing a value, time, and output of the cost function: work, salt.
GET         Packet containing the work hash, remote should reply with a message of op-code DAT, if available.



3.      Binary Serialisation

A message is serialized into binary using protobuf.

Transpiling Protobuf Spec for Go:
#!/bin/bash
protoc --go_out=. dave.proto

FIELD       DESCRIPTION                                 BYTE LENGTH

OP          Operation code.                             1
PEERS       List of peers.                              20*NPEER
VAL         The data.                                   0 | <= 1352
TIME        Little-endian unix milliseconds.            0 | 8
SALT        Random bytes used to solve WORK.            0 | 32
WORK        BLAKE2B256(SALT, BLAKE2B256(VAL, TIME)).    0 | 32



4.      Peer Discovery & Liveness

The protocol ensures a cohesive network by combining liveness and peer discovery into a single pair of messages (GETPEER & PEER). A node may reply to a GETPEER message with a PEER message with up to NPEER peer descriptors.



5.      Mass

MASS = DIFFICULTY * (1 / AGE_MILLISECONDS)

Where DIFFICULTY is the number of leading zero bytes in WORK, amd AGE_MILLISECONDS is calculated from message TIME and current time.



6.      Replacement by Mass

Every PRUNE EPOCH, a user defined (ShardCap) number of most massive dats per shard are kept, and the remaining dropped. In this way, mass may be thought of as a property that decays over time.



7.      Sharding

The data structure used to store dats is a [][uint64]. The first dimension is called the shard, and used to group dats by difficulty of the proof of work, in number of leading zero bits. The second dimension is derived by hashing the work with xxHash.



8.      Random Push 

Every SEED EPOCH, each node sends one random dat to one random peer, excluding edges. This ensures reliable distribution and sender anonymitiy.

Propagating dats in this way ensures that an adversary is unable to create a timing attack to discern the source of a dat, even with a broad view of network traffic.



9.     Random Push of Recent Dat

Every PUSH EPOCH, each node sends one recent dat to one random peer. The dat is read from a ring buffer. The ring buffer is written to when a novel dat is received. This ensures timely dissemination of data, even for large datasets.



10.     Edges

An edge is an ip:port address used to bootstrap to when starting. This entry will not be removed from the peer table, even if the remote becomes unreachable.