ItsGoin v0.3.2 — Decentralized social media network
No central server, user-owned data, reverse-chronological feed. Rust core + Tauri desktop + Android app + plain HTML/CSS/JS frontend. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Scott Reimers
commit
800388cda4
146 changed files with 53227 additions and 0 deletions
909
docs/DISCOVERY-PROTOCOL.md
Normal file
909
docs/DISCOVERY-PROTOCOL.md
Normal file
|
|
@ -0,0 +1,909 @@
|
|||
# distsoc Discovery Protocol v2
|
||||
|
||||
**Status:** Design spec (not yet implemented)
|
||||
**Replaces:** Phase F worm/wide-peer/selective-sync system
|
||||
|
||||
---
|
||||
|
||||
## 1. Overview
|
||||
|
||||
### 1.1 Connection Model
|
||||
|
||||
Every node maintains **101 persistent QUIC connections** over a single ALPN (`distsoc/2`):
|
||||
- **81 social peers** — followed users, audience members, recent contacts
|
||||
- **20 wide peers** — selected for maximum graph diversity
|
||||
|
||||
### 1.2 Three Map Layers
|
||||
|
||||
The system operates across three conceptual layers, each with its own routing map:
|
||||
|
||||
| Layer | Purpose | Map contents | Update mechanism |
|
||||
|---|---|---|---|
|
||||
| **1. Peer Discovery** | Find any node's address | NodeIds + hop distance + addresses (1-hop only) | 2-min diffs, worm lookup |
|
||||
| **2. File** | Content-addressed storage + author update propagation | `node:postid` + media blobs + `author_recent_posts` (256KB) | File replication, piggybacked updates |
|
||||
| **3. Social** | Direct routes to follows/audience | Recent routes to socially-connected nodes | Push (audience), pull (follows) |
|
||||
|
||||
### 1.3 Follow vs Audience
|
||||
|
||||
| | Follow | Audience |
|
||||
|---|---|---|
|
||||
| Initiation | Unilateral — no request needed | Requires request + author approval |
|
||||
| Delivery | **Pull only** — follower pulls updates | **Push** — author pushes via push worm |
|
||||
| Author awareness | Author does not know | Author knows (approved the request) |
|
||||
| Latency | Minutes (pull cycle) | Seconds (push) |
|
||||
| Resource cost | Follower bears cost | Author bears cost |
|
||||
|
||||
---
|
||||
|
||||
## 2. Layer 1: Peer Discovery
|
||||
|
||||
### 2.1 Persistent QUIC Connections
|
||||
|
||||
All 101 connections use a single ALPN (`distsoc/2`) with multiplexed message types over QUIC streams. Connections are kept alive with QUIC keep-alive (20-second interval).
|
||||
|
||||
```
|
||||
┌──────────────────────────────────┐
|
||||
│ This Node (101 conns) │
|
||||
├──────────────────────────────────┤
|
||||
│ 81 Social Peers │
|
||||
│ ├─ Mutual follows │
|
||||
│ ├─ Audience (granted) │
|
||||
│ ├─ Users we follow (online) │
|
||||
│ ├─ Recent sync partners │
|
||||
│ └─ (evicted by priority) │
|
||||
│ │
|
||||
│ 20 Wide Peers │
|
||||
│ ├─ Diversity-maximizing │
|
||||
│ ├─ Re-evaluated every 10 min │
|
||||
│ └─ At least 2 must be anchors │
|
||||
└──────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Mobile mode:** 10 social + 5 wide = 15 connections. Same protocol, smaller maps.
|
||||
|
||||
### 2.2 Social Peer Selection
|
||||
|
||||
Priority order for the 81 social slots:
|
||||
1. Mutual follows (bidirectional) — highest priority
|
||||
2. Audience members (we granted them push access)
|
||||
3. Users we follow who are currently online
|
||||
4. Recently interacted peers
|
||||
5. Random known peers (background routing diversity)
|
||||
|
||||
When a higher-priority peer comes online and all slots are full, evict the lowest-priority peer.
|
||||
|
||||
### 2.3 Wide Peer Selection
|
||||
|
||||
Goal: maximize the diversity of the 2-hop/3-hop map. Selected iteratively every 10 minutes:
|
||||
|
||||
```
|
||||
1. score(peer) = |peer.reported_neighbors − our_known_set| / |peer.reported_neighbors|
|
||||
2. Select highest-scoring peer → wide peer #1
|
||||
3. Merge their neighborhood into our_known_set
|
||||
4. Recalculate all scores (our view expanded)
|
||||
5. Repeat until 20 wide peers selected
|
||||
|
||||
Constraint: at least 2 must be anchors.
|
||||
```
|
||||
|
||||
### 2.4 The 3-Hop Discovery Map
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Hop 1: 101 direct peers │
|
||||
│ Stored: NodeId + SocketAddr + is_anchor + is_wide │
|
||||
│ Source: Direct QUIC connection observation │
|
||||
│ │
|
||||
│ Hop 2: ~5,500 unique nodes │
|
||||
│ Stored: NodeId + reporter_peer_id + is_anchor │
|
||||
│ Source: Peers' 1-hop diffs (their direct connections) │
|
||||
│ │
|
||||
│ Hop 3: ~350,000 unique nodes │
|
||||
│ Stored: NodeId only │
|
||||
│ Source: Peers' 2-hop diffs (their derived knowledge) │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Why ~350K:** With 81 social peers (clustered) + 20 wide peers (diverse), the wide-wide-wide paths cascade diversity through 3 levels. Without dedicated wide peers, 101 random social connections in a clustered graph would reach ~150-200K. With 20 wide peers: ~350K.
|
||||
|
||||
**Storage:** ~11.3 MB (101 × 96 bytes + 5,500 × 66 bytes + 350K × 32 bytes).
|
||||
|
||||
### 2.5 Diff-Based Gossip
|
||||
|
||||
Every 2 minutes, each node sends a diff to each of its 100 other persistent peers:
|
||||
|
||||
```rust
|
||||
pub struct RoutingDiff {
|
||||
pub hop1_changes: Vec<Hop1Change>, // our direct connection changes
|
||||
pub hop2_changes: Vec<Hop2Change>, // derived from received 1-hop diffs
|
||||
pub seq: u64,
|
||||
}
|
||||
|
||||
pub enum Hop1Change {
|
||||
Added { node_id: NodeId, address: SocketAddr, is_anchor: bool },
|
||||
Removed { node_id: NodeId },
|
||||
AddressChanged { node_id: NodeId, new_address: SocketAddr },
|
||||
}
|
||||
|
||||
pub enum Hop2Change {
|
||||
Added { node_id: NodeId, is_anchor: bool },
|
||||
Removed { node_id: NodeId },
|
||||
}
|
||||
```
|
||||
|
||||
**Propagation:** Each node forwards only changes to its OWN view. Recipients derive deeper-hop maps locally.
|
||||
- 1-hop change → peers see it as 2-hop change (next cycle)
|
||||
- 2-hop change → peers see it as 3-hop change (next cycle)
|
||||
- **3-hop propagation: ~6 minutes worst case, ~3 minutes average**
|
||||
|
||||
**No amplification:** You don't re-forward received diffs. You compute your own view's changes and report those.
|
||||
|
||||
### 2.6 Bandwidth (Layer 1 Only)
|
||||
|
||||
At 1% hourly connection churn:
|
||||
|
||||
| Item | Per day |
|
||||
|---|---|
|
||||
| Diff outbound (100 peers × ~200 bytes × 720 cycles) | ~14 MB |
|
||||
| Diff inbound (same) | ~14 MB |
|
||||
| Keep-alive (101 conns × 50 bytes × 3/min) | ~22 MB |
|
||||
| **Layer 1 total** | **~50 MB/day** |
|
||||
|
||||
Initial connection sync (new peer): ~11.3 MB per direction.
|
||||
Optimization: send 1-hop + 2-hop immediately (~364 KB), stream 3-hop in background.
|
||||
|
||||
### 2.7 Worm Lookup (Wide-Bloom + 11-Needle)
|
||||
|
||||
Used when a target NodeId is not in the local 3-hop map. The originator drives
|
||||
the entire search — no forward chain.
|
||||
|
||||
**11-Needle multiplier:** Each profile stores up to 10 `recent_peers` (currently
|
||||
connected peer NodeIds). When searching for target T, we also search for T's 10
|
||||
recent peers = 11 needles. Finding *any* of the 11 gives a route to T.
|
||||
|
||||
**Originator-driven cascade:**
|
||||
|
||||
```
|
||||
Step 0: LOCAL CHECK
|
||||
Look up all 11 needles in local 3-hop map (~350K entries).
|
||||
If any found → resolve address → done.
|
||||
|
||||
Step 1: DIRECT CONNECT
|
||||
If any of the 11 have a stored address, try connecting directly.
|
||||
If found via a recent_peer, connect to that peer, then query
|
||||
them for target's current address.
|
||||
|
||||
Step 2: FAN-OUT (101 peers, 500ms timeout)
|
||||
Send WormQuery { target, needle_peers, ttl=0 } to all connected peers.
|
||||
Each peer checks their local map for ALL 11 IDs (local only, no forwarding).
|
||||
Response includes: which needle found, addresses, ONE wide-peer referral.
|
||||
Collect ALL responses (not just first hit) to gather wide referrals.
|
||||
First hit → resolve address → done.
|
||||
|
||||
Step 3: WIDE-BLOOM (up to 101 referred wide peers, 1500ms timeout)
|
||||
Deduplicate collected wide-peer referrals from Step 2.
|
||||
Connect to each referred wide peer, send WormQuery { needles, ttl=1 }.
|
||||
Each wide peer checks locally AND fans out to their ~101 peers (ttl=0).
|
||||
First hit → done. Total timeout: 3s from start.
|
||||
```
|
||||
|
||||
**TTL semantics:** `ttl=0` = local-only check + return wide_referral.
|
||||
`ttl=1` = local check + fan-out to own ~101 peers (each at ttl=0).
|
||||
|
||||
**Coverage:**
|
||||
- Local check: 11 needles × 350K map = **3.85M effective**
|
||||
- Fan-out (101 peers): 11 × 101 × 350K = **~389M effective** → 98%+ for <200M networks
|
||||
- Bloom round (101 wide peers): 11 × 101 × 101 × 350K = **~39B effective**
|
||||
|
||||
**Expected resolution:** Most lookups resolve at Step 2 (fan-out). Total <500ms.
|
||||
Bloom round needed only for very large networks or cold starts. Total <3s.
|
||||
|
||||
### 2.8 Iterative Address Resolution
|
||||
|
||||
```
|
||||
1. DIRECT — T in 1-hop → have address → connect (instant)
|
||||
2. 2-HOP REF — T in 2-hop → ask reporter peer for address (1 RTT)
|
||||
3. 3-HOP REF — T in 3-hop → ask peers who's closer → chain (2 RTT)
|
||||
4. WORM — T not in map → worm search (1.5-5 sec)
|
||||
5. ANCHOR — Worm fails → target's profile anchor or bootstrap (1-5 sec)
|
||||
```
|
||||
|
||||
### 2.9 Database Schema (Layer 1)
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS discovery_map (
|
||||
node_id BLOB NOT NULL PRIMARY KEY,
|
||||
hop INTEGER NOT NULL, -- 1, 2, or 3
|
||||
reporter BLOB, -- which 1-hop peer reported this
|
||||
address TEXT, -- JSON addr, only for hop=1
|
||||
is_anchor INTEGER NOT NULL DEFAULT 0,
|
||||
is_wide INTEGER NOT NULL DEFAULT 0,
|
||||
updated_at INTEGER NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_discovery_hop ON discovery_map(hop);
|
||||
CREATE INDEX IF NOT EXISTS idx_discovery_reporter ON discovery_map(reporter);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Layer 2: File Storage + Content Routing
|
||||
|
||||
### 3.1 Core Concept
|
||||
|
||||
Every stored file (post + media) carries a small metadata blob: **`author_recent_posts`** (max 256 KB). This blob lists the author's recent post IDs and is updated whenever the file's author publishes new content.
|
||||
|
||||
**Key insight:** If you have *any* file by author X, you passively know X's recent posts. You can then request specific posts from *any peer who has them* — you don't need to find author X.
|
||||
|
||||
This creates a natural CDN: popular authors' post updates propagate through the file storage network as each copy of their files carries the latest post list.
|
||||
|
||||
### 3.2 File Structure
|
||||
|
||||
```rust
|
||||
pub struct StoredFile {
|
||||
/// The post itself (content-addressed, immutable)
|
||||
pub post: Post,
|
||||
pub post_id: PostId, // blake3(content)
|
||||
pub media_blobs: Vec<MediaBlob>,
|
||||
|
||||
/// Author's recent post updates (mutable, refreshed)
|
||||
pub author_recent_posts: AuthorRecentPosts,
|
||||
}
|
||||
|
||||
pub struct AuthorRecentPosts {
|
||||
pub author_id: NodeId,
|
||||
pub posts: Vec<RecentPostEntry>, // newest first
|
||||
pub updated_at: u64, // ms timestamp
|
||||
pub signature: Signature, // author signs this blob
|
||||
// Max 256 KB total serialized size
|
||||
}
|
||||
|
||||
pub struct RecentPostEntry {
|
||||
pub post_id: PostId,
|
||||
pub created_at: u64,
|
||||
pub content_size: u32,
|
||||
pub has_media: bool,
|
||||
pub visibility_hint: VisibilityHint, // Public / Encrypted
|
||||
}
|
||||
```
|
||||
|
||||
### 3.3 Update Propagation
|
||||
|
||||
```
|
||||
Scenario: Author A publishes a new post.
|
||||
|
||||
Path 1: Direct push (audience members, <12 min)
|
||||
A pushes new post to audience members via push worm.
|
||||
Each recipient updates their stored copies of A's files
|
||||
with the new author_recent_posts blob.
|
||||
|
||||
Path 2: File-chain propagation (followers, <12 min typical)
|
||||
A's direct peers receive the update (they have A's files).
|
||||
When ANY peer accesses a file by A, they see the fresh
|
||||
author_recent_posts and can request the new post.
|
||||
Propagates naturally as files are accessed/synced.
|
||||
|
||||
Path 3: Pull on staleness (>1 hour fallback)
|
||||
If author_recent_posts.updated_at is older than 1 hour,
|
||||
the holder triggers an update pull:
|
||||
- Check Layer 3 social route to author → pull directly
|
||||
- Or check other peers who hold author's files → pull from them
|
||||
- Or worm request for author's latest author_recent_posts
|
||||
|
||||
Result: Popular authors' updates reach most file holders within
|
||||
minutes. Unpopular authors' updates reach followers within 1 hour.
|
||||
```
|
||||
|
||||
### 3.4 Requesting Posts via File Layer
|
||||
|
||||
```
|
||||
Scenario: You see author A has new post P in author_recent_posts.
|
||||
You don't have post P stored locally.
|
||||
|
||||
1. Check if any persistent peer has P:
|
||||
→ Quick check via Layer 1 worm query (fan-out to 100 peers)
|
||||
→ Peers check their local post storage, not just discovery map
|
||||
|
||||
2. If found: request post P from that peer
|
||||
→ No need to contact author A at all
|
||||
→ Any peer with the file can serve it
|
||||
|
||||
3. Request posts newest-to-oldest:
|
||||
→ Prioritize catching up on recent content
|
||||
→ Older posts can wait or be skipped
|
||||
```
|
||||
|
||||
### 3.5 Popular Author Optimization (Audience >101)
|
||||
|
||||
When an author has more than 101 audience members, they can't push to all directly. Instead:
|
||||
|
||||
```
|
||||
1. Author pushes post + updated author_recent_posts to their
|
||||
101 persistent peers (including audience members in social slots).
|
||||
|
||||
2. Those peers now have the update in their file storage.
|
||||
|
||||
3. Other audience members / followers discover the update when:
|
||||
a. They access any of the author's files (see fresh author_recent_posts)
|
||||
b. The update hops naturally through file storage:
|
||||
peer A has file → peer B requests file → B gets update too
|
||||
c. The 1-hour staleness check triggers a pull
|
||||
|
||||
No destination declared. Updates flow through the file storage
|
||||
network like water through connected vessels.
|
||||
```
|
||||
|
||||
### 3.6 File Authority Chain
|
||||
|
||||
Each file carries a **route back to its authority (the author)**. This is a personal map of the shortest recently-working path:
|
||||
|
||||
```rust
|
||||
pub struct FileAuthorityRoute {
|
||||
pub author_id: NodeId,
|
||||
pub route: Vec<NodeId>, // chain of peers leading toward author
|
||||
pub last_verified: u64, // ms timestamp
|
||||
}
|
||||
```
|
||||
|
||||
When you need the latest `author_recent_posts` and your cached copy is stale:
|
||||
1. Follow the authority chain (hop by hop toward author)
|
||||
2. Each hop may have a fresher `author_recent_posts`
|
||||
3. Don't need to reach the author — just need a fresher copy
|
||||
4. Update your authority chain with the path that worked
|
||||
|
||||
### 3.7 Database Schema (Layer 2)
|
||||
|
||||
```sql
|
||||
-- File storage with author update metadata
|
||||
CREATE TABLE IF NOT EXISTS stored_files (
|
||||
post_id BLOB NOT NULL PRIMARY KEY,
|
||||
author_id BLOB NOT NULL,
|
||||
content BLOB NOT NULL,
|
||||
media_blobs BLOB, -- serialized Vec<MediaBlob>
|
||||
visibility TEXT NOT NULL,
|
||||
created_at INTEGER NOT NULL,
|
||||
stored_at INTEGER NOT NULL,
|
||||
keep_priority REAL NOT NULL DEFAULT 1.0,
|
||||
-- Denormalized from author_recent_posts for quick staleness check
|
||||
author_update_at INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_files_author ON stored_files(author_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_files_priority ON stored_files(keep_priority);
|
||||
|
||||
-- Author recent posts blobs (one per known author, updated independently)
|
||||
CREATE TABLE IF NOT EXISTS author_recent_posts (
|
||||
author_id BLOB NOT NULL PRIMARY KEY,
|
||||
recent_posts_blob BLOB NOT NULL, -- serialized AuthorRecentPosts (max 256KB)
|
||||
updated_at INTEGER NOT NULL,
|
||||
signature BLOB NOT NULL -- author's ed25519 signature
|
||||
);
|
||||
|
||||
-- File authority routes (personal routing cache)
|
||||
CREATE TABLE IF NOT EXISTS file_authority_routes (
|
||||
author_id BLOB NOT NULL PRIMARY KEY,
|
||||
route TEXT NOT NULL, -- JSON array of NodeIds
|
||||
last_verified INTEGER NOT NULL
|
||||
);
|
||||
```
|
||||
|
||||
### 3.8 File Keep Priority
|
||||
|
||||
When local storage is limited, files are evicted lowest-priority first:
|
||||
|
||||
```
|
||||
priority = pin + (relationship × heart_recency × post_age / (peer_copies + 1))
|
||||
```
|
||||
|
||||
**Pin:** 99999 (user explicitly saved). Even pins compete with each other when device storage is exhausted — lowest-priority pins evicted last.
|
||||
|
||||
**Relationship** (our relationship to the file's author):
|
||||
|
||||
| Relationship | Score |
|
||||
|---|---|
|
||||
| Self (our own post) | ∞ (never evicted unless unpinned) |
|
||||
| We are audience of author | 10 |
|
||||
| We follow author | 8 |
|
||||
| Author's posts have >10 hearts from our network | 5 |
|
||||
| Author's posts have >3 hearts | 3 |
|
||||
| Author's posts have >2 hearts | 2 |
|
||||
| No relationship | 1 |
|
||||
|
||||
**Heart Recency** (how recently the file was hearted by anyone we know):
|
||||
|
||||
| Window | Score |
|
||||
|---|---|
|
||||
| < 72 hours | 100 |
|
||||
| 3-14 days | 50 |
|
||||
| 14-45 days | 25 |
|
||||
| 45-90 days | 12 |
|
||||
| 90-365 days | 6 |
|
||||
| 1-3 years | 3 |
|
||||
| 4-10 years | 1 |
|
||||
|
||||
**Post Age** (how old the post content is):
|
||||
|
||||
| Window | Score |
|
||||
|---|---|
|
||||
| < 72 hours | 100 |
|
||||
| 3-14 days | 50 |
|
||||
| 14-45 days | 25 |
|
||||
| 45-90 days | 12 |
|
||||
| 90-365 days | 6 |
|
||||
| 1-3 years | 3 |
|
||||
| 4-10 years | 1 |
|
||||
|
||||
**Peer Copies** (how many copies exist within 3-hop range):
|
||||
|
||||
Dividing by `(peer_copies + 1)` means:
|
||||
- 0 other copies → full priority (you're the only holder nearby)
|
||||
- 1 other copy → half priority
|
||||
- 10 copies → 1/11 priority
|
||||
- Rare content is prioritized over heavily-replicated content
|
||||
|
||||
**Example calculations:**
|
||||
|
||||
```
|
||||
Pinned vacation photo from 2 years ago, 5 peer copies:
|
||||
99999 + (10 × 3 × 3 / 6) = 99999 + 15 = 100,014 → kept (pinned)
|
||||
|
||||
Audience author's post from yesterday, hearted today, 0 copies:
|
||||
0 + (10 × 100 × 100 / 1) = 100,000 → very high priority
|
||||
|
||||
Random person's post from 6 months ago, 3 hearts, 8 copies:
|
||||
0 + (3 × 6 × 6 / 9) = 12 → low priority, likely evicted
|
||||
|
||||
Followed author's post from last week, hearted 2 days ago, 2 copies:
|
||||
0 + (8 × 100 × 50 / 3) = 13,333 → high priority
|
||||
```
|
||||
|
||||
### 3.9 Storage Budget
|
||||
|
||||
Each node allocates a configurable amount of storage for files (default: 10 GB).
|
||||
|
||||
At 256 KB average file size: ~40,000 files stored.
|
||||
At 1 MB average (with media): ~10,000 files stored.
|
||||
|
||||
The keep priority formula ensures:
|
||||
- Your own posts are always kept
|
||||
- Audience/follow content prioritized
|
||||
- Popular (hearted) content replicated
|
||||
- Rare content preserved (fewer peer copies = higher priority)
|
||||
- Old, common, unrelated content evicted first
|
||||
|
||||
---
|
||||
|
||||
## 4. Layer 3: Social Routing
|
||||
|
||||
### 4.1 Purpose
|
||||
|
||||
Layer 3 is a **personal routing cache** for nodes you have a social relationship with: follows and audience. It stores recently-working routes so you can push/pull content without going through the Layer 1 worm every time.
|
||||
|
||||
### 4.2 Structure
|
||||
|
||||
```rust
|
||||
pub struct SocialRoute {
|
||||
pub target: NodeId,
|
||||
pub relationship: SocialRelationship,
|
||||
pub last_route: Vec<NodeId>, // path that worked last time
|
||||
pub last_success: u64, // ms timestamp
|
||||
pub address_hint: Option<SocketAddr>, // if direct connection worked
|
||||
}
|
||||
|
||||
pub enum SocialRelationship {
|
||||
Follow, // we follow them (pull)
|
||||
Audience, // they granted us audience (we push to them)
|
||||
Mutual, // both directions
|
||||
}
|
||||
```
|
||||
|
||||
### 4.3 How Social Routes Are Used
|
||||
|
||||
**For follows (pull updates):**
|
||||
|
||||
```
|
||||
1. Check if followed author is a persistent peer (Layer 1, 1-hop)
|
||||
→ Yes: content flows in real-time via persistent connection. Done.
|
||||
|
||||
2. Check social route cache (Layer 3)
|
||||
→ Have a recent route? Follow it to reach the author.
|
||||
→ Pull author_recent_posts + any new posts.
|
||||
→ Update the route cache with the path that worked.
|
||||
|
||||
3. Check file layer (Layer 2)
|
||||
→ Have any of author's files? Check author_recent_posts freshness.
|
||||
→ If <1 hour old: we're up to date. Request missing posts via worm.
|
||||
→ If >1 hour old: need fresher data. Follow file authority chain.
|
||||
|
||||
4. Fall back to Layer 1 worm for author's address.
|
||||
|
||||
Typical path for an active follow: step 1 or 2 (fast, no worm needed).
|
||||
```
|
||||
|
||||
**For audience (push updates):**
|
||||
|
||||
```
|
||||
When we create a new post and have approved audience members:
|
||||
|
||||
1. Audience members who are persistent peers (1-hop):
|
||||
→ Push post notification directly on persistent connection. Instant.
|
||||
|
||||
2. Audience members with social routes (Layer 3):
|
||||
→ Follow cached route. Push post via push worm along that route.
|
||||
→ Update route cache on success.
|
||||
|
||||
3. Audience members with no cached route:
|
||||
→ Layer 1 worm to find their address.
|
||||
→ Push post. Cache the route for next time.
|
||||
|
||||
For audience >101: post also pushed to file layer (Section 3.5).
|
||||
File storage network handles further propagation.
|
||||
```
|
||||
|
||||
### 4.4 Route Maintenance
|
||||
|
||||
Social routes go stale when peers change addresses. Maintenance:
|
||||
|
||||
- On successful push/pull: update route + address hint
|
||||
- On failure: clear route, fall back to Layer 1 for fresh address
|
||||
- Periodic: every 30 min, validate routes for top-priority follows/audience
|
||||
- Routes older than 2 hours without verification are considered stale
|
||||
|
||||
### 4.5 Database Schema (Layer 3)
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS social_routes (
|
||||
target_id BLOB NOT NULL PRIMARY KEY,
|
||||
relationship TEXT NOT NULL, -- 'follow', 'audience', 'mutual'
|
||||
route TEXT, -- JSON array of NodeIds (last working path)
|
||||
address_hint TEXT, -- last known direct address
|
||||
last_success INTEGER NOT NULL,
|
||||
last_attempt INTEGER NOT NULL,
|
||||
fail_count INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_social_relationship ON social_routes(relationship);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Unified Protocol
|
||||
|
||||
### 5.1 Single ALPN
|
||||
|
||||
All communication uses one ALPN: `distsoc/2`
|
||||
|
||||
Message types distinguished by 1-byte header on each QUIC stream:
|
||||
|
||||
```rust
|
||||
#[repr(u8)]
|
||||
pub enum MessageType {
|
||||
// Layer 1: Peer Discovery
|
||||
RoutingDiff = 0x01,
|
||||
InitialMapSync = 0x02,
|
||||
WormRequest = 0x10,
|
||||
WormQuery = 0x11,
|
||||
WormResponse = 0x12,
|
||||
AddressRequest = 0x20,
|
||||
AddressResponse = 0x21,
|
||||
|
||||
// Layer 2: File / Content
|
||||
FileRequest = 0x30, // request a post by PostId
|
||||
FileResponse = 0x31,
|
||||
AuthorUpdateRequest = 0x32, // request fresh author_recent_posts
|
||||
AuthorUpdateResponse= 0x33,
|
||||
AuthorUpdatePush = 0x34, // push updated author_recent_posts
|
||||
PostNotification = 0x35, // real-time new post notification
|
||||
|
||||
// Layer 3: Social
|
||||
PullSyncRequest = 0x40, // follower requests new posts since seq N
|
||||
PullSyncResponse = 0x41,
|
||||
PushPost = 0x42, // audience push: new post delivery
|
||||
AudienceRequest = 0x43, // request to join audience
|
||||
AudienceResponse = 0x44,
|
||||
|
||||
// General
|
||||
ProfileUpdate = 0x50,
|
||||
DeleteRecord = 0x51,
|
||||
VisibilityUpdate = 0x52,
|
||||
}
|
||||
```
|
||||
|
||||
### 5.2 Connection Lifecycle
|
||||
|
||||
```
|
||||
1. QUIC connect (distsoc/2 ALPN)
|
||||
2. TLS handshake (verify NodeId)
|
||||
3. Both sides exchange InitialMapSync (Layer 1: 1-hop + 2-hop, stream 3-hop)
|
||||
4. Connection live
|
||||
5. Every 2 min: RoutingDiff exchange (Layer 1)
|
||||
6. On demand: any message type via new QUIC stream (Layer 2/3)
|
||||
7. Keep-alive: QUIC PING every 20 seconds
|
||||
8. Connection persists until peer selection changes
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. How It All Fits Together
|
||||
|
||||
### 6.1 Lifecycle of a Public Post
|
||||
|
||||
```
|
||||
T=0 Author A creates post P. PostId = blake3(content).
|
||||
Stores in local DB. Updates own author_recent_posts.
|
||||
|
||||
Persistent peers (Layer 1):
|
||||
→ PostNotification sent on all 101 connections.
|
||||
→ All persistent peers now have P + updated author_recent_posts.
|
||||
|
||||
Audience push (Layer 3):
|
||||
→ For audience members who are persistent peers: already done above.
|
||||
→ For audience members with social routes: push via route.
|
||||
→ For audience members with no route: worm to find, then push.
|
||||
|
||||
T=<2m Peers' peers see updated author_recent_posts (Layer 2):
|
||||
→ Anyone who has a file by A and syncs with A's peers
|
||||
gets the update in the next gossip cycle.
|
||||
|
||||
T=<12m File-chain propagation (Layer 2):
|
||||
→ Update hops through file storage network.
|
||||
→ Anyone accessing any file by A sees the new post listed.
|
||||
→ Can request P from any peer who has it.
|
||||
|
||||
T=60m Pull cycle (Layer 3, for followers without persistent connection):
|
||||
→ Follower F checks author_recent_posts for A.
|
||||
→ Sees post P is new. Requests it.
|
||||
```
|
||||
|
||||
### 6.2 Discovering a New User to Follow
|
||||
|
||||
```
|
||||
User has author X's NodeId (from out-of-band sharing).
|
||||
|
||||
1. Check Layer 1 map: X in 3-hop? (~350K entries)
|
||||
→ If yes: resolve address via referral chain. Connect. Done.
|
||||
|
||||
2. Worm search (Layer 1): fan-out to 100 peers.
|
||||
25M entries checked per hop, 3-5 hops.
|
||||
→ If found: connect to X. Pull profile + recent posts. Done.
|
||||
|
||||
3. Does anyone we know have X's files? (Layer 2)
|
||||
→ Check if any peer has author_recent_posts for X.
|
||||
→ If yes: get X's recent posts without finding X directly.
|
||||
|
||||
4. Anchor fallback: contact bootstrap anchors.
|
||||
→ Anchors have broad maps. May know X's address.
|
||||
|
||||
5. Once connected to X (or X's file holders):
|
||||
→ Cache social route (Layer 3) for future pulls.
|
||||
→ Store X's files locally → future updates via file layer.
|
||||
```
|
||||
|
||||
### 6.3 Popular Author (1M Audience)
|
||||
|
||||
```
|
||||
Author A has 1,000,000 audience members. Posts a new photo.
|
||||
|
||||
Layer 1: A has 101 persistent connections. PostNotification sent to all.
|
||||
→ 101 audience members get it instantly.
|
||||
→ 101 copies of updated author_recent_posts now exist.
|
||||
|
||||
Layer 2: Those 101 peers have files by A. Each has 101 peers of their own.
|
||||
→ 101 × 100 = ~10,000 peers see fresh author_recent_posts within 2 min.
|
||||
→ 10,000 × 100 = ~1,000,000 peers within 4 min (2 more hops).
|
||||
→ Natural file-chain propagation covers the audience without
|
||||
A doing any extra work beyond the initial 101 pushes.
|
||||
|
||||
No destination declared. The file layer IS the CDN.
|
||||
Popular content replicates because many peers have the author's files.
|
||||
The author_recent_posts blob travels with every file copy.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. File Keep Priority — Detailed
|
||||
|
||||
### 7.1 Formula
|
||||
|
||||
```
|
||||
priority = pin_bonus + (relationship × heart_recency × post_age / (peer_copies + 1))
|
||||
```
|
||||
|
||||
### 7.2 Scoring Tables
|
||||
|
||||
**Pin Bonus:**
|
||||
- Pinned by user: 99,999
|
||||
- Not pinned: 0
|
||||
- Note: even pins compete when device storage is full. A pinned post from 10 years ago with 50 peer copies (priority 100,000) will be evicted before a pinned post from yesterday with 0 copies (priority 109,999).
|
||||
|
||||
**Relationship (to the file's author):**
|
||||
|
||||
| Condition | Score |
|
||||
|---|---|
|
||||
| Self (our own content) | ∞ (never auto-evicted) |
|
||||
| We are in author's audience | 10 |
|
||||
| We follow author | 8 |
|
||||
| Author has >10 hearts from known peers | 5 |
|
||||
| Author has >3 hearts from known peers | 3 |
|
||||
| Author has >2 hearts from known peers | 2 |
|
||||
| No relationship | 1 |
|
||||
|
||||
**Heart Recency (most recent heart from anyone in our network):**
|
||||
|
||||
| Window | Score |
|
||||
|---|---|
|
||||
| < 72 hours | 100 |
|
||||
| 3-14 days | 50 |
|
||||
| 14-45 days | 25 |
|
||||
| 45-90 days | 12 |
|
||||
| 90-365 days | 6 |
|
||||
| 1-3 years | 3 |
|
||||
| 4-10 years | 1 |
|
||||
|
||||
**Post Age (age of the content itself):**
|
||||
|
||||
| Window | Score |
|
||||
|---|---|
|
||||
| < 72 hours | 100 |
|
||||
| 3-14 days | 50 |
|
||||
| 14-45 days | 25 |
|
||||
| 45-90 days | 12 |
|
||||
| 90-365 days | 6 |
|
||||
| 1-3 years | 3 |
|
||||
| 4-10 years | 1 |
|
||||
|
||||
**Peer Copies (copies in 3-hop range):**
|
||||
- Discovered passively: when peers report having a file during sync/worm
|
||||
- Divides priority: more copies nearby = lower urgency to keep our copy
|
||||
|
||||
### 7.3 Examples
|
||||
|
||||
```
|
||||
YOUR OWN post (any age):
|
||||
priority = ∞ → never evicted
|
||||
|
||||
Pinned photo by audience author, 2 years old, 5 copies:
|
||||
99,999 + (10 × 3 × 3 / 6) = 100,014 → kept (pinned dominates)
|
||||
|
||||
Audience author's post from yesterday, hearted today, 0 nearby copies:
|
||||
0 + (10 × 100 × 100 / 1) = 100,000 → very high (rare + fresh + close relationship)
|
||||
|
||||
Followed author's post from last week, hearted 2 days ago, 2 copies:
|
||||
0 + (8 × 100 × 50 / 3) = 13,333 → high
|
||||
|
||||
Popular stranger's post (>10 hearts) from yesterday, 20 copies:
|
||||
0 + (5 × 100 × 100 / 21) = 2,381 → moderate (popular but well-replicated)
|
||||
|
||||
Random person's post from 6 months ago, 3 hearts, 8 copies:
|
||||
0 + (3 × 6 × 6 / 9) = 12 → very low, evicted early
|
||||
|
||||
Unknown person, no hearts, old post, many copies:
|
||||
0 + (1 × 1 × 1 / 11) = 0.09 → first to be evicted
|
||||
```
|
||||
|
||||
### 7.4 Priority Recalculation
|
||||
|
||||
- On heart received: recalculate heart_recency score
|
||||
- On peer_copies change (learned via gossip/worm): recalculate
|
||||
- On time passing: batch recalculate all files daily (age tiers change slowly)
|
||||
- On storage pressure: recalculate and evict below threshold
|
||||
|
||||
---
|
||||
|
||||
## 8. Bootstrap
|
||||
|
||||
### 8.1 New Node First Launch
|
||||
|
||||
```
|
||||
1. Read bootstrap anchors from anchors.json (shipped with app)
|
||||
2. Connect to 1-2 bootstrap anchors (persistent connection)
|
||||
3. Exchange Layer 1 maps (InitialMapSync) — learn ~350K NodeIds
|
||||
4. Begin wide peer selection from learned nodes
|
||||
5. Connect to 20 wide peers (use anchors for initial address resolution)
|
||||
6. Fill social peer slots based on follow list
|
||||
7. Worm search for followed users not yet found
|
||||
8. Within ~10 minutes: fully operational with 101 connections
|
||||
```
|
||||
|
||||
### 8.2 Lightweight Bootstrap (Future)
|
||||
|
||||
```
|
||||
1. Connect to anchor
|
||||
2. "I'm new, give me 200 diverse peers" (~15 KB response)
|
||||
3. Disconnect. Connect to received peers.
|
||||
4. Build maps via normal gossip.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. Network-Wide Numbers (2B Nodes, 10M Clusters)
|
||||
|
||||
### 9.1 Per-Node Summary
|
||||
|
||||
| Metric | Desktop (101 conns) | Mobile (15 conns) |
|
||||
|---|---|---|
|
||||
| Layer 1 map | ~350K entries, ~11 MB | ~15K entries, ~500 KB |
|
||||
| Layer 2 files | 10K-40K files, ~10 GB | 1K-5K files, ~1 GB |
|
||||
| Layer 3 routes | ~200-500 entries, ~50 KB | same |
|
||||
| Layer 1 bandwidth | ~50 MB/day | ~8 MB/day |
|
||||
| Layer 2 bandwidth | ~50-100 MB/day (varies) | ~10-30 MB/day |
|
||||
| Layer 3 bandwidth | ~10-20 MB/day | ~5-10 MB/day |
|
||||
| **Total bandwidth** | **~110-170 MB/day** | **~23-48 MB/day** |
|
||||
| Worm coverage/hop | ~25M (1.25%) | ~2M (0.1%) |
|
||||
| Worm hops to find any target | 3-5 | 5-8 (or anchor) |
|
||||
|
||||
### 9.2 Connection Overhead
|
||||
|
||||
| Resource | 101 conns (desktop) | 15 conns (mobile) |
|
||||
|---|---|---|
|
||||
| Memory (connection state) | ~1.5 MB | ~250 KB |
|
||||
| Keep-alive bandwidth | ~22 MB/day | ~3.2 MB/day |
|
||||
| CPU | Negligible | Negligible |
|
||||
|
||||
### 9.3 Comparison to Previous Design
|
||||
|
||||
| Aspect | Phase F (current code) | Protocol v2 (this spec) |
|
||||
|---|---|---|
|
||||
| Connections | Ephemeral | 101 persistent |
|
||||
| ALPNs | 4 | 1 |
|
||||
| Gossip | Full peer list each time | 2-min diffs, 1-hop forward |
|
||||
| Map depth | 2-hop (~5K) | 3-hop (~350K) |
|
||||
| Content delivery | Pull-only (60 min) | 3 layers: push + pull + file propagation |
|
||||
| File storage | Not managed | Priority-based with keep formula |
|
||||
| Worm coverage/hop | ~2M | ~25M |
|
||||
| Daily bandwidth | ~318 MB | ~110-170 MB |
|
||||
| Popular author scale | Author pushes to all (N work) | File layer propagates (log N hops) |
|
||||
| First-contact latency | 10-30s | 1-5s |
|
||||
|
||||
---
|
||||
|
||||
## 10. Implementation Order
|
||||
|
||||
### Phase 1: Foundation
|
||||
1. Single ALPN (`distsoc/2`) with message type multiplexing
|
||||
2. Persistent connection manager (81 social + 20 wide slots)
|
||||
3. `discovery_map` table (Layer 1)
|
||||
4. 1-hop map population from persistent connections
|
||||
|
||||
### Phase 2: Layer 1 Gossip + Worm
|
||||
5. `RoutingDiff` and 2-min gossip cycle
|
||||
6. 2-hop + 3-hop derivation
|
||||
7. Wide peer diversity scoring
|
||||
8. Worm v2 with fan-out
|
||||
9. Address resolution chain
|
||||
|
||||
### Phase 3: Layer 2 File Storage
|
||||
10. `stored_files` + `author_recent_posts` tables
|
||||
11. File keep priority calculation + eviction
|
||||
12. `author_recent_posts` update propagation
|
||||
13. File authority chain routing
|
||||
14. Post request via file layer (fetch from any holder)
|
||||
|
||||
### Phase 4: Layer 3 Social Routing
|
||||
15. `social_routes` table
|
||||
16. Follow pull via cached routes
|
||||
17. Audience push via cached routes + worm fallback
|
||||
18. Route maintenance (validation, staleness)
|
||||
|
||||
### Phase 5: Integration + Optimization
|
||||
19. Popular author file-chain propagation
|
||||
20. Lazy 3-hop streaming on connection
|
||||
21. Mobile mode (15 conns, smaller maps)
|
||||
22. Delta sync for content (sequence numbers)
|
||||
23. Bloom filter caching (optional)
|
||||
|
||||
---
|
||||
|
||||
## 11. Open Questions
|
||||
|
||||
1. **Peer eviction policy:** When all 81 social slots are full and a higher-priority peer comes online, which peer gets dropped? Need to prevent thrashing (repeatedly connecting/disconnecting the same borderline peers).
|
||||
|
||||
2. **`author_recent_posts` authenticity:** The blob is author-signed, but a malicious peer could serve a stale (valid but old) signed blob. How do we detect this? Perhaps include a sequence number — if you see seq 50 from one peer and seq 45 from another, the seq 45 is stale.
|
||||
|
||||
3. **Peer copy counting:** How do we learn `peer_copies` for the keep priority formula? Passively from worm responses and file requests. Could also be estimated from the author's popularity (audience size / total network penetration). Exact counting isn't needed — order-of-magnitude is sufficient.
|
||||
|
||||
4. **File layer bandwidth:** If every file carries 256 KB of `author_recent_posts`, that's substantial overhead on small posts. Consider: the blob could be fetched separately on demand rather than always bundled. Or: use a compact format (just PostIds, ~32 bytes each, 256 KB = ~8000 recent posts — plenty).
|
||||
|
||||
5. **Storage quotas / 3x hosting rule:** The design spec mentions 3x hosting quota (store 3x as much others' content as your own). How does this interact with the keep priority formula? The quota could set the overall storage budget, with the priority formula deciding what fills it.
|
||||
|
||||
6. **Global lookup for truly isolated nodes:** Worms + anchors handle most cases. For the remaining ~0.01% of lookups that fail, do we need a structured DHT layer? Or is the anchor fallback sufficient at scale?
|
||||
1116
docs/peer-discovery-design.html
Normal file
1116
docs/peer-discovery-design.html
Normal file
File diff suppressed because it is too large
Load diff
Loading…
Add table
Add a link
Reference in a new issue