v0.5.3-beta — 2026-04-19

# Design Document

This is the canonical technical reference for ItsGoin. It describes the vision, the architecture, and the current state of every subsystem — with full implementation detail. See the download page for the release changelog.

## 1. The Vision

"A decentralized fetch-cache-re-serve content network that supports public and private sharing without a central server. It replaces 'upload to a platform' with 'publish into a swarm' where attention creates distribution, privacy is client-side encryption, and availability comes from caching, not money."

The honest promise: The CDN is an attention-driven delivery amplifier, not a storage guarantee. Hot content spreads naturally through demand; cold content decays unless intentionally hosted. Authors are responsible for their own content durability — a post backup/export tool is the author's safety net, not the network's job. The system is a loss-risk network — best-effort availability, not durability guarantees.

### Guiding principles

- Our distributed network first, direct connections always preferred
- Social graph and friendly UX in front, infrastructure truth in back
- Privacy by design: public profile is minimal, private profiles are per-circle, social graph visibility is controlled
- Don't break content addressing (`PostId = BLAKE3(post)`, visibility is separate metadata)
- Your feed is yours: reverse-chronological by default, no algorithmic ranking, user-controlled discovery
- Three separate layers — Mesh (structural backbone), Social (follows/audience/DMs), File (content storage/distribution) — each with its own connections and routing

## 2. Identity & Bootstrap

### First startup

1. Identity: Load or generate ed25519 keypair from `{data_dir}/identity.key`. `NodeId` = 32-byte public key. A unique device identity is also generated for multi-device coordination (see Section 23).
2. Storage: Open SQLite database (`distsoc.db`), auto-migrate schema.
3. Blob store: Create `{data_dir}/blobs/` with 256 hex-prefix shards (`00/` through `ff/`).
4. UPnP mapping: Attempt UPnP/NAT-PMP port mapping (2s timeout). If successful, store external address for advertisements. Do not block startup if unavailable. See Section 11.
5. NAT type detection: STUN probes to two public servers (3s timeout each). Classifies as Public/Easy/Hard/Unknown. UPnP success overrides to Public. Anchors skip probing. Result stored on `ConnectionManager`, shared in `InitialExchangePayload`, stored per-peer. See Section 10.
6. Stale N2/N3 sweep: Remove all N2/N3 entries tagged to peers not in the current mesh. Clears stale reach data from previous sessions (e.g., unclean shutdown).
7. Bootstrap anchors: Load from `{data_dir}/anchors.json`. If missing, use hardcoded default anchor.
8. Bootstrap: If peers table is empty, connect to a bootstrap anchor. Request referrals and matchmaking (unless self or the other node is an anchor). Persist on that anchor's referral list until released (at referral count limit) while beginning the growth loop immediately.

### Startup cycles

Spawned after bootstrap completes:

Cycle | Interval | Purpose
--- | --- | ---
Pull sync | On demand (3h Self Last Encounter threshold) | Pull new posts from social + upstream file peers
Routing diff | 120s (2 min) | Broadcast N1/N2 changes to mesh + keep-alive sessions
Rebalance | 600s (10 min) | Clean dead connections, reconnect preferred, signal growth
Growth loop | 60s + reactive (on N2/N3 receipt) | Fill empty mesh slots until 101 (90% threshold for reactive mode)
Recovery loop | Reactive (mesh empty) | Emergency reconnect via anchors
Social/File connectivity check | 60s | Verify <N4 access to N+10 of active social + file peers; open keep-alive sessions as needed
UPnP lease renewal | 2700s (45 min) | Refresh UPnP port mapping before TTL expiry (desktop only)

Removed: Anchor register loop. Anchors are for forming initial mesh connections when bootstrapping, not for ongoing registration. Nodes only connect to anchors during bootstrap or recovery.

## 3. N+10 Identification

### Concept

Every node is identified not just by its NodeId but by its N+10: the node's own NodeId plus the NodeIds of its 10 preferred peers. This accelerates the capacity to find any node — if you can reach any of the 11 nodes in someone's N+10, you can find them.

### Where N+10 appears

Context | What's included
--- | ---
Self identification | All self-identification messages include the sender's N+10
Following someone | When you follow a peer, you store and maintain their N+10 in your social routes
Post headers | Every post header includes the author's current N+10. Updated whenever they post.
Blob headers | Blob/file headers include: (1) the author's N+10, (2) the upstream file source's N+10 (if not the author), (3) N+10s of up to 100 downstream file hosts
Recent post lists | Author manifests include the author's N+10 alongside their recent post list

### Why this works

Preferred peers are bilateral agreements — stable, long-lived connections. By including them in identification, any node that can find any of your 10 preferred peers can transitively find you within one hop. This eliminates most discovery cascades for socially-connected nodes.

### Status: Partial

N+10 is partially implemented — preferred peers exist and are tracked, but N+10 is not yet included in all identification contexts (post headers, blob headers, self-identification messages). Currently `preferred_tree` in social routes provides similar functionality for relay selection.

## 4. Connections & Growth

### Connection types

- Mesh connection — long-lived routing slot. Structural backbone for discovery and propagation. DB table: `mesh_peers`.
- Keep-alive session — long-lived connection for social or file layer peers that aren't in the mesh 101. Participates in N2/N3 routing. See Section 16.
- Session connection — short-lived, held open for active interaction (DM conversations, group activity, anchor matchmaking). Tracks `remote_addr` so the relay can inject observed addresses for session peers during introductions.
- Ephemeral connection — single request/response, no slot allocation.

### Slot architecture

Slot kind | Desktop | Mobile | Purpose
--- | --- | --- | ---
Preferred | 10 | 3 | Bilateral agreements, eviction-protected
Non-preferred | 91 | 12 | Growth loop fills these with diverse peers
Total mesh | 101 | 15 | Long-lived routing backbone
Keep-alive sessions | No hard limit | No hard limit | Social/file layer peers not in mesh (max 50% of session capacity reserved for keep-alive)
Sessions (interactive) | No hard limit | No hard limit | Active DM, group interaction, anchor matchmaking
Relay pipes | 10 | 2 | Own-device relay by default; opt-in for relaying for others

v0.2.0 change: Removed the distinction between "local" (71) and "wide" (20) non-preferred slots. The growth loop goes wide by default. Session counts are no longer hard-limited — an average computer can sustain ~1000 QUIC sessions without strain. The 50% keep-alive reservation ensures sessions remain available for interactive use.

### MeshConnection struct

Each mesh connection tracks: `node_id`, `connection` (QUIC), `slot_kind` (Preferred or NonPreferred), `remote_addr` (captured from Incoming before accept), `last_activity` (AtomicU64), `created_at`.

### Mutual mesh blacklist Planned

Targeted two-node stranger relationship. Both nodes opt in, maintaining genuine N2 stranger status indefinitely regardless of growth loop behavior. Stored in a local `mesh_blacklist { node_id }` table.

- Growth loop skips blacklisted nodes during candidate selection
- Incoming mesh upgrade from blacklisted node → respond with `RefuseRedirect` (`0x05`)
- Both nodes must add each other — asymmetric blacklist is valid but only prevents the blacklisting side from upgrading
- Blacklisted nodes remain visible in N2 via shared N1 peers
- Full session/ephemeral interaction still works — messages, probes, routing participation
- Never consume each other's mesh slots

Production utility: Operators maintaining intentional stranger relationships for network diversity, preventing specific nodes from becoming preferred peers, or any scenario where two nodes want to cooperate at session level without mesh entanglement.

### `--max-mesh <n>` CLI flag Planned

Topology control at network scale. Forces a node to cap its mesh connections, keeping it permanently in N2 of other nodes. Testing affordance only — not for production use.

- `--max-mesh 0`: Pure N2 participant, never takes mesh slots. Warning: free rider — consumes routing knowledge without carrying mesh load.
- `--max-mesh 3`: Partial mesh, useful for testing sparse topologies
- `--max-mesh 101`: Default, full normal behavior
- Node responds to all protocol messages normally, never initiates or accepts mesh upgrades beyond the cap
- Reuses existing `RefuseRedirect` (`0x05`) — no new protocol machinery

### Keepalive

- Interval: 30 seconds (`MeshKeepalive` message, `0xE0`)
- Zombie detection: No stream activity for 600s (10 min) = zombie, removed in rebalance
- `last_activity` updated on every stream accept

## 5. Connection Lifecycle

### 5.1 Growth Loop (60s timer + reactive on N2/N3 receipt)

Timer: Fires every 60 seconds. Checks current mesh count. If < 101, runs a growth cycle.

Reactive trigger: Fires immediately after receiving a peer's N2/N3 list (from initial exchange or routing diff). Continues firing on each new N2/N3 receipt until mesh is 90% full (~91 connections). After 90%, switches to timer-only mode.

Candidate selection (N2 diversity scoring):

    score = 1.0 / reporter_count + (0.3 if not_in_N3)

- Fewer reporters = higher diversity = better candidate
- Bonus for locally-discovered peers (not transitive)
- Sorted descending, best candidate tried first
- Growth loop goes wide by default — no local/wide distinction
- Blacklist filter: Skip nodes in `mesh_blacklist` table (see Section 4)

Connection attempt cascade:

1. Direct connect (15s timeout) — use stored/resolved address
2. Introduction fallback — find N2 reporters who know this peer, ask each to relay-introduce us

Failure handling: Track consecutive failures. After 3 consecutive failures, back off (break loop, wait for next signal). Mark unreachable peers for future skipping.

### 5.2 Rebalance Cycle (every 600s)

Executed in priority order:

1. Dead connection removal: Remove connections with `close_reason()` set, or idle > 600s (zombie)
2. Stale entry pruning: N2/N3 entries tagged to a peer that is no longer connected are pruned immediately (on disconnect and on startup sweep). Age-based fallback: entries older than 7 days. Social route watchers older than 30 days.
3. Priority 0 — Preferred peer reconnection: Iterate `preferred_peers` table, reconnect any that are disconnected. If at capacity, evict the lowest-diversity non-preferred peer to make room. Prune preferred peers unreachable for 7+ days (slot released, does NOT auto-return on reconnect — must re-negotiate via MeshPrefer). After 7 days, social checkin frequency drops from 1–4 hours to daily until the 30-day reconnect watcher expires.
4. Priority 1 — Reconnect recently dead: Re-establish dropped non-preferred connections. Skip blacklisted nodes — do not attempt reconnection to peers in `mesh_blacklist`.
5. Priority 2 — Signal growth loop: Fill remaining empty slots via growth loop
6. Idle session cleanup: Reap interactive sessions idle > 300s (5 min). Keep-alive sessions are NOT reaped by idle timeout.
7. Relay intro dedup pruning: Clear `seen_intros` entries older than 30s, cap at 500

Note: Low diversity score alone does NOT trigger eviction. The only eviction path is Priority 0 (making room for a preferred peer).

### 5.3 Recovery Loop (reactive, mesh empty)

Trigger: `disconnect_peer()` fires when last mesh connection drops.

1. Debounce 2 seconds (wait for cascading disconnects to settle)
2. Gather anchors: `known_anchors` table ordered by `last_seen DESC` (LIFO — most recently seen is most likely still reachable) → fallback to hardcoded default anchor(s) only if known_anchors empty or exhausted
3. For each anchor: connect, request referrals and matchmaking, try direct connect to each referral, fallback to hole punch via anchor for unreachable referrals
4. Persist on anchor's referral list until released, begin growth loop immediately
5. Post-bootstrap stale anchor cleanup: After successful bootstrap/recovery, probe `known_anchors` entries where `last_seen > 7 days`. Success: update `last_seen`. Failure: DELETE from `known_anchors`. Reuses existing anchor probe machinery (`0xC3`/`0xC4`). No new cycle or timer — runs as final step of bootstrap/recovery.

### 5.4 Initial Exchange (on every new connection)

When two nodes connect, they exchange:

- N+10: Our NodeId + 10 preferred peers' NodeIds
- N1 share: mesh peers + social contacts NodeIds (merged, no addresses)
- N2 share: deduplicated N2 NodeIds (no addresses)
- Profile: PublicProfile (display name, bio, avatar CID, `public_visible` flag)
- Delete records: Signed post deletions
- Post IDs: All local post IDs (for replica tracking)
- Peer addresses: N+10 address list for connected peers

Processing: Their N1 → our N2 table (tagged to reporter). Their N2 → our N3 table (tagged to reporter). Store profile, apply deletes, record replica overlaps. Trigger growth loop immediately with new N2/N3 candidates if mesh < 90% full.

### 5.5 Incremental Routing Diffs (every 120s + on change)

`NodeListUpdate` (`0x01`) contains N1 added/removed, N2 added/removed. Sent via uni-stream to all mesh peers and keep-alive sessions. Receiver processes: their N1 adds → our N2 adds, their N2 adds → our N3 adds, etc.

## 6. Network Knowledge Layers (N1/N2/N3)

Layer | Source | Contains | Shared? | Stored in
--- | --- | --- | --- | ---
N1 | Our connections + social contacts | NodeIds only | Yes (as "N1 share") | `mesh_peers` + `social_routes`
N2 | Peers' N1 shares | NodeIds tagged by reporter | Yes (as "N2 share") | `reachable_n2`
N3 | Peers' N2 shares | NodeIds tagged by reporter | Never | `reachable_n3`

### <N4 access

A node has <N4 access to a target if the target appears in its N1, N2, or N3 tables. This means the target is reachable within 3 hops without needing worm search or relay introduction. The social/file connectivity check (see Section 16) uses <N4 access to determine whether keep-alive sessions are needed.

### What is NEVER shared

- Addresses (resolved on-demand via chain queries)
- N3 entries (search-only, never forwarded)
- Duplication counts (topology leak)
- Which NodeIds are social contacts vs mesh peers (merged in N1 share)

### Address resolution cascade (`connect_by_node_id`)

Step | Method | Timeout | Source
--- | --- | --- | ---
0 | Social route cache | — | `social_routes` table (cached addresses for follows/audience)
1 | Peers table | — | Stored address from previous connection
2 | N2 ask reporter | varies | Ask the mesh peer who reported target in their N1
3 | N3 chain resolve | varies | Ask reporter's reporter (2-hop chain)
4 | Worm search | 3s total | Burst to all peers → nova to N2 referrals (each does own burst)
5 | Relay introduction | 15s | Hole punch via intermediary relay
6 | Session relay | — | Pipe traffic through intermediary (own-device or opt-in)

## 7. Three-Layer Architecture (Mesh / Social / File)

The network operates across three distinct layers, each with its own connections, routing, and purpose. The separation enables specialized behavior without the layers interfering with each other.

Layer | Purpose | Connections | Sync trigger
--- | --- | --- | ---
Mesh | Structural backbone: N1/N2/N3 routing, diversity, discovery | 101 mesh slots (preferred + non-preferred) | N/A — mesh is infrastructure, not content
Social | Follows, audience, DMs — the human relationships | Social routes + keep-alive sessions as needed | Pull posts when Self Last Encounter > 4 hours
File | Content storage and distribution — blobs, CDN trees | Upstream/downstream file peers + keep-alive sessions as needed | Pull on blob request, push on post creation

### Key principle: mesh is not for content

Pull sync does not pull posts from mesh peers. Mesh connections exist for routing diversity and discovery. Content flows through the social layer (posts from people you follow) and the file layer (blobs from upstream/downstream hosts). This separation means mesh connections can be optimized purely for network topology without social bias.

### Cross-layer benefits

Each layer's connections contribute to finding nodes and referrals for the other layers. Keep-alive sessions from the social and file layers participate in N2/N3 routing, which improves <N4 access for all three layers. A social keep-alive session might provide the N2 entry that helps the mesh growth loop find a diverse new peer, and vice versa.

## 8. Anchors

### Intent

Anchors are "just peers that are directly reachable" — standard ItsGoin nodes with a routable address. They run the same code with no special protocol. Their value comes from being directly connectable for bootstrapping new nodes into the network and matchmaking (introducing peers to each other). Anchors include VPS-deployed nodes (always-on) and desktop nodes with UPnP port mappings (see Section 11).

Each profile can carry a preferred anchor list — infrastructure addresses, not social signals.

### Status: Complete (with gaps)

### When anchors are used

- Bootstrap: First startup with empty peers table. Connect to anchor, request referrals and matchmaking, persist on referral list while growing mesh.
- Recovery: When mesh drops to 0 connections. Same flow as bootstrap.
- Not ongoing: Nodes do NOT register with anchors on a loop. Anchors are for forming initial connections, not for ongoing presence.
- itsgoin.net node: A permanent, well-connected ItsGoin node runs on itsgoin.net as part of the share link redirect infrastructure (see Section 26). This node participates in the network as a standard anchor — it bootstraps new nodes, accepts referral requests, and is included in `known_anchors` by peers that connect through it. It is not special-cased in the protocol. Its value as an anchor comes from permanent uptime and high mesh connectivity, not from any privileged role.

### Anchor referral mechanics

When a bootstrapping node connects, the anchor provides referrals from its mesh and referral list. The node persists on the anchor's referral list until released at the referral count limit. During this time, the anchor can matchmake — introducing the new node to other peers requesting referrals.

### Anchor selection order

1. `known_anchors` table — `ORDER BY last_seen DESC` (LIFO). The most recently seen anchor is most likely still reachable, particularly given short-lived home desktop anchors.
2. Hardcoded default anchor(s) — only if `known_anchors` is empty or exhausted. A brand-new node hits hardcoded anchors once on first bootstrap, populates `known_anchors` from that session, and the hardcoded list recedes to pure fallback.

No scoring, no success counting, no prediction. Attempt, move to next on failure. The `known_anchors` table stores only: `node_id`, `addresses`, `last_seen`.

### Anchor self-verification Complete

Nodes with UPnP-mapped IPv4 or IPv6 public addresses cannot self-certify as anchors — they need external verification that they are genuinely reachable by cold direct connect. A node is a viable anchor only if a complete stranger can connect to it directly with no introduction, no hole punch, and no relay.

#### Witness selection

Node A (candidate anchor) selects a witness from its own N2 table entries NOT present in its N1. These are genuine strangers — no prior connection, no cached address, no warm path. A selects one (call it C) and knows C's address via the N1 reporter (call it B) who reported C in their N1 share.

#### Probe message flow

    A → B (N1 reporter of C): AnchorProbeRequest {
        target_addr,     // A's external address to test
        witness,         // C's NodeId
        return_via,      // B's NodeId (for failure reporting)
    }

    B → C: forward AnchorProbeRequest

    C: cold direct QUIC connect to target_addr
       — MUST use only raw QUIC connect (step 1 of connect_by_node_id)
       — MUST skip entire resolution cascade, hole punch, introduction, relay
       — 15s timeout

    SUCCESS: C → A directly (on new connection): AnchorProbeResult { reachable: true }
    FAILURE: C → B → A: AnchorProbeResult { reachable: false }

Asymmetric return path: If cold connect fails, by definition there is no direct path from C to A. C reports failure through B (who has a live connection to A). On success, C has a fresh direct connection and uses it. The `return_via` field tells C which node to route failure through.

Why bypass the cascade: The normal `connect_by_node_id` cascade has 7 steps including hole punch and relay. If C uses the full cascade, a successful result via relay is a false positive. The probe handler must be a special code path: raw QUIC connect only.

#### Anchor candidacy checklist

    is_anchor_candidate():
      - has UPnP mapping OR has IPv6 public address
      - probe succeeded within last 30 minutes
      - mesh ≥ 50 peers (sufficient N2 density)
      - uptime ≥ 2 hours continuous
      - NOT mobile (platform check at build time)

#### Probe refresh schedule

Trigger | Action
--- | ---
Startup (after UPnP attempt) | Run initial probe
UPnP renewal if address changed | Re-probe
Every 30 minutes while anchor-declared | Periodic re-probe
Any failed inbound connection | Immediate re-probe
Two consecutive probe failures | Stop advertising as anchor, revert to normal peer

### Session fallback for full anchors

When an anchor's mesh is full (101/101), new nodes fall back to a session connection for matchmaking. The anchor accepts referral requests over session connections, not just mesh.

### Remaining gaps

Gap | Impact
--- | ---
Profile anchor lists not used for discovery | Profiles have an `anchors` field but it's not consulted during address resolution
No anchor-to-anchor awareness | Anchors don't discover each other unless they connect through normal mesh growth
Bootstrap chicken-and-egg | A fresh anchor with few peers produces few N2 candidates for new nodes. Growth stalls because there's nothing to grow from.

## 9. Referrals

### Status: Complete

### Referral list mechanics (anchor side)

Anchors maintain an in-memory HashMap of registered peers. Each entry: `{ node_id, addresses, use_count, disconnected_at }`.

Property | Value
--- | ---
Tiered usage caps | 3 uses if list < 50, 2 uses at 50+, 1 use at 100+
Disconnect grace | 2 minutes before pruning
Sort order | Least-used first (distributes load)
Auto-supplement | When explicit list is sparse (< 3 entries), supplement with random mesh peers

## 10. Relay & NAT Traversal

### Status: Complete

### Relay selection (`find_relays_for`)

Find up to 3 relay candidates, prioritized:

1. Preferred tree intersection: Target's `preferred_tree` (from `social_routes`, ~100 NodeIds) intersected with our connections. Prefer our own preferred peers within that tree. TTL=0.
2. N2 reporters: Our mesh peers who reported the target in their N1 share. TTL=0.
3. N3 via preferred tree: Target's `preferred_tree` intersected with N3 reporters. TTL=1.
4. N3 reporters: Any N3 reporter for the target. TTL=1.

### RelayIntroduce flow (`0xB0`/`0xB1`)

1. Requester → opens bi-stream to relay, sends `RelayIntroduce { target, requester, requester_addresses, ttl }`
2. Relay handles three cases:
   - We ARE the target: Return our addresses, spawn hole punch to requester
   - Target is our mesh or session peer: Forward request to target on new bi-stream, relay response back. Inject observed public addresses for both parties (session peers carry `remote_addr` from their inbound connection).
   - TTL > 0 and target in our N2: Forward to the reporter with TTL-1 (chain forwarding, max TTL=2)
3. Requester receives `RelayIntroduceResult { target_addresses, relay_available }`, then:
   - `hole_punch_parallel()`: Try all returned addresses in parallel, retry every 2s, 30s total timeout
   - If hole punch fails and `relay_available`: open `SessionRelay` (`0xB2`) pipe through the intermediary

### Session relay (relay pipes)

Intermediary splices bi-streams between requester and target. Desktop: max 10 concurrent pipes. Mobile: max 2. Each pipe has a 50MB byte cap and 2-min idle timeout.

v0.2.0 change: Relay pipes are own-device-only by default. A node will only relay traffic between its own devices (same identity key, different device identity). Users can opt in to relaying for others in Settings, but this is not enabled automatically. This prevents nodes from unknowingly burning bandwidth for random peers while still enabling personal multi-device routing.

### Deduplication & cooldowns

Mechanism | Window | Purpose
--- | --- | ---
`seen_intros` | 30s | Prevents forwarding loops
`relay_cooldowns` | 5 min per target | Prevents relay spamming

### Hole punch mechanics

Both sides filter self-reported addresses to publicly-routable only (no Docker bridge, VPN, or LAN IPs) and prepend UPnP external address if available. The relay injects each party's observed public address (from the QUIC connection) at the front of the list. All paths use `hole_punch_parallel()`: parse returned addresses into QUIC `EndpointAddr`, spawn parallel connect attempts to every address simultaneously. Each attempt: 2s timeout, retried until 30s total deadline. First successful connection wins.

### NAT type detection

### Status: Complete (interim: public STUN servers)

On startup, each node classifies its NAT type as one of four categories:

- Public — observed address matches local, or UPnP-mapped. Directly reachable.
- Easy — same mapped port from multiple probes (endpoint-independent mapping / cone NAT). Hole punch will likely succeed.
- Hard — different mapped ports per destination (symmetric / address-dependent mapping). Port is unpredictable.
- Unknown — detection failed or not yet run.

#### Current implementation (interim)

Raw STUN Binding Requests (20 bytes, no crate dependency) sent to `stun.l.google.com:19302` and `stun.cloudflare.com:3478` from a single UDP socket. XOR-MAPPED-ADDRESS parsed from each response (IPv4 + IPv6 supported). Comparison: same mapped port = Easy, different = Hard, matches local = Public. 3s timeout per server. UPnP success overrides to Public. Anchors skip probing entirely (already Public).

#### Target design (multi-anchor STUN)

When the network has enough anchors, replace public STUN servers with anchor-reported `your_observed_addr` from InitialExchange. Connecting to two or more anchors at different public IPs provides the same classification without external dependencies.

#### NAT type sharing

NAT type is included as a string field (`"public"`/`"easy"`/`"hard"`/`"unknown"`) in `InitialExchangePayload`. Stored per-peer in the `peers` table (`nat_type TEXT` column). Available for hole punch decisions before any connection attempt.

#### Hole punch strategy

Peer A | Peer B | Strategy
--- | --- | ---
Public / Easy | Any | Hole punch (likely success)
Hard NAT | Easy NAT | Hole punch (B's port is predictable)
Hard NAT | Hard NAT | Port scanning — `hole_punch_with_scanning()` tries standard punch first, then escalates to tiered port scanning (±500, ±2000, full ephemeral range)

All hole punch paths use `hole_punch_with_scanning()` which replaces the former hard+hard skip. NAT profiles (NatMapping + NatFiltering) from InitialExchange determine whether scanning is attempted. Behavioral inference updates filtering classification from connection outcomes.

### Advanced NAT traversal

### Status: Complete

NAT "hardness" has two independent dimensions:

- Mapping: Endpoint-Independent Mapping (EIM / "easy") uses the same external port for all destinations. Endpoint-Dependent Mapping (EDM / "hard") assigns a different port per destination.
- Filtering: Address-restricted (Open) accepts from any port on an IP the host has sent to. Port-restricted accepts only from the exact IP:port the host has sent to.

STUN probing at startup classifies mapping (EIM/EDM). Filtering is determined reliably via the anchor filter probe.

#### NAT filter probe (0xC6/0xC7)

After anchor registration, each node with Unknown filtering sends a `NatFilterProbe` bi-stream request to its anchor. The anchor creates a temporary QUIC endpoint on a random port and attempts to connect to the node’s observed address (2s timeout). If the connection succeeds, the node is Open (address-restricted or better — accepts packets from any port on the anchor’s IP). If it times out, the node is PortRestricted.

This probe runs once at startup (during anchor register cycle) and the result feeds into all subsequent InitialExchange payloads, so peers know each other’s exact filtering type.

Note: “Public” NAT type does not automatically mean Open filtering. A node may be public on IPv6 but NATed on IPv4. The filter probe tests actual reachability from a different port, regardless of self-declared NAT type. Startup logs now report `public (v4 only)`, `public (v6 only)`, or `public (v4+v6)`.

#### NAT combination matrix

Side A | Side B | Result
--- | --- | ---
addr-restricted, EIM | addr-restricted, EDM | Basic hole punch
port-restricted, EIM | addr-restricted, EDM | A scans to find+open port; B punches A’s stable port regularly
addr-restricted, EDM | port-restricted, EDM | B scans to find+open port; A waits then responds
port-restricted, EDM | port-restricted, EDM | Both scan+punch alternately
addr-restricted, EIM | addr-restricted, EIM | Basic hole punch
port-restricted, EIM | addr-restricted, EIM | Basic hole punch
addr-restricted, EDM | port-restricted, EIM | B scans to find+open port; A punches B’s stable port regularly
port-restricted, EDM | port-restricted, EIM | B scans to find+open port; A punches B’s stable port regularly

Key insight: if both sides have Open (address-restricted) filtering, scanning is never needed — `should_try_scanning()` returns false and basic hole punch handles it.

#### Role-based scanning protocol

Each side independently determines its role based on its own NAT profile:

- EIM (stable port) → Puncher: Punch peer’s anchor-observed address every 2s. Our port is stable — the peer’s scanner will find us.
- EDM or Unknown → Scanner+Puncher: Walk outward from peer’s anchor-observed base port at ~100 ports/sec (base, base+1, base-1, base+2, base-2, ...). Each probe opens a firewall entry on our NAT. Also punch every 2s to check if peer has opened their port for us.

The scanner opens ports on its own firewall. The other side’s periodic punch (one every 2s to the scanner’s observed address) checks if the scanner has opened a port matching the puncher’s actual port. For both-EDM pairs, both sides scan and punch simultaneously.

#### Scan parameters

- Rate: ~100 ports/sec (one probe every 10ms)
- In-flight: ~20 concurrent (100/sec × 200ms connect timeout)
- Direction: Outward walk from anchor-observed base port
- Target address: Anchor-observed (relay-injected) address only — not VPN/cellular/LAN addresses
- Max duration: 5 minutes (covers full 65K port space at ~100/sec in ~11 minutes; ±2000 range covered in first 40 seconds)
- Task management: JoinSet with abort_all() on success or exhaustion — no orphaned tasks
- Punch interval: Every 2s to peer’s anchor-observed address

#### Why 5-minute scan duration is acceptable

The cost is time, not resources (~20 in-flight at any time, ~100 probes/sec). For connections that would otherwise be impossible (both EDM + port-restricted), accepting a longer setup time is far better than giving up entirely. Most successful connections resolve within the first 40 seconds (±2000 port range).

Design principle: This protocol eliminates the need for full relay in virtually all NAT scenarios. Session relay remains opt-in only — it is never used as an automatic fallback. The scanning approach respects the user’s intent that peers communicate directly whenever physically possible.

## 11. UPnP Port Mapping

### Status: Complete

### Purpose

UPnP (Universal Plug and Play) allows a node to request its home router to forward an external port to its local QUIC port. This makes the node directly reachable from the internet without hole punching — any peer with the external address can connect immediately. This dramatically improves connection success rates for desktop nodes on home networks.

### Startup flow

    bind Endpoint → attempt UPnP mapping (2s timeout) → store external addr → bootstrap

1. Discover gateway: Search for UPnP/NAT-PMP gateway with a 2-second timeout. If no gateway found, proceed without — do not block startup.
2. Request mapping: Map both UDP and TCP for the local QUIC port to the same external port (or next available). UDP is required for QUIC (existing). TCP enables HTTP post delivery (see Section 25). Both use the same external port number. If the router supports one but not the other, accept the partial mapping gracefully — QUIC connectivity is not affected by TCP mapping failure. Request lease TTL of 3600s.
3. Store external address: The resulting external `SocketAddr` is stored alongside iroh's observed addresses. It feeds into N+10 identification, InitialExchange, anchor registration, and all peer address advertisements.
4. Log result: Clearly log whether UPnP succeeded, failed, or was unavailable. This is critical for diagnosing connectivity issues.

### Lease renewal cycle (every 2700s / 45 min)

UPnP mappings have a TTL (typically 3600s but varies by router). A renewal loop runs every 45 minutes to refresh the mapping before it expires. If renewal fails, the external address is removed from advertisements and the node falls back to hole punch / relay paths gracefully.

### Shutdown

Explicitly release the UPnP mapping on clean shutdown. Routers have finite mapping tables — releasing is good citizenship. Tauri's shutdown hook handles this.

### Integration with existing address logic

The UPnP external address is treated the same as any other address the node knows about. It feeds into:

- N+10 identification: Included in self-identification so peers store a routable address
- InitialExchange: Advertised to new connections
- Anchor registration: Included in bootstrap/recovery registration
- Social routing: Available in social route address cache for follows/audience
- Relay introduction results: Returned alongside hole-punch candidate addresses
- Share link host lists: The UPnP external address, when mapped for TCP, determines whether this node includes itself in share link host lists (see Section 26). A node only self-includes if it has confirmed TCP reachability — either via UPnP TCP mapping or a known public IPv6 address.

### Why this matters for mobile

Mobile devices on cellular networks cannot use UPnP (carrier NAT doesn't expose it). However, if the peers they're trying to reach (especially desktop nodes and anchors) have UPnP mappings, those peers become directly reachable from the phone without hole punching. The phone doesn't need UPnP — the other side does.

### Honest limitations

Limitation | Impact
--- | ---
UPnP disabled on router | Some ISPs ship routers with UPnP off. Mapping silently fails, fallback to hole punch.
Double NAT | ISP modem + user router: mapping reaches inner router but not outer. Partial help at best.
Cellular networks | No UPnP at all. This is purely a desktop/home-network feature.
Carrier-grade NAT (CGNAT) | ISP shares one public IP across many customers. UPnP maps to the ISP's NAT, not the internet. Same as double NAT.

Design principle: UPnP is a best-effort enhancement that improves direct connection reliability for the common case. It is not a dependency. The hole punch + relay fallback chain already handles all failure cases — UPnP just reduces how often you fall back to them.

### UPnP nodes are anchors

A node with a successful UPnP mapping is directly reachable from the internet — which is the only thing that makes an anchor an anchor. When UPnP mapping succeeds, the node self-declares as an anchor (`is_anchor = true`). Other peers will add it to their `known_anchors` table, providing diverse bootstrap paths back into the network.

When the UPnP mapping is lost (lease renewal fails, shutdown), the node reverts to non-anchor. Peers that stored it as an anchor will naturally age it out via `last_seen` — LIFO ordering means stale anchors drop to the bottom. The 7-day post-bootstrap cleanup probes stale entries and removes failures. No special cleanup needed beyond the existing anchor infrastructure.

This means any desktop on a home network with UPnP-capable router becomes a potential bootstrap point for the network, dramatically increasing the number of available anchors without any manual server deployment.

### Implementation

Crate: `igd-next` (async support, well-maintained fork of `igd`). Implementation lives in `network.rs` alongside the iroh Endpoint — UPnP mapping is an Endpoint concern, not a connection concern.

## 12. LAN Discovery

### Status: Complete

mDNS-based LAN discovery is integrated via iroh's built-in `MdnsAddressLookupBuilder`. It works automatically — peers on the same local network are discovered and connected without manual configuration. iroh's mDNS address lookup broadcasts peer presence on the local network via multicast DNS (service name `"irohv1"`, backed by the `swarm-discovery` crate).

### Discovery flow

1. Hold the mDNS handle: Build `MdnsAddressLookup` explicitly (not via the endpoint builder) so we retain a clone for subscribing.
2. Spawn a LAN scan loop: Call `mdns.subscribe().await` to get a stream of `DiscoveryEvent::Discovered` and `DiscoveryEvent::Expired` events.
3. On discovery: Extract NodeId + LAN addresses from the event. If not already connected, initiate a direct connection + initial exchange. Register as a LAN session (a keep-alive session tagged as local).
4. On expiry: Clean up the LAN session. Peer left the network or powered off.

### LAN sessions

LAN peers are special: zero-cost bandwidth, sub-millisecond latency, and very likely someone you know (same household/office). They deserve their own treatment beyond regular mesh or session slots:

- Automatic keep-alive: LAN sessions stay open as long as the peer is on the network (mDNS heartbeat). No idle timeout. Not counted against session slot limits.
- Sync priority: Pull sync and push notifications go to LAN peers first — instant delivery over the local link.
- Local relay: LAN peers can relay for each other to the wider internet. A phone behind carrier NAT can relay through the desktop's UPnP-mapped connection. Bandwidth is free (local network), so relay limits can be much more generous than over the internet.
- Blob transfer: Large blob transfers between LAN peers are essentially free. Prefer LAN peers as blob sources when available.

### Design rationale

Today, two distsoc devices on the same WiFi network can only find each other if they happen to share a peer that reports them in N2. This is absurd — they're on the same network segment. LAN discovery turns mDNS from a passive address resolver into an active peer source, exploiting the fact that local bandwidth is essentially unlimited.

The keep-alive + relay pattern means a household with one well-connected desktop and several phones creates its own mini-mesh: the desktop provides anchor-like connectivity, the phones stay connected through it, and everyone syncs instantly over the LAN even when the internet connection drops.

Implementation note: iroh's `MdnsAddressLookup::subscribe()` returns a `Stream<DiscoveryEvent>`. The `DiscoveryEvent::Discovered` variant includes `EndpointInfo` with NodeId + IP addresses. Custom `user_data` can be set via `endpoint.set_user_data_for_address_lookup()` to embed distsoc-specific metadata (e.g., display name) in the mDNS TXT record.

## 13. Worm Search

### Status: Complete

Used at step 4 of `connect_by_node_id`, after N2/N3 resolution fails.

### Algorithm

1. Build needles: target NodeId + target's N+10 (up to 10 preferred peers from their profile/cached N+10)
2. Local check: Search own connections + N2/N3 for any of the 11 needles. Also check local storage, CDN downstream tree, and blob store for any requested post/blob content.
3. Burst (500ms timeout): Send `WormQuery{ttl=0}` (`0x60`) to all mesh peers in parallel. Each peer checks their local connections + N2/N3, plus local storage and CDN tree for post/blob content.
4. Nova (1.5s timeout): Each burst response includes a random "wide referral" — an N2 peer. Connect to those referrals and send `WormQuery{ttl=1}`. The referred peer does its own 101-burst (fans out to all its mesh peers with ttl=0). This reaches ~10K nodes with only ~202 relay hops, keeping network pressure low by expanding one hop at a time rather than flooding.
5. Total timeout: 3 seconds for the entire search.

### Content search

`WormQuery` carries optional `post_id` and `blob_id` fields, enabling unified search for nodes, posts, and blobs in a single query. Each peer checks:

- Posts: local storage (direct match), CDN downstream tree (`post_downstream` — up to 100 known hosts per post)
- Blobs: local blob store, CDN post ownership (`get_blob_post_id` → `post_downstream`)

`WormResponse` carries `post_holder` and `blob_holder` fields alongside the existing node search results. A content hit (post or blob holder found) is treated as a successful response even without a node match.

The CDN layer is the key multiplier: each node's downstream tree can cover hundreds of posts across dozens of hosts, giving every peer thousands of "I know where that is" answers. Combined with social layer knowledge, even a 202-hop nova covers enormous content space.

### PostFetch (`0xD4`/`0xD5`)

Lightweight single-post retrieval after worm search identifies a holder. Opens a bi-stream to the holder and requests one post by ID. Much lighter than full `PullSync` — no follow filtering, no batch processing, just the target post.

### Dedup & cooldown

Mechanism | Window | Purpose
--- | --- | ---
`seen_worms` | 10s | Prevents loops during fan-out
Miss cooldown | 5 min (in DB) | Prevents repeated searches for unreachable targets

## 14. Preferred Peers

### Status: Complete

### Negotiation (`MeshPrefer`, `0xB3`)

- Bilateral: Requester sends `MeshPrefer{requesting: true}`, responder accepts/rejects
- Acceptance: Both sides persist to `preferred_peers` table, upgrade slot to `PeerSlotKind::Preferred`
- Rejection reasons: "not connected", "preferred slots full (N/M)"

### Properties

- Eviction-protected: Never evicted during rebalance (only non-preferred peers can be evicted)
- Priority reconnect: Reconnected first in rebalance (Priority 0), before any growth
- Pruned after 7 days unreachable: If a preferred peer can't be reached for 7 days, the slot is released. The bilateral agreement is cleared — reconnection requires a new MeshPrefer handshake. A reconnect watcher persists for 30 days at low priority (daily check). This prevents churn from aggressive pruning while ensuring slots aren't held indefinitely for offline peers.
- N+10 component: Your 10 preferred peers' NodeIds are included in your N+10 for all identification (see Section 3)
- Preferred tree: Each social route caches a `preferred_tree` (~100 NodeIds) — the target's preferred peers' preferred peers. Used for relay selection.

## 15. Social Routing

### Status: Complete

Caches addresses for follows and audience members, separate from mesh connections.

### `social_routes` table

Field | Purpose
--- | ---
`node_id` | The social contact's NodeId
`nplus10` | Their N+10 (NodeId + 10 preferred peers)
`addresses` | Their known IP addresses
`peer_addresses` | Their N+10 contacts (PeerWithAddress list)
`relation` | Follow / Audience / Mutual
`status` | Online / Disconnected
`last_connected_ms` | When we last connected
`reach_method` | Direct / Relay / Indirect
`preferred_tree` | ~100 NodeIds for relay tree

### Wire messages

Code | Name | Stream | Purpose
--- | --- | --- | ---
`0x70` | SocialAddressUpdate | Uni | Sent when a social contact's address changes or they reconnect
`0x71` | SocialDisconnectNotice | Uni | Sent when a social contact disconnects
`0x72` | SocialCheckin | Bi | Keepalive with address + N+10 updates

### Reconnect watchers

`reconnect_watchers` table: when peer A asks about disconnected peer B, A is registered as a watcher. When B reconnects, A gets a `SocialAddressUpdate` notification. Watchers pruned after 30 days. Low priority — daily check frequency for watchers older than 7 days.

### Social route lifecycle

- Follow → store their N+10, upgrade to Mutual (if audience)
- Unfollow → downgrade/remove
- Approve audience → Mutual/Audience

## 16. Keep-Alive Sessions

### Status: Planned

### Purpose

When the mesh 101 doesn't provide <N4 access to all the nodes we need for social and file operations, keep-alive sessions bridge the gap. These are long-lived connections that participate in N2/N3 routing but are not part of the mesh 101.

### Social/File connectivity check (every 60s)

Periodically check whether we can reach every node we need. A node is considered reachable if either:

- We have <N4 access to their N+10 (within N1/N2/N3), or
- There is an anchor within N2 of them — we can ask that anchor to matchmake on demand without maintaining a persistent connection

Only when neither condition is met do we open a keep-alive session. With UPnP auto-anchors (see Section 11) scattered throughout the network, the odds of an anchor being within N2 of any given peer increase significantly, reducing the number of keep-alive sessions needed.

Nodes to check:

- Nodes we DM'd in the last 4 hours
- All follows
- All audience members
- All file upstream peers (for blobs we host)
- All file downstream peers (for blobs we serve)

For any node whose N+10 is NOT reachable within N3, open a keep-alive session to the closest available node in their N+10 (or to them directly if possible). This ensures we can always find and reach our social and file contacts without worm search.

### Keep-alive session behavior

- N2/N3 routing: Keep-alive sessions exchange N1/N2 diffs and participate in routing, similar to mesh connections. They expand our network knowledge without consuming mesh slots.
- Not counted in mesh 101: Keep-alive sessions are a separate pool. They don't affect mesh diversity scoring or slot management.
- Capacity limit: Max 50% of total session capacity is reserved for keep-alive sessions. The other 50% remains available for interactive sessions (DMs, group activity).
- Not idle-reaped: Unlike interactive sessions (5-min idle timeout), keep-alive sessions persist as long as the connectivity need exists.
- Reevaluated periodically: The 60s connectivity check closes keep-alive sessions that are no longer needed (e.g., the target now appears in N3 via a mesh connection).

### Practical ceilings

Platform | Ceiling | Binding constraint
--- | --- | ---
Desktop | ~300–500 | Routing diff broadcast overhead — `NodeListUpdate` to all sessions every 120s. Memory and connection count are not the bottleneck.
Mobile | ~25–50 | Battery (radio wake-ups per heartbeat cycle) and OS background restrictions (iOS/Android will kill background sockets).

### Mobile priority stack

When approaching the mobile ceiling, keep-alive sessions are prioritized:

1. DMs last 30 min — active conversations take highest priority
2. Follows — people you follow
3. Audience — people following you
4. File peers — upstream/downstream blob hosts

Lower-priority sessions are closed first to make room.

### Hysteresis

Don't open a keep-alive session for a contact who just barely fell outside N3. Wait for persistent unreachability — the contact must be absent from N1/N2/N3 for multiple consecutive connectivity checks (e.g., 3 checks = 3 minutes) before opening a keep-alive. This prevents churn from nodes that transiently appear and disappear at the N3 boundary.

### Reject + redirect

When a node is at its keep-alive session capacity (50% of total sessions), it refuses new keep-alive requests with a redirect — offering a random N2 node that also has <N4 access to the target. Same pattern as mesh `RefuseRedirect` but for the keep-alive pool. The requester tries the suggested peer instead.

### Cross-layer benefit

Keep-alive sessions from the social and file layers feed N2/N3 entries back into the mesh layer. A social keep-alive to a friend's preferred peer might provide N2 entries that help the mesh growth loop. Similarly, a file keep-alive to an upstream host might provide access to nodes the mesh has never seen. The three layers compound each other's reach.

## 17. Content Propagation

### Intent

"Attention creates propagation": when you view something, you cache it. The cache is optionally offered for serving. Hot content spreads naturally through demand. Cold content decays unless intentionally hosted.

The CDN vision: every file by author X carries an author manifest with the author's N+10 and recent post list. If you hold any file by author X, you passively know X's recent posts and can find X through their N+10.

### Status: Partial

- `BlobRequest`/`BlobResponse` (`0x90`/`0x91`) for peer-to-peer blob fetch
- AuthorManifest (ed25519-signed, 10+10 post neighborhood) travels with blob responses
- CDN hosting tree (1 upstream + 100 downstream per blob)
- ManifestPush propagates updates down the tree
- BlobDeleteNotice for tree healing on eviction
- Blob eviction with social-aware priority scoring

### Passive discovery via neighborhood diffs

Passive file-chain propagation is enabled through BlobHeader neighborhood diffs. Every blob header carries the author's 25+25 post neighborhood (25 previous + 25 following). When a host receives a `BlobHeaderDiff` (`0x96`), they learn about the author's newer posts without explicit subscription. Hosts of old content are pulled toward new content by the same author naturally — attention creates propagation.

### Remaining gaps

Gap | Impact
--- | ---
N+10 not yet in file headers | Blob headers should include author N+10, upstream N+10, and downstream N+10s. Currently only AuthorManifest travels with blobs.
No "fetch from any peer who has it" | Blobs are fetched from specific peers. No content-addressed routing ("who has blob X?").

## 18. Files & Storage

### Blob storage Complete

Property | Value
--- | ---
CID format | BLAKE3 hash of blob data (32 bytes, hex-encoded)
Filesystem path | `{data_dir}/blobs/{hex[0..2]}/{hex}` (256 shards)
Metadata table | `blobs` (cid, post_id, author, size_bytes, created_at, last_accessed_at, pinned)
Max blob size | 10 MB
Max attachments per post | 4

### Blob content immutability

Blob data is BLAKE3-addressed — the CID is the hash of the content. This means blob content is immutable by definition. Any mutable metadata (neighborhood, host lists, signatures) MUST be stored separately in a BlobHeader. Inline mutable headers are architecturally incompatible with content addressing.

### BlobHeader Complete

Mutable structure stored and transmitted separately from blob data. Carries engagement state, CDN metadata, and encrypted slots for private posts.

    BlobHeader {
        post_id,                // PostId this header belongs to
        author,                 // Author NodeId
        reactions,              // Vec of public reactions (emoji + reactor NodeId + timestamp)
        comments,               // Vec of comments (text + author + timestamp + signature)
        policy,                 // Author-controlled comment/react policy
        updated_at,             // Timestamp of last header update
        thread_splits,          // Linked thread posts when comments exceed 16KB
        receipt_slots,          // Encrypted delivery/read/react receipt slots (private posts)
        comment_slots,          // Encrypted comment slots (private posts)
    }

- Post neighborhood: 25 previous + 25 following PostIds. Forward slots are empty at publish time and populate via `BlobHeaderDiff` propagation as the author continues posting. Empty forward slots are not an error condition.
- Downstream host count: `min(100, floor(170MB / blob_size_bytes))` — smaller blobs allow more downstream hosts, larger blobs reduce the count to cap per-host storage overhead.
- BlobHeaderRequest: Lightweight header-only fetch — retrieve just the header without retransferring blob data. Useful for neighborhood updates and host discovery.
- Self Last Encounter: Stored per-author, becomes the newer of what's stored and "file last update." Determines when to trigger pull sync.

### Blob transfer flow (`0x90`/`0x91`)

1. Requester sends `BlobRequest { cid, requester_addresses }`
2. Host checks local BlobStore:
   - Has blob: Return base64-encoded data + CDN manifest + file header (N+10s, recent posts). Try to register requester as downstream (max 100). If full, return existing downstream as redirect candidates.
   - No blob: Return `found: false`
3. Requester verifies CID, stores blob locally, records upstream in `blob_upstream` table. Updates Self Last Encounter for the author based on file header.

### CDN hosting tree Complete

- AuthorManifest: ed25519-signed by post author, contains post neighborhood (25 previous + 25 following posts — see BlobHeader above), author N+10, author addresses
- CdnManifest: AuthorManifest + hosting metadata (host NodeId/addresses, source, downstream count)
- Tree structure: Each blob has 1 upstream source + up to 100 downstream hosts
- ManifestPush (`0x94`): Author/admin pushes updated manifests downstream, which relay to their downstream
- ManifestRefreshRequest/Response (`0x92`/`0x93`): Check if manifest has been updated since last fetch
- BlobDeleteNotice (`0x95`): Notify tree when blob is deleted; includes upstream info for tree healing

### Blob eviction Complete

    priority = pin_boost + share_boost + (relationship * heart_recency * freshness / (peer_copies + 1))

Factor | Calculation
--- | ---
`pin_boost` | 1000.0 if pinned, else 0.0. Own blobs auto-pinned.
`relationship` | 5.0 (us), 3.0 (mutual follow+audience), 2.0 (follow), 1.0 (audience), 0.1 (stranger)
`heart_recency` | Linear decay over 30 days: `max(0, 1 - age/30d)`
`freshness` | `1 / (1 + post_age_days)`
`peer_copies` | Known replica count (from `post_replicas`, only if < 1 hour old)
`share_boost` | +100.0 if 3+ downstream peers (shared link with healthy distribution), scaled linearly for 1–2 downstream peers (33.3 per peer). Keeps shared content cached longer.

### Pin modes Planned

The CDN is delivery infrastructure, not storage. Authors own durability. Pinning extends content in the local delivery pool — it is not a network obligation.

Concept | Status
--- | ---
Anchor pin vs Fork pin | Not started. Anchor pin = host the original (author retains control). Fork pin = independent copy (you become key owner).
Personal vault | Not started. Private durability for saved/pinned items.

## 18b. Erasure-Coded CDN Replication Planned

### Problem

The existing CDN hosting tree (Section 18) replicates full blob copies to downstream peers. This works well when the replicating node chose to pull the content — a follow relationship or explicit action establishes user consent. But the `ReplicationRequest` (0xE1) protocol also pushes content to infrastructure nodes that never chose to host it. A node holding a full copy of content it never reviewed faces potential liability for that content.

Encryption does not solve this for public posts: the content is plaintext by definition. A different mechanism is needed that makes it technically impossible for a CDN node to possess reconstructable content.

### Approach: sub-threshold erasure shards

Instead of replicating full blobs, public post auto-replication distributes erasure-coded shards using a 3-of-10 scheme (k=3, n=10). Each shard contains 1/3 of the data. Reconstruction requires cooperation from any 3 of the 10 shard holders. A single shard is mathematically meaningless noise — not encrypted content where the full payload exists behind a key, but genuinely incomplete data that cannot be reconstructed alone.

### Where sharding applies

The existing storage tiers each have their own liability story. Sharding only fills the gap for public auto-replication:

Tier | Storage | Defense
--- | --- | ---
Author’s node | Full copy | Publisher responsibility (content originator)
Pulled content (follows) | Full copy | User consent — explicit follow relationship
Private auto-replication | Encrypted (CEK envelope, Section 20) | Replicating nodes are provably not keyring recipients — existing encryption architecture handles this
Public auto-replication | Erasure-coded shards | Sub-threshold shard — reconstruction impossible from any single holder

### Shard assignment

Slot assignment is deterministic from the PostId via DHT-style hashing, carried in the existing BlobHeader metadata — no additional discovery round required. Each node enforces single-slot acceptance: it only accepts shard push offers for its assigned slot, rejecting others. This prevents a bad actor from accumulating multiple shards toward the reconstruction threshold. Slot assignment is acceptance policy, not exclusivity — transient duplicate holders for the same slot are harmless and add redundancy.

### Health monitoring

- ≥5 live slots: healthy, no action
- 4 live slots: trigger background re-replication, targeting the longest-dark slot first
- <3 live slots: content at risk (requires catastrophic loss of 8+ nodes simultaneously)

### Interaction with full copies

As content gains followers, the follow graph naturally absorbs redundancy through full-copy pull sync. The shard layer can back off:

- 2+ full copies in mesh: equivalent to ≥4 live shards → shard chain deprioritizes, may decay
- 1 full copy: shard chain reformation trigger
- 0 full copies: shard chain is sole redundancy, maintain aggressively

This means popular content automatically shifts from CDN shard infrastructure to the social follow graph. The shard layer only works hard for content nobody has explicitly chosen to keep — exactly the content with the highest liability exposure.

### Re-replication

When a slot goes dark, a new shard holder is assigned via DHT. The new holder determines which chunks belong to its slot and pulls only those chunks from the live shard holders that have them. No shard holder ever reconstructs the full content — each node only ever possesses its own slot’s chunks. The pulling node identifies what it needs, requests those specific chunks, and aggressively refuses anything outside its assigned slot. The author’s node can go offline permanently once mesh replication is established.

### Replication window

Active shard push replication applies the same 72-hour window as full-copy replication (Section 19): only posts less than 72 hours old are actively pushed to shard holders. Beyond 72 hours, the shard chain relies on natural decay and pull-based replication only.

Exception — share link re-promotion: When a share link is added to a post’s BlobHeader, the 72-hour active replication window resets from that event. This ensures that content being actively shared gets CDN-prioritized delivery regardless of original post age. The re-promotion window is 72 hours from the share link addition, not from the original post timestamp.

### Implementation path

Extends the existing `ReplicationRequest`/`ReplicationResponse` (0xE1/0xE2) protocol. Shard slot metadata fits in the existing BlobHeader. The CDN hosting tree, downstream registration, and eviction scoring (Section 18) continue to work unchanged for full copies — sharding is an additional layer for the auto-replication path only.

## 19. Sync Protocol

### Wire format

    [1 byte: MessageType] [4 bytes: length (big-endian)] [length bytes: JSON payload]

Max payload: 16 MB. ALPN: `itsgoin/3`.

### Pull sync: social + file layers, not mesh

v0.2.0 change: Pull sync pulls posts from social layer peers (follows, audience) and upstream file peers, NOT from mesh peers. Mesh connections exist for routing diversity, not content. This separates infrastructure from content flow.

Self Last Encounter: For each peer we sync with, we track the timestamp of our last successful sync. When Self Last Encounter ages beyond 4 hours, a pull sync is triggered. Self Last Encounter is updated to the newer of: (a) what's currently stored, or (b) the "file last update" timestamp from file headers received during blob transfers. Since file headers include the author's recent post list, downloading a blob from any peer hosting that author's content can update Self Last Encounter for the author.

### Pull sync filtering

- PullSyncRequest: Includes requester's follow list + post IDs they already have
- PullSyncResponse: Sender filters posts through `should_send_post()`:
  1. Author is requester → always send (own posts relayed back)
  2. Public + author in requester's follows → send
  3. Encrypted + requester in wrapped key recipients → send
  4. Otherwise → skip

### Message types (49 total)

Hex | Name | Stream | Purpose
--- | --- | --- | ---
`0x01` | NodeListUpdate | Uni | Incremental N1/N2 diff broadcast
`0x02` | InitialExchange | Bi | Full state exchange on connect
`0x03` | AddressRequest | Bi | Resolve NodeId → address via reporter
`0x04` | AddressResponse | Bi | Address resolution reply
`0x05` | RefuseRedirect | Uni | Refuse mesh + suggest alternative
`0x40` | PullSyncRequest | Bi | Request posts filtered by follows
`0x41` | PullSyncResponse | Bi | Respond with filtered posts
`0x42` | PostNotification | Uni | Lightweight "new post" push to social contacts
`0x43` | PostPush | Uni | Direct encrypted post delivery to recipients
`0x44` | AudienceRequest | Bi | Request audience member list
`0x45` | AudienceResponse | Bi | Audience list reply
`0x50` | ProfileUpdate | Uni | Push profile changes
`0x51` | DeleteRecord | Uni | Signed post deletion
`0x52` | VisibilityUpdate | Uni | Re-wrapped visibility after revocation
`0x60` | WormQuery | Bi | Burst/nova search for nodes, posts, or blobs beyond N3
`0x61` | WormResponse | Bi | Worm search reply (node + post_holder + blob_holder)
`0x70` | SocialAddressUpdate | Uni | Social contact address changed
`0x71` | SocialDisconnectNotice | Uni | Social contact disconnected
`0x72` | SocialCheckin | Bi | Keepalive + address + N+10 update
`0x90` | BlobRequest | Bi | Fetch blob by CID
`0x91` | BlobResponse | Bi | Blob data + CDN manifest + file header
`0x92` | ManifestRefreshRequest | Bi | Check manifest freshness
`0x93` | ManifestRefreshResponse | Bi | Updated manifest reply
`0x94` | ManifestPush | Uni | Push updated manifests downstream
`0x95` | BlobDeleteNotice | Uni | CDN tree healing on eviction
`0xA0` | GroupKeyDistribute | Uni | Distribute circle group key to member
`0xA1` | GroupKeyRequest | Bi | Request group key for a circle
`0xA2` | GroupKeyResponse | Bi | Group key reply
`0xB0` | RelayIntroduce | Bi | Request relay introduction
`0xB1` | RelayIntroduceResult | Bi | Introduction result with addresses
`0xB2` | SessionRelay | Bi | Splice bi-streams (own-device default)
`0xB3` | MeshPrefer | Bi | Preferred peer negotiation
`0xB4` | CircleProfileUpdate | Uni | Encrypted circle profile variant
`0xC0` | AnchorRegister | Uni | Register with anchor (bootstrap/recovery only)
`0xC1` | AnchorReferralRequest | Bi | Request peer referrals from anchor
`0xC2` | AnchorReferralResponse | Bi | Referral list reply
`0xC3` | AnchorProbeRequest | Bi | A → B → C: test cold reachability of address
`0xC4` | AnchorProbeResult | Bi | C → A (success) or C → B → A (failure)
`0xD0` | BlobHeaderDiff | Uni | Incremental engagement update (reactions, comments, policy, thread splits)
`0xD1` | BlobHeaderRequest | Bi | Request full engagement header for a post
`0xD2` | BlobHeaderResponse | Bi | Full engagement header response (JSON)
`0xD3` | PostDownstreamRegister | Uni | Register as downstream for a post (CDN tree entry)
`0xD4` | PostFetchRequest | Bi | Request a single post by ID from a known holder
`0xD5` | PostFetchResponse | Bi | Single post response (SyncPost or not-found)
`0xD6` | TcpPunchRequest | Bi | Ask holder to punch TCP toward browser IP
`0xD7` | TcpPunchResult | Bi | Punch result + HTTP address for redirect
`0xE0` | MeshKeepalive | Uni | 30s connection heartbeat
`0xE1` | ReplicationRequest | Bi | Request peer to cache specific posts
`0xE2` | ReplicationResponse | Bi | Accept/reject replication request

### Engagement propagation

Reactions, comments, and policy changes propagate via `BlobHeaderDiff` (0xD0) through the CDN tree:

- Push (real-time): On react/comment, the diff is sent to both downstream peers (CDN tree children) and all upstream peers (up to 3, who we got the post from). Each intermediate node re-propagates both directions, excluding sender. This flows the diff up to the author and down to all holders.
- Auto downstream registration: Nodes that receive a post via pull sync or push notification automatically send `PostDownstreamRegister` (0xD3) to the sender, ensuring bidirectional diff flow.
- Pull (safety net): Tiered frequency based on content age: 5min (<72h), 1hr (3-14d), 4hr (14-30d), 24hr (>30d). Requests `BlobHeaderRequest` (0xD1) with local header timestamp. Peers respond with full header only if newer. Additive merge — `store_reaction` upserts, `store_comment` inserts with ON CONFLICT DO NOTHING. Writes batched per chunk (single lock acquisition).
- Tombstones: Deleted reactions and comments are not hard-deleted. Instead, a `deleted_at` timestamp is set on the record. Tombstones propagate via pull sync headers. Prevents deleted engagement from being re-introduced by peers that haven't yet received the deletion.
- Planned: Pull engagement from both upstream and downstream peers to catch missed diffs from either direction.

### Engagement security Complete

Engagement operations are cryptographically verified on receipt to prevent forgery and unauthorized modification:

- Reaction signatures: Each reaction carries an ed25519 signature over `BLAKE3(reactor || post_id || emoji || timestamp_ms)`. Verified before storing. Unsigned reactions from older nodes accepted for backward compatibility (`#[serde(default)]` on signature field).
- Comment signatures: Comments carry an ed25519 signature (existing field). `verify_comment_signature()` now called on receipt via `BlobHeaderDiff`. Forged comments rejected.
- Reaction removal: Only the reactor (original author of the reaction) or the post author can remove a reaction. Sender verified against `reactor` and `payload.author`.
- Comment edit/delete: Only the comment author can edit; comment author or post author can delete. Sender identity verified via QUIC transport authentication (iroh ed25519).
- BlobHeader author: When rebuilding headers after engagement diffs, the author is verified against the stored post’s actual author, not trusted from the payload. Prevents relay nodes from misattributing headers.
- BlobHeader source of truth: The `reactions` and `comments` tables are authoritative. The `blob_headers` JSON is a derived snapshot rebuilt after each engagement operation. When they diverge (e.g., after a `BlobHeaderResponse` with a newer snapshot), the next engagement op rebuilds from tables.

### Device roles & bandwidth budgets Complete

Each node advertises its device role in `InitialExchange`, which determines its bandwidth budgets for replication (pulling posts to cache) and delivery (serving requests from peers):

Role | Replication / hour | Delivery / hour
--- | --- | ---
Intermittent (phones) | 100 MB | 1 GB
Available (desktops) | 200 MB | 2 GB
Persistent (anchors) | 200 MB | 1 GB

- Budgets auto-reset every hour
- Role is self-declared based on device type and advertised to peers in `InitialExchange`
- Peers respect advertised budgets when selecting replication targets

### Active CDN replication Complete

All devices proactively replicate recent under-replicated posts to peers, not just passively serve on request:

- 10-minute cycle: All devices initiate replication checks every 10 minutes
- Target prioritization: Desktops > anchors > phones, scored by available bandwidth budget and connection quality
- Selection criteria: Posts less than 72 hours old with fewer than 2 downstream replicas are selected for replication
- Protocol: `ReplicationRequest` (0xE1) asks a peer to cache specific posts; `ReplicationResponse` (0xE2) accepts or rejects based on available budget and storage
- Graceful degradation: In small networks with few peers, the cycle runs but finds few or no viable targets — no wasted bandwidth. As the network grows, replication naturally increases.

### Connection rate limiting

Incoming QUIC connections that fail authentication are rate-limited per source IP to prevent CPU exhaustion from rogue or stale nodes:

- First 3 failures: logged normally, connection attempts proceed
- 4+ failures: silently dropped with exponential backoff (2n-3 seconds, capped at ~256s)
- Successful connection: clears the failure count for that IP
- Cleanup: stale entries removed every 60 seconds (pruned after 5 minutes of inactivity)

### N2/N3 freshness

- TTL: N2/N3 entries expire after 5 hours (pruned during rebalance cycle)
- Full state broadcast: Every 4 hours, nodes re-broadcast their complete N1/N2 state (not just diffs) to catch any missed incremental updates
- Disconnect cleanup: When a mesh peer disconnects, all N2/N3 entries they reported are immediately removed (`clear_peer_n2`/`clear_peer_n3`)
- Startup sweep: On boot, all N2/N3 and mesh_peers entries are cleared — they're stale from the previous session and will be rebuilt as connections establish

### Bootstrap isolation recovery

Prevents network segments from becoming permanently isolated from the main network:

- 24-hour check: Starting 24 hours after boot, nodes verify the bootstrap anchor is within their N1/N2/N3 reach
- If absent: reconnect to bootstrap, request referrals, connect to referred peers
- Sticky N1: The bootstrap is added to a sticky N1 set for 24 hours, so mesh peers learn about it via routing diffs and can independently bridge back to the main network
- Self-limiting: Once the bootstrap is in N3, the check passes and no action is taken. Goes silent as the network grows.

### Schema versioning

SQLite databases track their schema version via `PRAGMA user_version`. On startup:

- If the database version is older than `MIN_MIGRATABLE_VERSION`, the database is reset (preserving identity key)
- If older than the current version, data migrations run once and the version is bumped
- Schema-level changes (new tables, columns) are handled by `init_tables()` (`CREATE TABLE IF NOT EXISTS`) and `migrate()` (column-level `ALTER TABLE` checks)

### Upstream tracking (`post_upstream`)

Each post tracks which peer it was received from in the `post_upstream` table (post_id → peer_node_id). Set during pull sync and push notification. Used for:

- Engagement diff propagation toward the post author (hop-by-hop upstream)
- Future: N+10 identification in blob headers (upstream source's N+10)

### IPv6 HTTP address advertisement

Nodes with public IPv6 addresses advertise their actual routable address (from `endpoint.addr().ip_addrs()`) paired with their bound port, rather than the bind address (`0.0.0.0`). This enables direct browser-to-node HTTP serving for share links. Unroutable addresses (`0.0.0.0`, `127.x`) are filtered out in the tiered web serving redirect path.

### Encrypted receipt & comment slots

Private posts (Friends, Circle, Direct) carry encrypted slots in their BlobHeader for delivery receipts, read receipts, reactions, and private comments. The CDN propagates these as opaque bytes — only participants with the post’s CEK can decrypt them.

#### Design principles

- Pre-filled noise: All slots are filled with random bytes on post creation. Writing to a slot replaces noise with encrypted content of the same size, making writes indistinguishable from creation to observers.
- Slot key derivation: `slot_key = BLAKE3_derive_key("itsgoin/slot/v1", CEK)`. Only participants who can decrypt the post can read/write slots.
- CDN-safe: Relay nodes store and propagate slot bytes without decryption. No new protocol messages needed — slots travel via `BlobHeaderDiff`.

#### Receipt slots (64 bytes each)

- Allocation: 1 per participant (including author)
- Slot assignment: Participants sorted by NodeId; slot index = position in sorted list
- Decrypted format: `[1 byte: state][8 bytes: timestamp_ms BE][23 bytes: emoji/padding]`
- States: 0=empty/noise, 1=delivered, 2=seen, 3=reacted
- Author pre-feed: Author can write their own slot with a reaction emoji at creation time

#### Comment slots (256 bytes each)

- Allocation: `ceil(participants / 3)` initial slots, expandable via `AddCommentSlots` diff op
- Decrypted format: `[32 bytes: author_node_id][8 bytes: timestamp_ms BE][216 bytes: UTF-8 content + zero padding]`
- Slot selection: First available (all-zero content after decryption = available)
- Growth: When all comment slots are used, any participant can append new noise-filled slots

#### Wire operations

Op | Purpose
--- | ---
`WriteReceiptSlot` | Update a receipt slot (state change: delivered → seen → reacted)
`WriteCommentSlot` | Write encrypted comment to a slot
`AddCommentSlots` | Append new noise-filled comment slots when capacity is exhausted

#### UI

DM conversations display delivery indicators: single checkmark (sent), double checkmark (delivered/on device), blue double checkmark (seen), emoji (reacted). Opening a conversation auto-marks incoming messages as seen. Messages have a react button for emoji responses.

### Protocol v4: Header-Driven Sync Complete

Major sync protocol revision that replaces the current pull-everything-from-everyone model with header-driven discovery, per-author tracking, and tiered engagement polling. Reduces bandwidth by ~90% for established nodes.

#### Core principle: headers as notification

The AuthorManifest already carries a post neighborhood (20 previous + 20 following PostIds). When this neighborhood is pushed via `ManifestPush` (0x94) or travels with blob responses, receiving nodes can diff their local post list against the neighborhood to discover new posts without a full pull sync. The CDN tree becomes the notification system.

#### Self Last Encounter (per-author sync tracking)

Implements the v0.2.0 design intent: track `last_sync_ms` per followed author. Pull sync triggers only when `now - last_sync_ms > check_interval`. Updated on:

- Successful pull sync response containing that author's posts
- ManifestPush/BlobHeaderDiff received for that author
- Blob transfer that includes that author's file header

#### Slim PullSyncRequest

Current: `{ follows: Vec<NodeId>, have_post_ids: Vec<PostId> }` — sends ALL post IDs every request.

v4: `{ follows: Vec<NodeId>, since_ms: HashMap<NodeId, u64> }` — per-author timestamps. Response contains only posts newer than the requester's timestamp for each author. Drops request size from O(posts) to O(follows).

#### Tiered pull sync frequency

Instead of pulling from ALL mesh peers every 5 minutes, pull is driven by Self Last Encounter staleness:

- Pull only triggers for authors where `last_sync_ms` is stale AND no header updates were received in the interval
- Default check interval: 4 hours (reduced from current 5-minute blanket pull)
- Pull targets known CDN sources first (peers in `post_upstream` / `post_downstream` for that author), falling back to mesh peers only if CDN sources are unavailable
- Serial, not parallel — one peer at a time, stop when delta is received

#### Tiered engagement check rates

Engagement header polling (`BlobHeaderRequest` 0xD1) frequency scales with content age and activity:

Content age / last engagement | Check interval
--- | ---
< 72 hours | 5 minutes
3–14 days | 1 hour
14–30 days | 4 hours
> 30 days | 24 hours

DB tracks `last_engagement_ms` and `last_check_ms` per post. A single SQL query filters posts due for check:

    SELECT post_id FROM posts
    WHERE last_check_ms < now_ms - CASE
      WHEN last_engagement_ms > now_72h THEN 300000
      WHEN last_engagement_ms > now_14d THEN 3600000
      WHEN last_engagement_ms > now_30d THEN 14400000
      ELSE 86400000
    END

Checks are serial (one peer at a time). A single “no new engagement” response is treated as authoritative — if that peer missed an update, it would have been replicated to them by now.

On connect/wake: if `since_last_check > check_rate` for any post, an automatic check runs immediately.

#### Multi-upstream (3 max)

Extend `post_upstream` from 1 entry to up to 3 per post:

- Primary upstream: First source of the post (current behavior)
- Backup upstreams: Additional sources discovered via CDN tree or replication
- Upstream unavailable: If primary is unreachable, promote backup. If no backups, use the CDN node map to find the author’s upstream chain and request connection — accepted or directed to an alternate via RefuseRedirect
- Unregistration: `BlobDeleteNotice` (0x95) already handles “I no longer hold this.” Extend to upstream: “I’m no longer your upstream for this post.”
- Overhead is minimal: 3 rows per post vs 100 rows per post for downstream

#### Auto-prefetch followed authors

When a post by a followed author appears in any header (PullSyncResponse, ManifestPush, BlobHeaderDiff neighborhood), prefetch the post and its blobs if:

- Post age < 90 days
- Post not already in local storage
- Within replication budget

Encrypted posts without key: Store the encrypted post in DB marked as “not-for-us.” The node contributes to CDN availability for those who CAN decrypt it. Normal cache decay handles cleanup — these posts are given low eviction priority (stranger relationship score of 0.1) and will be evicted when space is needed.

#### Header-driven post discovery flow

1. Author creates post → updates their AuthorManifest neighborhood (20 before + 20 after)
2. ManifestPush (0x94) propagates to downstream CDN tree
3. Each receiving node diffs the neighborhood against local post list
4. New post IDs discovered → fetch via PostFetch (0xD4/0xD5) from the peer that sent the manifest
5. Blobs prefetched for followed authors (within budget)
6. Node registers as downstream for the new post
7. Self Last Encounter updated for the author → suppresses pull sync for that author

This makes the CDN tree the primary content distribution channel, with pull sync serving only as a safety net for missed headers.

#### Migration path

v4 is backward-compatible via ALPN negotiation. Nodes running v3 continue to work with the existing pull model. v4 nodes detect peer capability during InitialExchange and use the optimized paths when both sides support them. The v3 pull cycle remains as a fallback for mixed-version networks.

#### New DB columns required

Table | Column | Purpose
--- | --- | ---
`posts` | `last_engagement_ms` | Timestamp of most recent reaction/comment
`posts` | `last_check_ms` | Timestamp of last engagement check
`follows` | `last_sync_ms` | Self Last Encounter per followed author
`post_upstream` | (expand to 3 rows) | Multi-upstream with priority

## 20. Encryption

### Envelope encryption (1-layer) Complete

1. Generate random 32-byte CEK (Content Encryption Key)
2. Encrypt content: `ChaCha20-Poly1305(plaintext, CEK, random_nonce)`
3. Store as: `base64(nonce[12] || ciphertext || tag[16])`
4. For each recipient (including self):
   - X25519 DH: `our_ed25519_private (as X25519) * their_ed25519_public (as montgomery)`
   - Derive wrapping key: `BLAKE3_derive_key("distsoc/cek-wrap/v1", shared_secret)`
   - Wrap CEK: `ChaCha20-Poly1305(CEK, wrapping_key, random_nonce)` → 60 bytes per recipient

### Visibility variants

Variant | Overhead | Audience limit
--- | --- | ---
`Public` | None | Unlimited
`Encrypted { recipients }` | ~60 bytes per recipient | ~500 (256KB cap)
`GroupEncrypted { group_id, epoch, wrapped_cek }` | ~100 bytes total | Unlimited (one CEK wrap for the group)

### PostId integrity

`PostId = BLAKE3(Post)` covers the ciphertext, NOT the recipient list. Visibility is separate metadata. This means visibility can be updated (re-wrapped) without changing the PostId.

### Group keys (circles) Complete

- Each circle gets its own ed25519 keypair
- `group_id = BLAKE3(initial_public_key)` — permanent identifier
- Group seed wrapped per-member via X25519 DH (KDF domain: `"distsoc/group-key-wrap/v1"`)
- Epoch rotation: On member removal, generate new keypair, increment epoch, re-wrap for remaining members
- Wire: `GroupKeyDistribute` (`0xA0`), `GroupKeyRequest/Response` (`0xA1`/`0xA2`)

### Three-tier access revocation

Three levels of revocation, chosen based on threat level:

### Tier 1: Remove Going Forward (default)

Revoked member is excluded from future posts automatically. They retain access to anything they already received. This is the default behavior when removing a circle member — no special action needed.

When to use: Normal membership changes. Someone leaves a group, you unfollow someone. The common case.

Cost: Zero. Just stop including them in future recipient lists.

### Tier 2: Rewrap Old Posts (cleanup)

Same CEK, re-wrap for remaining recipients only. The revoked member can no longer unwrap the CEK even if they later obtain the ciphertext. Propagate updated visibility headers via `VisibilityUpdate` (`0x52`).

When to use: Revoked member never synced the post (common with pull-based sync — encrypted posts only sent to recipients). You want to clean up access lists.

Cost: One WrappedKey operation per remaining recipient, no content re-encryption.

### Tier 3: Delete & Re-encrypt (nuclear)

Generate new CEK, re-encrypt content, wrap new CEK for remaining recipients, push delete for old post ID, repost with new content but same logical identity. Well-behaved nodes honor the delete.

When to use: Revoked member already has the ciphertext and could unwrap the old CEK. Only for content that poses an actual danger/risk if the revoked member retains access. Recommended against in most cases.

Cost: Full re-encryption + delete propagation + new post propagation. Heavy.

Trust model: The app honors delete requests from content authors by default. A modified client could ignore deletes, but this is true of any decentralized system. For legal purposes: the author has proof they issued the delete and revoked access.

### Private profiles (Phase D-4) Complete

Different profile versions per circle, encrypted with the circle/group key. A peer sees the profile version for the most-privileged circle they belong to. `CircleProfileUpdate` (`0xB4`) wire message. Public profiles can be hidden (`public_visible=false` strips display_name/bio).

## 21. Delete Propagation

### Status: Complete

### Delete records

`DeleteRecord { post_id, author, timestamp_ms, signature }` — ed25519-signed by author. Stored in `deleted_posts` table (INSERT OR IGNORE). Applied: DELETE from `posts` table WHERE post_id AND author match.

### Propagation paths

1. InitialExchange: All delete records exchanged on connect
2. DeleteRecord message (`0x51`): Pushed via uni-stream to connected peers on creation
3. PullSync: Included in responses for eventual consistency

### CDN cascade on delete

1. Send BlobDeleteNotice to all downstream hosts (with our upstream info for tree healing)
2. Send BlobDeleteNotice to upstream
3. Clean up blob metadata, manifests, downstream/upstream records
4. Delete blob from filesystem

## 22. Social Graph Privacy

### Status: Complete

- Follows are never shared in gossip or profiles
- N1 share merges mesh peers + social contacts into one list (indistinguishable)
- No addresses ever shared in routing updates
- N3 is never shared outward (search-only)

Known temporary weakness: An observer who diffs your N1 share over time can infer your social contacts (they're the stable members while mesh peers rotate). This will be addressed when CDN file-swap peers are added to N1, making the stable set larger and harder to distinguish.

## 23. Identity Management

### Multi-identity per device Complete

A single device can hold multiple identities, each with its own ed25519 keypair, database, blob store, follows, and posts. One identity is active at a time — switching performs a hot-swap (Node teardown + rebuild, ~3-5 seconds).

- Directory structure: `itsgoin-data/identities/{node_id_hex}/` — each identity gets its own subdirectory with `identity.key`, `itsgoin.db`, `blobs/`, and `meta.json`
- Legacy migration: Flat `itsgoin-data/` layout auto-migrates to per-identity subdirectories on first launch
- Create, import, switch, delete via Settings UI
- Key permissions: `identity.key` files written with 0600 permissions (Unix)

### Multi-device identity Planned

Multiple devices share the same identity key (ed25519 keypair, same NodeId). All devices ARE the same node from the network’s perspective. Posts from any device appear as the same author.

- Setup: Export `identity.key` from one device, import on another using the identity management UI
- Device identity: Each device generates a unique device keypair for self-routing and conflict resolution (planned)
- Own-device relay: Route traffic through your own devices (planned)

### Post import & merge Planned

Import posts from another identity into the current one. Public posts imported directly. Encrypted posts require the original identity’s key for decryption, then re-encrypted under the current identity. Merge creates new posts (new PostId, new author) with original timestamps preserved and prior author noted in BlobHeader.

## 24. Phase 2: Reciprocity (Reconsidered)

### Status: Reconsidered

The original Phase 2 design centered on hosting quotas (3x rule), chunk audits, and tit-for-tat QoS. On reflection, the attention-driven delivery model makes quota enforcement unnecessary. The CDN is a delivery amplifier, not a storage system — hot content propagates through demand, cold content decays. Authors are responsible for their own content durability.

Tit-for-tat QoS solves the wrong problem: it optimizes for fairness in a storage-obligation model that no longer exists. What matters is that the delivery network functions efficiently, which it does through natural attention dynamics.

If reciprocity mechanisms are needed at scale, they should address delivery quality (bandwidth, latency, uptime) rather than storage quotas. This remains an open design area.

## 25. HTTP Post Delivery

### Status: Complete

Direct peer-to-browser HTTP serving is implemented. For share link delivery, this is now part of the tiered web serving strategy (redirect → TCP punch → QUIC proxy) described in Section 26.

### Intent

Every ItsGoin node that is publicly reachable can serve its cached public posts directly to browsers over HTTP — no extra infrastructure, no additional dependencies, no new binary. The same QUIC UDP port used for app traffic is accompanied by a TCP listener on the same port number. UDP goes to the QUIC stack as always. TCP goes to a minimal raw HTTP/1.1 handler baked into the binary.

This makes every publicly-reachable node a browser-accessible content endpoint, enabling share links that deliver content peer-to-browser without routing any post bytes through itsgoin.net.

### Dual listener architecture

    <port>/UDP  →  QUIC (existing app protocol)
    <port>/TCP  →  HTTP/1.1 (new, read-only, single route)

Both listeners bind on the same port. The OS routes UDP and TCP to separate sockets — no conflict, no protocol ambiguity.

### HTTP handler

The handler is intentionally minimal — implemented with raw `tokio::net::TcpListener`, no HTTP crate, no new dependencies. Approximately 150–200 lines of Rust.

Single valid route: `GET /p/<postid_hex> HTTP/1.1`

- `postid_hex` must be exactly 64 lowercase hex characters (BLAKE3 hash)
- Any other path, method, or malformed request: hard close with no response (not even a 400). Do not be helpful to malformed requests.
- Post must be public (`PostVisibility::Public`). Encrypted posts are never served over HTTP regardless of whether the node holds the content.

Response: Minimal HTML page containing the post content with a small footer:

    <footer>
      This post is on the ItsGoin network — content lives on people's devices,
      not servers. <a href="https://itsgoin.com">Get ItsGoin</a>
    </footer>

The footer HTML is a static string constant compiled into the binary (~2KB). No template engine, no dynamic footer generation.

### Security constraints

Concern | Mitigation
--- | ---
Connection exhaustion | Hard cap: 20 concurrent HTTP connections. New connections over the cap are immediately closed. No queue, no wait.
Slow HTTP attacks | 5-second read timeout for complete request headers. Exceeded → hard close.
Content enumeration | Identical response (hard close) for “post not found” and “post not public.” No timing oracle, no distinguishable error codes.
Malformed requests | Hard close only. No error response.
Encrypted content | Never served. Public visibility check is mandatory before any response.

### Which nodes serve HTTP

A node serves HTTP only if it is publicly TCP-reachable:

- IPv6 public address — serves directly
- IPv4 + UPnP mapping — serves if TCP is included in the UPnP mapping (see Section 11 update)
- IPv4 behind NAT without UPnP — cannot serve HTTP, but can still appear as a host in share links for app-protocol delivery. The CDN tree and itsgoin.net redirect handler route around unreachable nodes automatically.

### 302 load shedding via CDN tree

When a node is overwhelmed (at the 20-connection cap) or chooses to redirect:

1. Query `post_downstream` table for the requested postid
2. Filter downstream hosts to those with a known public address (IPv6 or UPnP-mapped IPv4)
3. `302 → http://[their_address]:<port>/p/<postid>`

The receiving node applies the same logic recursively if needed. This mirrors the app-layer CDN tree behavior at the HTTP layer — the same attention-driven propagation model, the same tree structure, now accessible to browsers.

### Binary size impact

Zero new dependencies. Negligible compiled size delta (~10–20KB). No App Store size concerns. No install size impact for existing users.

## 26. Share Links

### Intent

Every public post can be shared as a URL that works for both app users and browser users:

- App installed: OS intercepts the URL via Universal Links (iOS) / App Links (Android) before the browser loads. App opens directly to the post, fetched via QUIC. Zero browser involvement.
- No app: Browser loads itsgoin.net, which searches the ItsGoin network for the post and redirects the browser to a live node serving it over HTTP. The share link becomes a product demo and install opportunity.

### URL format

    https://itsgoin.net/p/<postid_hex>/<author_nodeid_hex>

- `postid_hex`: 64 hex characters (BLAKE3 post hash)
- `author_nodeid_hex`: 64 hex characters (author's ed25519 public key). Enables direct QUIC connection as fast path; worm search handles the case where author is unreachable.

Example: `https://itsgoin.net/p/3a7f...c921/b4e2...f817`

Simple, human-inspectable, no binary encoding needed. Author ID in the URL is sufficient for the tiered serving strategy below.

### Tiered web serving

When a browser visits a share link, itsgoin.net attempts three tiers to deliver the post:

1. 302 redirect: If a publicly-reachable holder is known (IPv6, UPnP TCP), redirect the browser directly to that node's HTTP endpoint. Zero post bytes flow through itsgoin.net.
2. TCP punch: If a holder is connected but not publicly reachable, send `TcpPunchRequest` (0xD6) asking the holder to punch TCP toward the browser's IP. On success, `TcpPunchResult` (0xD7) returns the HTTP address for a redirect.
3. QUIC proxy: If neither redirect nor punch works, itsgoin.net fetches the post on-demand via PostFetch (0xD4/0xD5), renders HTML, and serves directly to the browser.

### itsgoin.net QUIC proxy handler

Route: `GET /p/<postid_hex>/<author_nodeid_hex>`

    1. Check local storage (fast path — post already fetched recently)
    2. Connect to author via QUIC (connect_by_node_id cascade) → PostFetch
    3. Content search (extended worm with post_id) → find holder → PostFetch
    4. Found? Store temporarily + render HTML + serve to browser
    5. Not found? Serve "unavailable" page

The itsgoin.net server acts as a QUIC proxy — it fetches posts on-demand from the peer network and renders them as HTML for the browser. Posts are not permanently stored on the server. This is a marketing/convenience service, not a data store. The proxy model works because most nodes are behind NAT and can't serve HTTP directly to browsers.

Scalable via additional instances: `1.itsgoin.net`, `2.itsgoin.net`, etc. Each runs its own ItsGoin node with its own mesh connections, increasing total search coverage.

### itsgoin.net node

itsgoin.net runs a permanent, well-connected ItsGoin node (`--bind 0.0.0.0:4433 --daemon --web 8080`). This serves two purposes:

1. Content search — when a share link is visited, the server-side node uses the extended worm search (with `post_id`) to find any peer holding the post. The CDN tree means each peer knows about hundreds of posts across their downstream hosts, making hits highly likely even for content the server has never synced.
2. Anchor — a permanently-online, high-uptime node that bootstrap peers can rely on. Strengthens the network’s anchor infrastructure without any special protocol — it’s just a well-connected peer that happens to also serve the web handler.

### Share link format

`https://itsgoin.net/p/<postid_hex>/<author_nodeid_hex>`

- PostID: 64 hex chars (32 bytes, BLAKE3 content hash)
- Author NodeID: 64 hex chars (32 bytes, ed25519 public key)
- Author ID enables direct QUIC connection as fast path; worm search handles the case where author is unreachable
- Only public posts can generate share links

### Unavailable page

    ⬡ This content isn't currently reachable.

      It may be available again when someone
      who has it comes back online.

      [ Install ItsGoin to find it when it resurfaces ]

This is not a 404. It communicates the honest model: content lives on devices, not servers. Cold content decays. The install CTA is the honest answer to “how do I get this.”

### Universal Links / App Links

Same URL — `itsgoin.net/p/...` — intercepts to the native app for users who have ItsGoin installed. No separate URL scheme, no `app://` links.

Required static files on itsgoin.net:

    /.well-known/apple-app-site-association     (iOS Universal Links)
    /.well-known/assetlinks.json                (Android App Links)

Both are static JSON deployed once, pointing to the ItsGoin app package ID for the path pattern `/p/*`.

App-side handling: Register the URL pattern in Tauri config. On receipt of `itsgoin.net/p/<postid>/<hostlist>`, parse the postid, decode the hostlist, fetch via QUIC from the hostlist peers. If the post is already in local SQLite, render immediately. Universal Links intercept before the browser loads — itsgoin.net sees zero traffic for app users.

iOS caveat: Universal Links require the app to have been opened manually at least once before OS interception activates. First-time tap from a link goes to the browser fallback. All subsequent taps open the app directly. The browser fallback is the full loading screen experience — first-time users see the product demo, which is the right outcome anyway.

### QR codes

Share links are also valid QR codes. At ~127 chars base64url for 5 IPv6 hosts plus postid and domain, total URL length stays well under 200 characters — comfortably scannable at low error correction.

QR codes for share links use the same Universal Links / App Links interception path. A generic phone camera scanning the QR sees an itsgoin.net URL and offers to open it — either in the app (if installed) or in the browser.

No custom QR scheme needed. The HTTPS URL is the QR payload.

### Chrome HTTPS (October 2026)

Chrome 154 (October 2026) enables “Always Use Secure Connections” by default, warning before HTTP sites. This does not affect the share link architecture:

- itsgoin.net is HTTPS — no warning, Universal Links work normally
- The 302 redirect takes the browser off the HTTPS page before content loads, so no mixed-content issue
- Node HTTP endpoints are raw IP:port addresses, which Chrome treats as private network addresses (exempt from the public-site warning requirement)
- The redirect itself is a header only — no content flows through itsgoin.net

No architecture changes needed before or after October 2026.

## 27. Directory Service (Planned)

The directory is an opt-in convenience layer for discovery and creator protection. It is not node access — losing directory presence does not disconnect anyone from the network or from their existing connections. This asymmetry is load-bearing: humans with mature relationships shrug off directory loss; bots and content thieves depend on it entirely.

### Scope

- Whitelist track — discoverability, vouch-based entry, graph-scoped visibility.
- Blacklist track — community-flagged accounts and content; voluntary node-level replication refusal.
- Out of scope — node access, message delivery, post sync, existing follows. All continue to work without directory membership.

Because directory loss is a low-cost outcome for real humans and a high-cost outcome for bad actors, enforcement thresholds can be deliberately aggressive without meaningful false-positive risk.

### Entry

Two paths to directory listing, both yielding equal discoverability but different trust-building capacity:

- Vouch entry — 1 vouch from a directory member with remaining vouch capacity.
- Paid entry — fee in lieu of vouch. Grants directory presence only; vouch capacity starts at zero and grows only from received human vouches. Prevents ring-bootstrapping: a bad actor cannot pay their way to vouch power.

### Vouch capacity

A member's outbound vouch capacity is derived from received vouches:

- Base rate: 1 outbound vouch per 30 days per received vouch, up to a hard cap of 5 outbound vouches per 30 days.
- A member with zero received vouches (paid-entry only) has zero outbound capacity.
- Capacity regenerates; unused capacity does not accumulate beyond the monthly window.

Trust signals are internal, not purchasable. Aged off-platform accounts (FB, X, etc.) can be bought cheaply; in-system tenure crossed with graph density cannot. Vouch weight derives from the voucher's in-system graph depth and their history of non-revoked vouches — not account age.

### Cascade punishment

A "bad vouch" is a vouch later determined to have been extended to a bot, impersonator, content thief, or other directory-removable actor. Punishment is asymmetric — mild for humans who miscalled a single vouch, devastating for botnets whose strength is their dense internal vouch graph.

Offense | Immediate consequence | Recovery requirement
--- | --- | ---
1st bad vouch | All given and received vouches invalidated (member remains listed, but cannot vouch) | 2 NEW vouches received before outbound vouching resumes
2nd bad vouch | Removed from directory | 1 new vouch to relist + 2 additional new vouches before vouching again
3rd bad vouch | Removed from directory; 1-year outbound vouch freeze | 4 new vouches to relist; no outbound vouching for 12 months regardless of received vouches

"NEW" is strictly defined: a voucher who has never previously vouched this member and is not within the first-degree graph of any prior voucher for this member. This blocks ring members from cycling each other through as "recovery" vouchers.

Cascade radius. Invalidation of received vouches propagates the effective penalty upward: the bad actor's vouchers lose the credibility granted by that downstream relationship. One verified human assertion against a single botnet node can cascade through the entire dense cluster because bots' own topology carries the invalidation. They build the weapon that points back at them.

### Graph-relative visibility

The directory is not global. A viewer sees directory entries N hops from their own social graph (N tunable; start with N=3). Bots at the fringe of the graph are structurally invisible to most humans. Visibility is further rate-limited (Y new profiles per viewer per day, Y tunable) to make harvesting economically unattractive without affecting normal discovery.

This scoping is also why the vouch system's bot-ring problem is bounded: even a successful ring has no harvest value if it cannot be seen by real users.

### Verification (circuit breaker)

Verification is an emergency override, not a standing credential. In normal operation, the vouch system runs unassisted. When a creator notices their content suppressed or a bot cluster drowning legitimate signals, they invoke verification on their own terms.

- Trigger: creator-initiated only. No automatic verification prompts.
- Method: human interaction + fresh content submission (specifics TBD; must resist CAPTCHA-farmed and LLM-automated completion).
- Effect: verified status outweighs normal vouches for the duration it applies. A verified creator's reports of content theft or impersonation trigger cascade invalidation across the offender's vouch graph.
- Cost asymmetry: a botnet must sustain attacks indefinitely; the defender verifies once.

### Reporting

The reporting pipeline is lightweight, member-facing, and feeds a single review queue with multiple severity tracks:

- Impersonation — a member reports an account copying their (or someone they know's) identity. High-confidence signal: verifiable via profile comparison and content signatures.
- Bad vouch — a member flags a directory entry they vouched for, or vouches in their extended graph. Triggers cascade punishment on confirmation.
- Content theft — a creator reports unauthorized reposting. Evidence is the signed original versus the repost.
- Automated topology detection — graph analysis flags ring structures, abnormal clustering coefficients, and low-bridging clusters. Feeds the same queue as human reports.

Reports from verified accounts carry higher weight. A confirmed report against one member in a tight cluster opens review on the whole cluster.

### Blacklist

Blacklist is a higher-severity tier than directory suspension. Two states:

- Under review — enough independent reports to warrant scrutiny; member still discoverable but flagged.
- Confirmed blacklisted — entry persists at the identity level. A bad actor creating a new identity starts from zero; the old identity's blacklist entry remains as a reputational cost that outlasts the account.

Escalation paths:

- Confirmed impersonation → direct blacklist.
- Vouch violations → directory suspension first, blacklist only on repeated/escalated offenses.
- Content theft → voluntary replication refusal (below); blacklist for repeat offenders.

The blacklist must be slower and more evidence-bound than directory suspension. Suspension costs a bot everything but costs a human almost nothing; blacklist has real network consequences (below) and must not be a censorship weapon.

### Voluntary network compliance

Nodes may configure policies to decline replication or delivery of blacklisted content and accounts. This is opt-in, not forced. The effect is architectural starvation: stolen or abusive content cannot sustain replication when hosts collectively decline to be its infrastructure, while the legitimately signed original continues propagating normally.

This is the key lever for creator protection. ItsGoin does not enforce copyright; it gives creators a network that structurally prefers their signed original over any derivative copy. Combined with embedded ads (below), there is no version of content theft that economically benefits the thief inside ItsGoin.

### Creator-embedded ads

The platform does not insert or intermediate ads. Creators may embed ads directly in their content or feed, as part of the signed post.

- Ads inside signed content cannot be stripped without breaking the ed25519 signature — tampering is detectable.
- A repost that strips ads produces an unsigned-or-differently-signed copy, which compliance rules (below) treat as non-compliant.
- A repost that preserves ads intact monetizes the original creator automatically.
- Ad revenue therefore follows the legitimate copy by construction, not by enforcement.

### Repost framework

A two-track fair-use model. Compliant reposts are unblocked; non-compliant reposts feed the content-theft reporting pipeline.

Track | Content limit | Ads | Backlink | Value requirement
--- | --- | --- | --- | ---
Amplification | Full | Must preserve | Required to signed original | Reach is value; no further justification needed
Discussion / criticism | ≤1 minute per 4 minutes of original | Not required | Required to signed original | Commentary, review, or response

Edge cases (what counts as "amplification" vs. "wholesale copy masquerading as amplification") route to human reviewer capacity. The signed original is cryptographic evidence against the repost in any contested case — reviewers do not need to take anyone's word for what the original contained.

### Philosophical position

Most platforms have a structural conflict of interest with creators: fakes inflate engagement metrics, thieves generate content volume, both drive ad revenue. ItsGoin's incentives are inverted by design. Fakes degrade the vouch system; thieves attack the network's most valuable users; bots pollute the replication layer the network depends on. Every bad actor makes ItsGoin worse as software, not just ethically. The enforcement mechanisms above are therefore load-bearing, not policy theater.

### Implementation status

Designed, not implemented. Requires:

- Directory storage schema (vouch records, blacklist records, report queue, graph-density caches)
- New protocol messages (vouch, revoke, report, verify challenge/response)
- Graph-topology analysis job (automated ring/cluster detection)
- Client UX for discovery, vouching, reporting, verification
- Node-side replication policy config (blacklist honoring)
- Review-queue tooling for contested cases

Recommended staging: minimum viable slice = invite-tree directory + single-vouch entry + impersonation reports → manual review queue. Defer cascade math, automated topology detection, verification override, and repost compliance until real graph data exists to calibrate thresholds.

### Tunable parameters

Constant | Initial value | Purpose
--- | --- | ---
DIRECTORY_VOUCH_INTERVAL | 30 days | Time between outbound vouches per received vouch
DIRECTORY_VOUCH_CAP | 5 | Hard cap on outbound vouches per 30-day window
DIRECTORY_GRAPH_HOPS | 3 | Visibility radius for discovery
DIRECTORY_DISCOVERY_RATE | TBD / day | New profiles visible per viewer per day
RECOVERY_VOUCHES_1ST | 2 | NEW vouches to restore voucher status after 1st bad vouch
RECOVERY_VOUCHES_2ND | 1 + 2 | Relist + additional vouches after 2nd bad vouch
RECOVERY_VOUCHES_3RD | 4 | Relist vouches after 3rd bad vouch
RECOVERY_FREEZE_3RD | 365 days | Outbound vouch freeze after 3rd bad vouch
REPOST_DISCUSSION_RATIO | 1:4 | Max embed duration relative to original (discussion track)

## 28. Identity Architecture Mostly Shipped (v0.6.0)

v0.6.0 separates network identity (per-device routing/connection key) from posting identity (the face/persona authoring content). This is the architectural foundation for multi-device, multi-persona, and DM-level traffic-graph privacy. Five of the original six phases shipped as v0.6.0; ephemeral rotating DM IDs (§28.4) are deferred pending connection-model design work.

### 28.1 Two layers of identity Shipped

Each device has ONE network key — used for QUIC connections, endpoint ID, mesh routing. It's never linked on the wire to any posting key.

Each user can hold MANY posting keys simultaneously — no "active" persona, no switching. Each posting key is a persona (Public, Private, Work, Family, etc). Posts are signed with the posting key chosen at compose time. On first launch after upgrading, the existing identity becomes the default posting key (same bytes as the network key for upgraders, so all prior signed content keeps verifying).

Privacy invariant: peers cannot determine which network IDs belong to a given posting ID, which posting IDs belong to the same network ID, or which posting IDs belong to the same user. These associations are private to the device owner.

### 28.2 Persona types Shipped

- Public posting IDs — main persona(s), openly associated with "you"
- Private posting IDs — smaller-context personas for close contacts or specific groups
- Contextual posting IDs — per-relationship or per-group; user creates one per context and sticks with it

### 28.3 Multi-device is a special case Partial

"Two devices holding the same posting key" is a trivial case of the multi-key model. Link happens out-of-band between the user's own devices via the existing identity export/import bundle — the export includes all posting identities so they restore on the second device. The network sees no cross-device message announcing the relationship; each device pulls content for its posting IDs via the normal CDN. The fact that two network nodes hold the same posting key is only discoverable if an observer has private knowledge (which they shouldn't).

In-app QR / file-share flows for easy linking between an existing device and a new one are not wired yet — today it's manual export-then-import.

### 28.4 Ephemeral rotating IDs for DM threads Deferred

The intent: DM threads and group messages use per-thread unique posting IDs that rotate per message. Each encrypted message would include a handshake field — the next posting ID to use. Observer sees a stream of distinct posting IDs with no cryptographic tie between them, defeating thread-level traffic correlation. Desync recovery via a sliding window of the last N accepted IDs. Message history kept in a local encrypted-to-self archive that replicates across linked devices via self-follow.

Why deferred: the current connection model is per-network-ID (peers establish QUIC sessions keyed by stable NodeId; holder-set learning assumes durable addressable authors). Ephemeral authors that come into existence per-thread need a crisper design for how a freshly generated posting ID becomes reachable fast enough to receive replies before the next rotation. The pieces needed to implement rotation (per-file holder sets, merged pull with recipient-match, per-posting-ID signing) are all in place — what's missing is the cold-contact discovery semantics for ephemeral recipients.

### 28.5 CDN: per-file holder sets Shipped

The earlier upstream/downstream tree (which assumed a single author network endpoint) is replaced by a flat per-file holder set with header-diff propagation.

Each file (post, blob, manifest) has its own holder set (up to 5 most-recent peers, LRU-capped). When a new post is created, the creator updates the headers of recent prior posts (their manifests now reference the new post), then pushes the header diff to each updated prior post's holder set. Recipients apply the header diff, see the reference to the new post, and pull it through normal sync.

Notifications thus route via network-ID peers who happen to hold related files — not via any tree rooted at the author. Content always arrives via pull, never pushed directly. Four legacy tables (`post_upstream`, `post_downstream`, `blob_upstream`, `blob_downstream`) were dropped in v0.6; a one-way migration seeds `file_holders` from them on first launch.

### 28.6 DM privacy model Shipped

Two of the three originally-planned mechanisms shipped in v0.6.0 and eliminate the "A messaged B" traffic signal for follower-to-follower and cold-contact-via-pull scenarios:

1. CDN-only propagation. Direct `PostPush` for encrypted posts is removed. All encrypted posts propagate via the file-holder CDN, indistinguishable on the wire from any other encrypted content.
2. Merged pull + recipient-match. Pull sync's query is a uniform list of NodeIds; server returns posts matching `author ∈ query_list` OR `wrapped_key.recipient ∈ query_list`. Client always includes its own NodeId alongside follows. No distinguishable "searching for DMs" traffic fingerprint — the query looks identical to a routine follow-pull.
3. Comment-as-introduction for cold contact. Any public post with open comments serves as a message-request surface. The machinery exists (engagement diffs flow through the post's holder network); a dedicated UX affordance for "start a conversation by commenting" is still on the todo list.

### 28.7 What the user sees

- One merged incoming feed (all content to all personas), with filter-by-persona pills Shipped
- Per-post "(you) as <persona>" label on own posts authored by a non-default persona Shipped
- Persona picker at post-creation time, auto-hidden with a single persona Shipped
- Settings → Personas to create, delete, and set default Shipped
- Reply/comment default persona = whichever key decrypted the post Not yet
- Contextual compose defaults (e.g., posting to a circle auto-picks that circle's last-used persona) Not yet
- DMs to different personas as distinct inbox threads Not yet

### 28.8 Key/collision safety Shipped

Posting keys and network keys are ed25519 seeds (256 bits of entropy). Birthday paradox reaches 50% collision at ~2128 keys generated — not a concern even at aggressive rotation rates across a global userbase. The operational risk is weak RNG during key generation; we rely on the platform CSPRNG everywhere.

### 28.9 Rollout status

1. Phase 1 Shipped v0.6.0 — Direct `PostPush` for encrypted posts removed. Encrypted DMs propagate via ManifestPush like any other content.
2. Phase 2 Shipped v0.6.0 — File-holder model + header-diff propagation replaces upstream/downstream. Legacy tables dropped with a one-way seed migration.
3. Phase 3 Shipped v0.6.0 — Merged pull + recipient-match search. DM search indistinguishable from follow-pull. `post_recipients` index added.
4. Phase 4 Shipped v0.6.0 — Posting-key / network-key split plumbing. `posting_identities` table; seed migration copies network key in as default persona for upgraders.
5. Phase 5 Shipped v0.6.0 — Multi-persona UX (Settings page, compose picker, feed filter pills, per-post persona labels).
6. Phase 6 Deferred — Ephemeral rotating IDs for DM threads + local self-archive. See §28.4 for the open design question.

v0.6 is a hard network fork from v0.5. The two versions do not interoperate. This was an explicit simplification once the user base was small enough to migrate everyone directly — no dual-write tables, no legacy message handlers, no mixed-network testing. Users cross via the existing export/import identity bundle; see the download page for the upgrade path.

## Appendix A: Timeout Reference

Constant | Value | Purpose
--- | --- | ---
MESH_KEEPALIVE_INTERVAL | 30s | Ping to prevent zombie detection
ZOMBIE_TIMEOUT | 600s (10 min) | No activity → dead connection
SESSION_IDLE_TIMEOUT | 300s (5 min) | Reap idle interactive sessions (NOT keep-alive)
SELF_LAST_ENCOUNTER_THRESHOLD | 14400s (4 hours) | Trigger pull sync when last encounter exceeds this
QUIC_CONNECT_TIMEOUT | 15s | Direct connection establishment
HOLE_PUNCH_TIMEOUT | 30s | Overall hole punch window
HOLE_PUNCH_ATTEMPT | 2s | Per-address attempt within window
RELAY_INTRO_TIMEOUT | 15s | Relay introduction request
RELAY_PIPE_IDLE | 120s (2 min) | Relay pipe idle before close
RELAY_COOLDOWN | 300s (5 min) | Per-target relay cooldown
RELAY_INTRO_DEDUP | 30s | Dedup intro forwarding
WORM_TOTAL_TIMEOUT | 3s | Entire worm search
WORM_FAN_OUT_TIMEOUT | 500ms | Per-peer fan-out query
WORM_BLOOM_TIMEOUT | 1.5s | Bloom round to wide referrals
WORM_DEDUP | 10s | In-flight worm dedup
WORM_COOLDOWN | 300s (5 min) | Miss cooldown before retry
REFERRAL_DISCONNECT_GRACE | 120s (2 min) | Anchor keeps peer in referral list after disconnect
N2/N3_STALE_PRUNE | Immediate on disconnect + 7 day fallback | Remove reach entries tagged to disconnected peers; age-based fallback for stragglers
N2/N3_STARTUP_SWEEP | On boot | Remove all N2/N3 entries tagged to peers not in current mesh
PREFERRED_UNREACHABLE_PRUNE | 7 days | Release preferred slot (must re-negotiate MeshPrefer on reconnect)
RECONNECT_WATCHER_EXPIRY | 30 days | Low-priority reconnect awareness; daily check after 7 days
GROWTH_LOOP_TIMER | 60s | Periodic growth loop check
CONNECTIVITY_CHECK | 60s | Social/file <N4 access check for keep-alive sessions
DM_RECENCY_WINDOW | 14400s (4 hours) | DM'd nodes included in connectivity check
UPNP_DISCOVERY_TIMEOUT | 2s | Gateway discovery on startup (do not block)
UPNP_LEASE_RENEWAL | 2700s (45 min) | Refresh port mapping before TTL expiry
ANCHOR_PROBE_INTERVAL | 1800s (30 min) | Periodic re-probe while anchor-declared
ANCHOR_PROBE_TIMEOUT | 15s | Cold connect attempt by witness
ANCHOR_STALE_THRESHOLD | 7 days | Post-bootstrap cleanup probes known_anchors older than this

## Appendix B: Design Constraints

Constraint | Value | Notes
--- | --- | ---
Visibility metadata cap | 256 KB | Applies to WrappedKey lists in encrypted posts
Max recipients (per-recipient wrapping) | ~500 | 256KB / ~500 bytes JSON per WrappedKey
Max blob size | 10 MB | Per attachment
Max attachments per post | 4 |
Public post encryption overhead | Zero | No WrappedKeys, no sharding, unlimited audience
Max payload (wire) | 16 MB | Length-prefixed JSON framing
Mesh slots | 101 (Desktop) / 15 (Mobile) | Preferred + non-preferred, no local/wide distinction
Keep-alive session cap | 50% of session capacity | Ensures interactive sessions remain available
Keep-alive ceiling (desktop) | ~300–500 | Binding constraint: routing diff broadcast overhead
Keep-alive ceiling (mobile) | ~25–50 | Binding constraint: battery + OS background restrictions
`mesh_blacklist` table | `{ node_id }` | Targeted mutual stranger relationships for testing/diversity
`known_anchors` table | `{ node_id, addresses, last_seen }` | LIFO ordered, 7-day stale cleanup via probe

## Appendix C: Implementation Scorecard

Area | Status
--- | ---
Mesh connection architecture (101 slots, preferred/non-preferred) | Complete
N1/N2/N3 knowledge layers | Complete
Growth loop (60s timer + reactive on N2/N3) | Partial (timer exists, reactive trigger needs update)
Preferred peers + bilateral negotiation | Complete
N+10 identification | Partial (preferred peers exist, N+10 not in all headers)
Worm search (nodes + content search for posts/blobs) | Complete
Relay introduction + hole punch | Complete
Session relay (own-device default) | Partial (relay works, own-device restriction not implemented)
Social routing cache | Complete
Three-layer architecture (Mesh/Social/File) | Partial (layers exist conceptually, pull sync still uses mesh)
Keep-alive sessions | Planned
Self Last Encounter sync trigger | Planned
Algorithm-free reverse-chronological feed | Complete
Envelope encryption (1-layer) | Complete
Group keys for circles | Complete
Three-tier access revocation | Partial (Tier 1+2 work, Tier 3 crypto exists but no UI)
Private profiles per circle | Complete
Pull-based sync with follow filtering | Complete
Push notifications (post/profile/delete) | Complete
Blob storage + transfer | Complete
CDN hosting tree + manifests | Complete
Blob eviction with priority scoring | Complete
Anchor bootstrap + referrals | Complete
Delete propagation + CDN cascade | Complete
Multi-device identity | Planned
UPnP port mapping (desktop) | Complete
NAT type detection (STUN) + hard+hard skip | Complete
Advanced NAT traversal (role-based scanning + filter probe) | Complete
LAN discovery (mDNS scan + auto-connect) | Complete
Content propagation via attention | Partial
BlobHeader separation from blob content | Complete
25+25 neighborhood with HeaderDiff propagation | Partial (engagement diffs work, neighborhood diffs planned)
BlobHeaderDiff message (engagement) | Complete
Reactions (public + private encrypted) | Complete
Comments + author policy enforcement | Complete
Engagement sync via BlobHeaderRequest after pull sync | Complete
Notification settings (messages/posts/nearby) | Complete
Tiered DM polling (recency-based schedule) | Complete
Auto-sync on follow | Complete
Post CDN tree (post_downstream) | Complete
Anchor self-verification (reachability probe) | Complete
Mutual mesh blacklist | Planned
`--max-mesh` flag (test affordance) | Planned
Audience sharding | Planned
Custom feeds | Planned
HTTP post delivery (TCP listener, single route, load shedding) | Complete
Share link generation (postid + author NodeId) | Complete
itsgoin.net QUIC proxy handler (on-demand fetch + render) | Complete
PostFetch (0xD4/0xD5) single-post retrieval | Complete
Universal Links / App Links (itsgoin.net/p/*) | Planned
itsgoin.net ItsGoin node (anchor + web handler) | Complete
UPnP TCP port mapping alongside UDP | Complete

## Appendix D: Critical Path Forward

The highest-impact items, in priority order:

### 1. Three-layer separation (pull sync from social/file, not mesh)

Implement Self Last Encounter tracking and move pull sync to social + upstream file peers. This is the foundation for the layered architecture.

### 2. N+10 in all identification

Add N+10 (NodeId + 10 preferred peers) to self-identification, post headers, blob headers, and social routes. Dramatically improves findability.

### 3. Keep-alive sessions

Implement social/file connectivity check and keep-alive sessions for peers not reachable within N3. Cross-layer N2/N3 routing from keep-alive sessions.

### 4. UPnP port mapping

Best-effort NAT traversal for desktop/home networks. Makes nodes directly reachable without hole punching. External address feeds into N+10 and all peer advertisements. Especially impactful for mobile-to-desktop connectivity.

### 5. Growth loop reactive trigger

Fire growth loop immediately on N2/N3 receipt until 90% full. Currently only timer-based.

### 6. Multi-device identity

Same identity key across devices with device-specific identity for self-discovery and own-device relay.

### 7. File-chain propagation

Make AuthorManifest with N+10 and recent posts work passively. Enable discovery of new content from any blob holder.

### 8. Share links + HTTP post delivery

The viral growth mechanism. Every share becomes a product demo for non-app users and opens natively for app users. Dependencies in order:

1. UPnP TCP mapping (small addition to existing UPnP code)
2. Raw TCP HTTP listener (150–200 lines, zero new dependencies)
3. Host list generation at share time (query post_downstream, encode, embed in URL)
4. itsgoin.net redirect handler + known_good DB (server-side, independent of app releases)
5. itsgoin.net loading screen
6. Universal Links / App Links registration (static JSON files + Tauri config)
7. itsgoin.net ItsGoin node (run the binary, configure as anchor)

Steps 4–7 are itsgoin.net infrastructure, deployable independently of app releases. Steps 1–3 ship in the app. Step 6 requires an app store release to activate but can be deployed to itsgoin.net ahead of time.

### 9. Own-device relay restriction

Restrict relay pipes to own-device by default, opt-in for relaying for others.

## Appendix E: Features Designed But Not Built

Feature | Source | Status
--- | --- | ---
Three-layer pull sync (social/file, not mesh) | v0.2.0 design | Planned
N+10 in all identification & headers | v0.2.0 design | Planned
Keep-alive sessions | v0.2.0 design | Planned
Multi-device identity | v0.2.0 design | Planned
Own-device relay restriction | v0.2.0 design | Planned
Self Last Encounter sync trigger | v0.2.0 design | Planned
Anchor pin vs Fork pin distinction | project discussion.txt | Planned
Audience sharding for groups > 250 | ARCHITECTURE.md | Planned
Repost as first-class post type | project discussion.txt | Planned
Custom feeds (keyword/media/family rules) | project discussion.txt | Planned
Bounce routing (social graph as routing) | ARCHITECTURE.md | Planned
Reactions (public + private encrypted) | v0.2.11 | Complete
RefuseRedirect handling (retry suggested peer) | protocol.rs | Partial (send-only)
Profile anchor list used for discovery | ARCHITECTURE.md | Partial (field exists)
File-chain propagation (passive post discovery) | Design | Partial (manifest exists)
Anchor-to-anchor gossip/registry | Observed gap | Planned
BlobHeader as separate mutable structure | v0.2.11 | Complete
BlobHeaderDiff incremental propagation (engagement) | v0.2.11 | Complete
Post export/backup tooling (author durability) | v0.2.4 design | Planned
Anchor reachability probe (self-verification) | v0.2.6 | Complete
Mutual mesh blacklist | v0.2.4 design | Planned
`--max-mesh` flag (test topology control) | v0.2.4 design | Planned
Relay-assisted port scanning (advanced NAT traversal) | v0.2.6 | Complete

## Appendix F: File Map

    crates/core/
      src/
        lib.rs          — module registration, parse_connect_string, parse_node_id_hex
        types.rs        — Post, PostId, NodeId, PublicProfile, PostVisibility, WrappedKey,
                          VisibilityIntent, Circle, PeerRecord, Attachment
        content.rs      — compute_post_id (BLAKE3), verify_post_id
        crypto.rs       — X25519 key conversion, DH, encrypt_post, decrypt_post, BLAKE3 KDF
        blob.rs         — BlobStore, compute_blob_id, verify_blob
        storage.rs      — SQLite: posts, peers, follows, profiles, circles, circle_members,
                          mesh_peers, reachable_n2/n3, social_routes, blobs, group_keys,
                          preferred_peers, known_anchors; auto-migration
        protocol.rs     — MessageType enum (39 types), ALPN (itsgoin/3),
                          length-prefixed JSON framing, read/write helpers
        connection.rs   — ConnectionManager + ConnHandle/ConnectionActor (actor pattern):
                          mesh QUIC connections (MeshConnection), session connections,
                          slot management, initial exchange, N1/N2 diff broadcast,
                          pull sync, relay introduction. All external access via ConnHandle.
        network.rs      — iroh Endpoint, accept loop, connect_to_peer,
                          connect_by_node_id (7-step cascade), mDNS discovery
        node.rs         — Node struct (ties identity + storage + network), post CRUD,
                          follow/unfollow, profile CRUD, circle CRUD, encrypted post creation,
                          startup cycles, bootstrap, anchor register cycle
        web.rs          — itsgoin.net web handler: QUIC proxy for share links,
                          on-demand post fetch via content search, blob serving
        http.rs         — HTML rendering for shared posts (render_post_html)

    crates/cli/
      src/main.rs       — interactive REPL + anchor mode (--bind, --daemon, --web)

    crates/tauri-app/
      src/lib.rs        — Tauri v2 commands (38 IPC handlers), DTOs

    frontend/
      index.html        — single-page UI: 5 tabs (Feed / My Posts / People / Messages / Settings)
      app.js            — Tauri invoke calls, rendering, identicon generator, circle CRUD
      style.css         — dark theme, post cards, visibility badges, transitions

## License

ItsGoin is released under the Apache License, Version 2.0. You may use, modify, and distribute this software freely under the terms of that license.

This is a gift. Use it well.

## Contribute

ItsGoin is open source and built in public. This project was started by a network/tech web dev vibe-coding beyond his skill level with AI assistance. The design is ambitious and the implementation is real — but more hands make lighter work. If any of this interests you, jump in.

Source code on Forgejo Join the Discord