👋 howdy~! Claude-written report below, but it was sent after I reviewed and verified the issue. My workaround now is to hardcode CELLD_NODE values, which works in a happy-path scenario, but dynamically resizing a cluster and/or auto-healing are still prone to peering issues when trying to manipulate the cluster (in any fashion, including celld upgrades) w/ rolling approach

---

With `CELLD_DURABILITY=fleet` and no `CELLD_NODE`, every process start gets a new random node session ID (`random_node_session_id`). After a correlated restart of the whole fleet, node-log recovery cannot seal the logs of sessions that closed without a seal. It retries forever.

## What happened (0.5.1 → 0.6.1, 3 nodes)

1. We stopped all three nodes at the same time. Two nodes waited for the fleet drain token (74 s and 90 s), then hit `CELLD_SHUTDOWN_TOTAL_MS=120000` with their node logs still open (`shutdown_handoff_stalled`, `process_deadline_fired=true`). 2. The new processes started with new random IDs. Each node still had the follower fragments on disk (`/var/lib/celld/peerlog/<old leader>`, 5,291 files, 607 MB). 3. Recovery reads each member's lease and POSTs `/peer/log/seal` to the lease address. The process at that address has a different ID, so peer auth answers `WrongTarget` ("peer request targets a different node session"). Recovery counts an HTTP error as "answered", so the member stays inconclusive:

``` node-log recovery for node_941128…/7a5e…: no complete true witness among {"node_4c022e98…", "node_cad6c84…"} and 2 member(s) undecided; refusing to seal while member state remains unverified ```

The intact follower disks never become witnesses. `docs/guarantees.md` says "nodes that restart together can recover acknowledged writes from their surviving follower disks", but that is true only when each node keeps its ID.

## Suggestion

- Persist the generated node ID in the data directory and reuse it on restart, or refuse to start fleet durability without `CELLD_NODE`. - Document that `CELLD_NODE` must be stable per disk for fleet durability.

We now set `CELLD_NODE` to the VM hostname. When we restarted one node with its old ID, its peers could seal against it. The next step was then blocked by the tail timeout in #250.