## Summary

Each activation of a cell, a local wake from idle eviction included, starts a new epoch, and the new epoch writes a whole-database image to the bucket. Nothing ever deletes an older epoch, although a restore stops at the newest epoch that opens at TXID 1 and never reads below it. Bucket usage therefore grows with (database size × number of activations), forever, rather than with the data.

On our two fleets (`CELLD_IDLE_EVICT_S=180`), 158 GB of the 162 GB under `cells/` is superseded epochs that no restore can reach. The README says an inactive cell "costs almost zero" (docs/README.md:35). On our fleet, one cell woken every 10 minutes holds 38 GB for a 14.5 MB database.

## Environment

- celld v0.5.0 on both fleets. I also read v0.6.0 (`bad4649`); nothing below changes there. - Three nodes per fleet, default `CELLD_DURABILITY=fleet`, `CELLD_IDLE_EVICT_S=180`, compaction on (default). No epoch pages: Fleet A sets `CELLD_LTX_PAGED=0`, and Fleet B's databases are all under 40 MB. - Store: Tigris (S3-compatible). - Fleet A serves an agent Durable Object (a SQLite conversation plus a workspace file system). Fleet B serves an append-only stream Durable Object (the durable-streams Cloudflare server).

## What we measured

A read-only listing of every key and size in both buckets on 2026-09-29, grouped by `cells/<cell>/ltx/e<epoch>/<level>/`:

| | Fleet A | Fleet B | |---|---|---| | Size under `cells/` | 110.9 GB | 51.3 GB | | Cells / epochs | 637 / 8,372 | 693 / 7,687 | | Newest whole-database image per cell, summed | 2.07 GB | 0.40 GB | | Objects in epochs below each cell's newest epoch | 106.96 GB (96.5%) | 50.69 GB (98.9%) | | New objects per day, last four days | 12–15 GB | 2.6–3.5 GB | | New epochs per day, last four days | 240–280 | 190–230 |

Every cell's newest epoch holds a whole-database image of its own (an L0 or L1 starting at TXID 1, or a level-9 handoff snapshot), so for every cell everything below its newest epoch is unreachable.

One epoch per activation: on 2026-09-28 the fleets logged 242 and 230 `cell_runtime_publication` events, and 241 and 230 new epoch prefixes appeared that day. Where a cell's epoch numbers have gaps, they are in the middle of its history, never at the start, so nothing has been cleaned up.

### 1. A small cell woken every 10 minutes (Fleet B)

A health check appends one message every 10 minutes. The cell is evicted 3 minutes later and publishes a handoff snapshot of its whole database; the next check activates it in a new epoch.

``` published authoritative handoff snapshot event="ltx_handoff_snapshot" cell="B:fe93e4bd…" epoch=2924 max_txid=12 bytes=14335535 elapsed_ms=381 final durability barrier passed event="eviction_durability_barrier" cell="B:fe93e4bd…" epoch=2924 artifact=Snapshot elapsed_ms=645 ```

The cell has 3,035 epochs and 38.4 GB, almost all of it level-9 images of 10.6–14.5 MB, one per epoch. Its database is 14.5 MB. That is about 2 GB a day, and each copy is a little larger than the last.

### 2. A large cell woken on a person's schedule (Fleet A)

This database is 170–178 MB, above the ~80 MiB handoff-snapshot budget (crates/celld/ltx_repl.rs:881), so eviction drains the L0 chain instead of publishing a snapshot. Each local wake (same node, `fresh=false`) opens a new epoch whose first object is the whole database at TXID 1. The first compaction then copies it again into L1:

``` 17:38:38 reused local eviction snapshot cell="A:1709d6cd…" epoch=114 17:38:38 cell isolate startup completed event="cell_isolate_startup_timing" outcome="ready" node=node_9461… epoch=114 fresh=false total_us=224804 17:38:48 compacted an additive LTX level event="ltx_compaction" trigger="threshold" cell=A:1709d6cd… epoch=114 source_level=0 destination_level=1 min_txid=1 max_txid=1 input_objects=1 input_bytes=169751673 local_input_objects=1 remote_input_objects=0 output_bytes=169463964 elapsed_ms=4998 result="ok" 17:44:26 final durability barrier passed event="eviction_durability_barrier" cell="A:1709d6cd…" epoch=114 artifact=L0Chain elapsed_ms=1872 17:46:07 reused local eviction snapshot cell="A:1709d6cd…" epoch=115 17:46:20 compacted an additive LTX level event="ltx_compaction" trigger="threshold" cell=A:1709d6cd… epoch=115 source_level=0 destination_level=1 min_txid=1 max_txid=1 input_objects=1 input_bytes=174874142 output_bytes=174589121 elapsed_ms=5203 result="ok" 17:51:48 reused local eviction snapshot cell="A:1709d6cd…" epoch=116 ```

From 2026-09-26 to 09-28, this cell's transactions after each epoch's opening image added up to about 0.2 GB. In the same days, 17.7 GB of new objects landed in its prefix, about 350 MB per activation. The cell holds 27 GB for a 178 MB database.

## Why (v0.6.0 source)

- Every activation advances the epoch, "a takeover and a local wake alike" (docs/guarantees.md:130), and "A full activation opens its epoch with a whole-database snapshot" (docs/README.md:1055). A local wake reuses the evicted file (crates/celld/ltx_repl.rs:1778), yet its epoch still opens with the whole database at TXID 1, as the second log shows. - A restore walks down from the newest epoch and stops at the first one that opens at TXID 1 (crates/ltx/src/client/epochs.rs:49-76). Everything below it is never read again. - Nothing deletes LTX objects. Outside the `ReplicaClient` implementations, which only forward the call (crates/ltx/src/client/bundle.rs:184-193, crates/ltx/src/client/epochs.rs:163-175, crates/celld/ltx_repl.rs:160-166), nothing calls `delete_ltx_files` or `delete_all`. The compactor "never deletes a source" (crates/ltx/src/replica_compactor.rs:4). The docs say "Compaction keeps the source objects" (docs/README.md:1067). The port lists retention as "not implemented" (crates/ltx/src/db.rs:46). The one new bucket delete in v0.6.0, `delete_streams` (crates/celld/ltx_repl.rs:2852), runs only for `ctx.facets.delete()`. - Litestream, which the compactor is ported from, does enforce retention: snapshots are kept 24 hours, and L0 files 5 minutes after compaction into L1, checked hourly (`store.go:60-72`, `EnforceRetentionByTXID` at `compactor.go:293`, benbjohnson/litestream@3b91159).

## Expected

Bucket usage in proportion to the live data, plus whatever history window the operator chooses. A cell that has been activated thousands of times should still cost close to its database size once it is inactive.

## Possible shape (not a requirement)

1. Delete superseded epochs. Once an epoch's opening image is durable and older owners are fenced, the epochs below it are unreachable, so they can be deleted after a grace period. 2. Within an epoch, delete L0 objects covered by a published L1 after a retention window, as Litestream does. 3. When a local wake reuses an eviction snapshot that already equals the bucket state, continue the previous chain (like a paged activation's marker, docs/README.md:1055) instead of uploading the whole database again. 4. A supported way to reclaim what existing buckets already hold, for example `celld cell gc --bucket …`. A retention setting would also define the history that #125 (point-in-time restore) needs.

## Questions

Until this is fixed, we plan an out-of-band cleanup. Per cell, it would delete only epochs below the newest epoch that opens at TXID 1, and only epochs older than a day. Is that safe while the fleet runs? In particular:

- Can dead-node recovery or the tail fold at activation (crates/celld/ltx_repl.rs:1790-1799) still write into, or need to read, an epoch below the newest TXID-1 epoch? - Is anything else under `cells/<cell>/` (for example `own.json`) tied to an older epoch?

Related: #175 (delete a cell and reclaim its storage), #181 (prune old deployments), #188 (storage usage), #125 (point-in-time restore).