denoland/celldPublic

self-hosted, distributed Durable Objects

v0.3.0

- v0.3.0 adds a replicated write-behind log. Fleets of two or more nodes
  acknowledge writes after peer fsync while bucket upload finishes later. In
  testing, this delivers 10× lower write latency and over 100× fewer Class A
  S3 operations. A v0.2.1 fleet can roll forward one node at a time.

- Added support for Azure Blob Storage (`az://<BUCKET_NAME>`).

- Added support for D1 databases, including the `celld d1` CLI for migrations
  and SQL execution.

- Added support for cron triggers, with durable fleet-wide scheduling, retries
  for failed handlers, and recovery of one missed run after downtime.

- Streaming and rewrapping request bodies are up to 70× faster, and text
  decoding is up to 44× faster.

- WebSocket upgrades now work correctly for Durable Object subrequests and
  cross-script service bindings.

- `setInterval()` now works correctly.

- `celld deploy` now accepts Node built-in specifiers (E.G. `node:util`)

- Each isolate now has a configurable V8 heap limit. celld refuses new
  hibernatable WebSockets and oversized SQL materialization near that limit,
  then resumes service after memory use falls.

- Correctness fixes prevent overlapping alarm handlers, events from resuming in
  freed isolates, lost transactions at replication segment boundaries, and
  database corruption after eviction and reactivation.

- Closed issues: [#139](https://github.com/denoland/celld/issues/139),
  [#144](https://github.com/denoland/celld/issues/144),
  [#147](https://github.com/denoland/celld/issues/147),
  [#149](https://github.com/denoland/celld/issues/149),
  [#150](https://github.com/denoland/celld/issues/150),
  [#152](https://github.com/denoland/celld/issues/152),
  [#154](https://github.com/denoland/celld/issues/154),
  [#155](https://github.com/denoland/celld/issues/155),
  [#156](https://github.com/denoland/celld/issues/156),
  [#157](https://github.com/denoland/celld/issues/157), and
  [#158](https://github.com/denoland/celld/issues/158).

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Co-authored-by: Ben Plotnick <bplotnick@users.noreply.github.com>
Co-authored-by: Nathan Whitaker <17734409+nathanwhit@users.noreply.github.com>
Co-authored-by: Yusuke Tanaka <yusuktan@maguro.dev>
Co-authored-by: magurotuna <23649474+magurotuna@users.noreply.github.com>
MAYTNWBPBIRDRyan Dahl and 5 others authored
89e4ffcPatch
Parentae8fac0
97 files changed+18642-5262
StackedSplit
Cargo.lock+16 −1
Cargo.toml+2 −1
README.md+59 −9
clippy.toml+85 −0
crates/celld/Cargo.toml+7 −2
crates/celld/actor.rs+2658 −0
crates/celld/actor/production.rs+44 −0
crates/celld/assets.rs+4 −0
crates/celld/asyncrt.rs+329 −37
crates/celld/bucket.rs+417 −86
crates/celld/control_plane.rs+54 −51
crates/celld/d1_cli.rs+657 −0
crates/celld/dead_node_gc.rs+63 −22
crates/celld/deploy.rs+282 −7
crates/celld/env_vars.rs+0 −6
crates/celld/fault.rs+0 −288
crates/celld/fleet.rs+34 −5
crates/celld/host_services.rs+209 −0
crates/celld/js.rs+679 −124
crates/celld/js/bootstrap.rs+49 −3
crates/celld/js/crypto.rs+73 −7
crates/celld/js/harness.js+609 −25
crates/celld/js/modules.rs+2 −42
crates/celld/js/node_crypto.js+34 −4
crates/celld/js/storage_ops.rs+14 −0
crates/celld/js/text_encoding.js+59 −228
crates/celld/js/websocket.rs+213 −29
crates/celld/lib.rs+115 −9
crates/celld/ltx_repl.rs+1088 −271
crates/celld/machine.rs+24 −82
crates/celld/main.rs+862 −2749
crates/celld/main/cli.rs+15 −6
crates/celld/main/websocket.rs+132 −5
crates/celld/memory.rs+2 −1
crates/celld/node_log.rs+2940 −0
crates/celld/ownership_store.rs+108 −21
crates/celld/peer_auth.rs+4 −0
crates/celld/peer_probe.rs+3 −0
crates/celld/pool.rs+5 −3
crates/celld/protocol.rs+27 −2
crates/celld/replication.rs+310 −53
crates/celld/runtime.rs+341 −66
crates/celld/startup.rs+1 −0
crates/celld/storage.rs+1050 −42
crates/celld/telemetry.rs+3 −0
crates/celld/wake.rs+105 −17
crates/celld/ws_client.rs+3 −0
crates/logic/cell.rs+23 −10
crates/logic/cron.rs+603 −0
crates/logic/isolate.rs+35 −11
crates/logic/lib.rs+853 −531
crates/logic/log_evict.rs+266 −0
crates/logic/log_tier.rs+365 −0
crates/logic/pressure.rs+24 −1
crates/logic/types.rs+104 −67
crates/logic/wake.rs+22 −1
crates/ltx/Cargo.toml+4 −0
crates/ltx/README.md+15 −3
crates/ltx/scripts/capture-golden.sh+6 −6
crates/ltx/scripts/db_equal.sh+1 −1
crates/ltx/src/bundle.rs+190 −0
crates/ltx/src/client/bundle.rs+133 −0
crates/ltx/src/client/file.rs+5 −1
crates/ltx/src/client/mod.rs+1 −0
crates/ltx/src/client/object_store.rs+100 −11
crates/ltx/src/compactor.rs+2 −0
crates/ltx/src/db.rs+381 −119
crates/ltx/src/error.rs+1 −1
crates/ltx/src/host.rs+383 −0
crates/ltx/src/lib.rs+32 −14
crates/ltx/src/ltx.rs+5 −3
crates/ltx/src/replica.rs+66 −28
crates/ltx/src/replica_compactor.rs+31 −9
crates/ltx/src/replica_url.rs+2 −2
crates/ltx/src/wal.rs+3 −1
crates/ltx/tests/conformance.rs+0 −1
crates/ltx/tests/differential_xtool.rs+156 −25
crates/ltx/tests/faults_inject.rs+4 −1
crates/ltx/tests/fixtures/golden/MANIFEST.md+6 −7
crates/ltx/tests/fuzz_bundle.rs+218 −0
crates/ltx/tests/fuzz_parsers.rs+3 −0
crates/ltx/tests/integration_file.rs+48 −14
crates/ltx/tests/integration_resilience.rs+100 −0
crates/ltx/tests/integration_s3.rs+2 −0
crates/ltx/tests/litestream_helpers.rs+6 −7
crates/ltx/tests/property_roundtrip.rs+1 −1
docs/README.md+76 −3
docs/cloudflare-compat.md+220 −21
docs/fencing.md+135 −46
docs/limitations.md+16 −5
docs/security.md+20 −5
docs/testing.md+50 −2
examples/README.md+2 −0
examples/cron/index.js+46 −0
examples/cron/wrangler.jsonc+11 −0
examples/d1/index.js+60 −0
examples/d1/wrangler.jsonc+16 −0