denoland/celldPublic

self-hosted, distributed Durable Objects

v0.6.1

Upgrading from v0.6.0 can use a rolling update.

Setting `CELLD_LTX_RETENTION_SECS`, deploying a Python Worker, or raising
`CELLD_MAX_ASSET_FILE_BYTES` above 25 MiB can be done only after every node
runs v0.6.1.

- Add support for Python Workers.
- Bucket usage can now track a cell's data instead of growing with every
  activation: set `CELLD_LTX_RETENTION_SECS` to delete epochs that no restore
  reads. It is off by default.
- New `celld cell gc --dry-run` lists the superseded epoch prefixes that
  epoch GC can delete, with their sizes, and writes nothing.
- New `CELLD_MAX_ASSET_FILE_BYTES` (default 25 MiB) and
  `CELLD_MAX_DYNAMIC_WORKER_CODE_BYTES` (default 64 MiB) make the per-file
  asset limit and the Dynamic Worker module size limit configurable.
- Console log records now carry the OpenTelemetry severity (`DEBUG`, `INFO`,
  `WARN`, `ERROR`) in the OTLP export and as Parquet columns.
- WebSocket message handlers on one socket now start in arrival order and do
  not wait for the previous handler to finish. Hibernatable sockets keep send
  order across WebSocket handlers and RPC methods.
- A loaded Worker is now released once no stub, entrypoint,
  class or running facet refers to it, and a later `get()` runs `getCode`
  again. `dispose()` releases it explicitly.
- Durable Object facets: `ctx.id` keeps the name of a named
  `DurableObjectId`, and `ctx.exports.App({ props })` passes startup
  properties.
- The synchronous `ctx.storage.kv.list()` iterator releases its read between
  steps, so an unfinished iterator no longer blocks later writes after a
  replication checkpoint.

Issues fixed:

- #233: An unfinished `kv.list()` iterator no longer blocks later writes.
- #236: Hibernatable sockets keep send order across handlers and `send()`.
- #237: Facets started through `ctx.exports` keep `ctx.id.name`.
- #240: Epoch GC deletes superseded epochs (`CELLD_LTX_RETENTION_SECS`).
- #241: OTLP log records carry the console method's severity.
- #242: `webSocketMessage()` no longer blocks later messages on a connection.

Co-authored-by: Ryan Dahl <ry@tinyclouds.org>
Co-authored-by: Yusuke Tanaka <yusuktan@maguro.dev>
Co-authored-by: Nathan Whitaker <nathan@deno.com>
Co-authored-by: Kyle Kelley <rgbkrk@gmail.com>
KKNWYTRDBIBartek Iwańczuk and 4 others authored
f2bf648Patch
Parentbad4649
65 files changed+4677-1042
StackedSplit
.github/workflows/release.yml+1 −1
Cargo.lock+1 −1
Dockerfile+1 −1
crates/celld/Cargo.toml+1 −1
crates/celld/actor.rs+306 −333
crates/celld/actor/effects.rs+257 −0
crates/celld/assets.rs+13 −4
crates/celld/asyncrt.rs+12 −3
crates/celld/cell_cli.rs+270 −3
crates/celld/cell_dispatch.rs+379 −0
crates/celld/cell_host.rs+18 −0
crates/celld/control_plane.rs+1 −4
crates/celld/deploy.rs+30 −8
crates/celld/deploy/python.rs+677 −0
crates/celld/engine_api.rs+7 −1
crates/celld/env_vars.rs+22 −0
crates/celld/host_channels.rs+77 −0
crates/celld/host_services.rs+29 −1
crates/celld/js.rs+291 −100
crates/celld/js/bootstrap.rs+7 −5
crates/celld/js/harness.js+90 −36
crates/celld/js/modules.rs+16 −5
crates/celld/js/websocket.rs+8 −11
crates/celld/lib.rs+1 −0
crates/celld/ltx_repl.rs+538 −52
crates/celld/main.rs+100 −244
crates/celld/main/cli.rs+4 −0
crates/celld/main/websocket.rs+43 −2
crates/celld/node_log.rs+145 −18
crates/celld/otlp.rs+4 −3
crates/celld/ownership_store.rs+35 −60
crates/celld/peer_auth.rs+1 −1
crates/celld/peer_probe.rs+1 −1
crates/celld/pool.rs+72 −0
crates/celld/protocol.rs+6 −0
crates/celld/python/fatal.js+34 −0
crates/celld/python/shims.js+85 −0
crates/celld/python/worker.js+212 −0
crates/celld/python/worker.py+69 −0
crates/celld/runtime.rs+42 −3
crates/celld/storage.rs+73 −100
crates/celld/telemetry.rs+50 −0
crates/celld/ws_client.rs+3 −3
crates/celld/ws_registry.rs+167 −11
crates/logic/epoch_gc.rs+98 −0
crates/logic/lib.rs+4 −1
crates/ltx/src/paged_vfs.rs+5 −0
crates/ltx/src/replica.rs+12 −1
docs/README.md+65 −2
docs/cloudflare-compat.md+1 −1
docs/guarantees.md+35 −0
docs/limitations.md+1 −1
docs/security.md+1 −1
docs/services/durable-object-facets.md+13 −3
docs/services/durable-objects.md+15 −2
docs/services/dynamic-workers-flow.svg+2 −2
docs/services/dynamic-workers.md+22 −10
docs/services/static-assets.md+9 −2
docs/services/workers.md+114 −0
docs/telemetry.md+22 −0
examples/README.md+2 −0
examples/python/.gitignore+3 −0
examples/python/pyproject.toml+8 −0
examples/python/src/entry.py+34 −0
examples/python/wrangler.jsonc+12 −0