Run the CAS fencing contract in CI against SeaweedFS
chrislusf opened ·
Thanks for the exciting project!
fencing.md lists the three properties celld needs from a store, and warns that a store can accept the conditional headers without applying them and fail late and silently. The test that checks this contract, `put_cas_contract_against_real_bucket`, is gated on `CELLD_CAS_LIVE=1` and needs a real bucket, so it never runs in CI — the fencing contract is only exercised at release time against R2, and the local backend the ltx conformance suite runs against does not implement the conditional writes.
I maintain SeaweedFS. Its S3 gateway applies both preconditions atomically — the `If-None-Match: *` / `If-Match` check runs under a per-object write lock, so it is a real compare-and-swap, not the accept-and-ignore failure mode the doc warns about. That makes the fencing contract testable in CI from a single static binary, no Docker and no cloud credentials:
```sh curl -sL https://github.com/seaweedfs/seaweedfs/releases/download/4.41/linux_amd64.tar.gz | tar xz ./weed mini -dir=/tmp/celld-s3 -bucket=litestream -s3.port=8333 & ```
The S3 endpoint is ready in under a second. With release 4.41, both suites pass unchanged:
```sh RUSTYRIVER_MINIO_ENDPOINT=http://127.0.0.1:8333 AWS_ACCESS_KEY_ID=any AWS_SECRET_ACCESS_KEY=any \ cargo test -p celld-ltx --features s3 --test integration_minio # test result: ok. 2 passed (conformance suite + multipart round trip)
CELLD_CAS_LIVE=1 CELLD_CAS_BUCKET=litestream CELLD_CAS_ENDPOINT=http://127.0.0.1:8333 \ AWS_REGION=us-east-1 AWS_ACCESS_KEY_ID=any AWS_SECRET_ACCESS_KEY=any \ cargo test -p celld put_cas_contract -- --nocapture # CAS verified on litestream: create / reject-create / update / reject-stale ```
The raw dialect checks out too: PUT with `If-None-Match: *` returns 200 on an absent key and 412 on an existing one; PUT with `If-Match` returns 200 with the current etag and 412 with a stale one, and the rejected bodies never land. `Bucket::open` already selects path-style with a custom endpoint, so nothing in celld needs to change — this is CI wiring only.
Happy to send a PR with the workflow. Either way, fencing.md's support matrix could mention SeaweedFS as a store that implements the conditional writes.
- AA
I'd be interested in seeing this as well. I have been using tigris for this and have currently been unimpressed on their ability to show me where my usage is (Millions of Class A operations for celld presumably, but not any indication on there end). A self hostable object store would be great to test against
dmartosubscribed2 months ago on GitHub