githubnext/ace2Public

Ace: a collaborative coding environment on the team's own machines

Extract native desktop tools into an external, independently testable package

IA

Tracked by #8.

## Status

The extraction is implemented and published. The user authorized creating the public repository; nothing is published to npm, and there's no vendor-signed companion.

- **Package:** [githubnext/desktop-tools](https://github.com/githubnext/desktop-tools) at [`8efe67f`](https://github.com/githubnext/desktop-tools/commit/8efe67fbd38ed293c8e76778f01814b3bd67c5db). - It contains the native runtime and C ABI, the signed client, the 15 unchanged patches and pins, the reproducible build with license notices, the runtime-neutral `protocol` export, and the Node client. - The runtime advertises the protocol capability `dev.githubnext.desktop-tools.protocol.1`. - CI checks `dist/` freshness, imports the modules on Linux, and builds natively on macOS twice into the same output. - **Ace integration:** draft #213 pins the package commit. pi tools, the channel gate, hosted forwarding and the project picker stay in Ace. - **Verified:** - The standalone build, with a second build into the same output. - A fresh Git install of the public SHA (no build step) and both exports. - Handshake, inventory and clipboard read against a signed reference host. - A pre-dispatch refusal from a host without the capability, and handshake refusal of wrong or ad-hoc client signatures. - In signed Ace-dev: the loaded package dylib, a real model run, and a distinct-worker reopen with no new tool calls. - **Pending:** live capture and input checks (fresh click, stale refusal, host restart, held-operation cancellation). Two blockers: - The Mac's GUI session was locked during validation. - The reference host `dev.githubnext.desktop-tools.reference` is waiting for user-granted Accessibility and Screen Recording.

The proposal below is kept as written. The spike criteria still apply; the steps that created the repository and pointed Ace's build at the package are done.

## Recommendation

Move Ace's native desktop engine into an external repository and package that Ace keeps embedding, preserving its macOS grants. First, run a small external spike with a signed reference host and an ordinary Node consumer, without Ace.

A plain library isn't enough: macOS grants Accessibility, Screen Recording and Event Synthesizing to a signed host process, and the bridge accepts only a client from the host's team with the exact identifier.

Who signs depends on how the package is used:

- **Embedding apps** (Ace, the reference host) sign their own host and client pair. - **Consumers of a future vendor-signed companion** need no certificate of their own. That companion is a later decision.

Browser tools (loopback CDP, no native or signing dependency) are out of scope.

## Evidence

All Ace links are pinned to `ad3f731`; Peekaboo links are pinned to v4.8.0.

1. [Embedded runtime with operation, team and client allowlists](https://github.com/githubnext/ace2/blob/ad3f7313ea18f2fd9c5f9e3fb2ca9c5b1df67861/apps/desktop/native/sources/desktop/desktop.swift#L62-L81) and the [client handshake against the host's team](https://github.com/githubnext/ace2/blob/ad3f7313ea18f2fd9c5f9e3fb2ca9c5b1df67861/apps/desktop/native/sources/client/client.swift#L37-L53) 2. [Client signed with an exact identifier](https://github.com/githubnext/ace2/blob/ad3f7313ea18f2fd9c5f9e3fb2ca9c5b1df67861/apps/desktop/scripts/sign.ts#L44-L90) 3. [Pinned patch-stack build and license copy](https://github.com/githubnext/ace2/blob/ad3f7313ea18f2fd9c5f9e3fb2ca9c5b1df67861/apps/desktop/scripts/helper.ts#L151-L245) 4. [Patch semantics](https://github.com/githubnext/ace2/blob/ad3f7313ea18f2fd9c5f9e3fb2ca9c5b1df67861/apps/desktop/native/patches/README.md) 5. [Host adapter couplings: `ACE_DESKTOP_CLIENT` and `config.home`](https://github.com/githubnext/ace2/blob/ad3f7313ea18f2fd9c5f9e3fb2ca9c5b1df67861/apps/host/src/desktop.ts#L92-L105) 6. [Peekaboo's coordination root is the physical per-user `~/.peekaboo`](https://github.com/openclaw/Peekaboo/blob/v4.8.0/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/Support/DesktopOperationLaneCoordinator.swift#L333-L342)

## Package boundary

**Moves to the package:**

| Component | Size | | --- | --- | | Embedded Peekaboo Bridge runtime, with generic lifecycle, status and permission primitives | 174 lines (`desktop.swift`) | | Existing signed client CLI (7 files) | 1,580 lines | | Signing identity | 49 lines | | `Package.swift` / `Package.resolved` | | | 14 Peekaboo patches | 3,601 unified-diff lines, not product LOC | | Patch-stack builder | | | Neutral request/result types | about 210 lines from `packages/channel/src/desktop.ts` | | Explicit-config TypeScript client, derived from the host adapter | `apps/host/src/desktop.ts`, 1,063 lines |

The TypeScript client takes `{ client, socket, deadline }` and an `AbortSignal`, and returns structured outcomes and evidence. It makes no Bun runtime calls; Node support is unproven and resizing uses macOS `sips`.

**Optional adapter:** the pi tool definitions.

**Stays in Ace:**

- the Bun FFI wrapper (`apps/desktop/src/native.ts`) and the project picker (`project.swift`) - the permission and status UI - Ace Helper, the updater and the catalog - the channel desktop setting, hosted workspace forwarding, and pi history

Extract the existing signed client CLI; build no general-purpose CLI, MCP server or cross-platform framework. Upstream [Peekaboo](https://github.com/openclaw/Peekaboo) already ships a CLI and MCP. Windows and Linux aren't promised.

## Compatibility

The patches add eight Bridge operations (literal insert, menu command, clipboard text/image/files read and write) without a protocol-version bump. None exist on upstream main [`f54cc3a`](https://github.com/openclaw/Peekaboo/commit/f54cc3a19f46bd878c5234f668086c4c0bb79c7b), so Ace's bridge is a private dialect.

The package must carry its own protocol and capability identity. Before any mutation, the client verifies the host's version and required capabilities, refusing a mismatch (including an upstream Peekaboo.app host) with zero input.

On upstream main, 12 of 14 patches apply three-way; `peekaboo-menu.patch` fails at `PeekabooBridgeServer+Handlers.swift:115`. Upstreaming the operations would shrink the fork but isn't a prerequisite.

## Constraints

- **One coordination root.** Lane locks, pending-mutation records and the clipboard reservation stay in the shared physical `~/.peekaboo`. A standalone host is not a separate desktop: no second clipboard gate or native authority. - **Snapshots and reservations.** Snapshots die with their host. An unresolved paste reservation survives a host restart. - **No-replay is split.** - Cancellation or a deadline is not a kill. A live client may return an AbortError or a structured `unknown`. It must not report a safe refusal if input may have started. - The native lane stays held until the operation settles, and nothing retries automatically. - Only a killed caller can't return a result. For that case, the durable adapter (pi today) persists the unsafe intent and reports it as `unknown` on recovery, without replaying it. - The bare SDK must not claim crash persistence. - **Structured evidence.** `completed` can mean accepted delivery with an unverified effect; consumers shouldn't parse model text. - **Notices.** Every artifact ships the license files: - Peekaboo, AXorcist and Commander: MIT. - swift-log: Apache-2.0, and its NOTICE.txt must ship too. - swift-algorithms and swift-numerics: Apache-2.0 with the Runtime Library Exception.

The broader notices work stays in #41.

## External spike (next step)

From a clean clone outside the Ace checkout, with Ace and Ace Helper not running, use:

- a tiny signed AppKit reference host - a plain Node consumer with explicit paths and no model credentials - a fixture app that counts the events it receives

**Go** if all of these hold:

- [ ] The native build doesn't depend on Ace, Electrobun or Bun packaging. It needs macOS with the Xcode SDK, Swift 6.2+, Git and an Apple signing identity, and Node only for the TypeScript consumer probe. It reproduces the pinned patch stack or fails on drift. - [ ] A fresh inspect and one click produce exactly one fixture effect. - [ ] Stale AX snapshots, stale pixel snapshots, moved windows, a wrong team or client ID, and a protocol or capability mismatch are all refused with zero events. - [ ] If a consumer is cancelled or hits its deadline after dispatch, it returns an AbortError or `unknown`, never a safe refusal. If it's killed, it returns nothing. Either way, the host holds the lane until the operation settles, nothing retries, and a fresh observation shows at most one effect. - [ ] A host restart rejects old snapshots, and a pending paste reservation still refuses clipboard writes. - [ ] Ace re-embedding the package adds no new macOS prompts and matches Ace's pre-extraction baseline with no new regressions, including self-targeting coverage. The #73 Open Folder picker failure is a known existing bug, recorded as baseline until its own fix lands. This spike doesn't require completing #73. - [ ] pi's adapter records the interrupted intent as `unknown` and doesn't replay it after a distinct-worker reopen.

**No-go** if any of these happen:

- the package needs a second coordination root - embedders can't re-sign the host and client pair - Ace needs a divergent copy of the native code

## Migration after go

1. Create the external repository and move the boundary above into it. 2. Point Ace's build at the package; Ace-specific wrappers stay in Ace. 3. Decide separately on publishing and a vendor-signed companion.

  1. iamnbutlerreferenced this in [Meta] Complete native computer use in Ace #8last month on GitHub

  2. iamnbutlerparent-issue-addedlast month on GitHub

  3. iamnbutlerreferenced this in feat(desktop): consume native desktop tools from githubnext/desktop-tools #213last month on GitHub