githubnext/chopinPublic

Let's compose rich plans together

Move workers to structured output and add harness deployment modes #194

Mergedlavaman131merged 1 commit intofeat/harness-3-plannerfromrefs/pull/194/head
  1. LA

    Part of stack #202 (#181 harness).

    Part of #181. This is slice 4 of 5 stacked PRs for the pluggable agent harness RFC. It covers RFC phase 6 (workers on structured output) and phase 7 (deployment modes), plus the matching docs. Phase 9, the author's own runtime adapter, is deferred.

    Stacked on #192 (`feat/harness-3-planner`). Review #186, #187 and #192 first; this diff contains only the slice 4 commit.

    ## What changes

    - **Workers on structured output.** `harness/agents.ts` adds `summaryAgent` and `researchAgent` with `Output.object` schemas. `jobs/document-summary.ts` and `jobs/research-workspace.ts` now consume the parsed `result.output` instead of scraping text from a Copilot session. - The pinned `HarnessAgent` fixes the output schema per agent, so each research stage gets its own private stage agent. - **Host `web_search`.** `harness/web-search.ts` binds the research tool to GitHub's hosted MCP `web_search` (`api.githubcopilot.com/mcp/`, `X-MCP-Readonly: true`, `X-MCP-Toolsets: web_search`). The tool takes `{query}`, and source provenance reads the array form of MCP `content`. - **Credit limits per harness session.** `registerCredential` now takes an optional `maxAiCredits`. For a limited session, the Copilot adapter opens one native session on the first turn and reuses it. All description chunks and reductions therefore share the job's registered 64-credit limit, and each research stage gets a fresh 30-credit session. The adapter refuses the next turn, rather than opening a new credit window, after: - a settings or token change between turns, - a concurrent turn, - a failed or aborted turn.

    Unlimited Planner turns keep one native session per turn, and the adapter now discards the previous one before opening the next. - **Deployment modes.** `config.ts` adds `HARNESS` (default `copilot-sdk`) and `HARNESS_AUTH`; `AGENT` and `MODEL` keep their meaning. `harnessFor` refuses at startup: - an unknown harness; - any auth mode other than `direct` or `ai-gateway` unless `SERVER_HOST` is loopback, so a host-login fallback such as `auto` never lends the server's own subscription to users. - **Deleted.** The temporary `harness/copilot-sdk/workers.ts` from slice 3 (`openWorker`, `openPublicResearchWorker`, `terminalGate`, `publicResearchGate`, `PUBLIC_WEB_SEARCH_*`, and the `Tool` re-export), along with its tests and the separate worker runtime. `@github/copilot-sdk` is now imported only under `apps/server/src/harness/copilot-sdk/`, and the lint restriction has no exceptions. - **Docs.** `docs/hosted-agent.md`, `docs/self-hosting.md` (harness selection, adapter trust, shared-key billing with no Chopin quotas), `docs/architecture.md`, `docs/background-jobs.md`, `AGENTS.md`, and `.env.example`.

    ## Notes for reviewers

    - The shipped Copilot adapter accepts `HARNESS_AUTH` but keeps using each owner's App token. The variable matters for the startup check and for future adapters. - The adapter keeps a JSON fingerprint of the turn settings, including the owner token, in process memory so it can detect drift. That fingerprint is never logged, and error messages leave it out. - The pinned SDK doesn't define the accounting window for `maxAiCredits`. This PR restores the base branch's guarantee (one capped session per job stage) and makes no stronger claim.

    Commit: - `c4e9fe6` Run background workers through bounded HarnessAgent sessions

    ## Temporary pieces

    None. This slice removes the worker bridge that slice 3 kept temporarily. Slice 5 adds the Pi adapter and E2E harness coverage; this PR has nothing that slice 5 deletes.

    ## Interactive Edge verification

    I ran these on a signed-in local build in Microsoft Edge with `github-copilot/gpt-6-luna`.

    **Document description through `summaryAgent`.** Before the job ran, the `red-summit` reference picker showed "Empty document":

    ![Before the summary job: red-summit shows "Empty document"](https://github.com/user-attachments/assets/ea8d210c-0834-4f14-9567-1aeeec5b1478)

    After the job completed, search and the sidebar showed the generated description "Note about verifying generated descriptions in the search ref…":

    ![After the summary job: generated description in search and the sidebar](https://github.com/user-attachments/assets/523c47c3-9185-4cb3-aee4-9a7ce11dba7e)

    **Multi-turn description under one credit limit.** A 25,336-byte `flowering-cloud` document took at least seven description turns (chunks and reductions) through one native Copilot session. It got the description "Descriptive prose about seasonal change in a meadow":

    ![Long document described through one limited session](https://github.com/user-attachments/assets/79b0c5f8-4ca0-475f-afa3-7c4f98e967ba)

    **`/research` through the harness path and host `web_search`.** The brief "Find public documentation on AI SDK Harnesses structured output and cite one source" first showed as a pending inline card:

    ![Research request pending as an inline card](https://github.com/user-attachments/assets/9ab503f1-1229-4c0a-9082-fb774a13e2cc)

    It then reconciled into a ready card and a nested child row:

    ![Research ready card with one source](https://github.com/user-attachments/assets/b84665c5-9330-4360-82d4-f08637172fee)

    The child document holds findings with a cited source, "AI SDK Harnesses: HarnessAgent":

    ![Published child document with Findings and Sources](https://github.com/user-attachments/assets/9455bf71-edc0-42a1-aa6c-677cd84c730a)

    I checked the screen recordings with ffmpeg `freezedetect`. They're mostly frozen frames (long idle waits, and a frozen tail through the end), so I didn't upload them. Two other recordings in the evidence directory are from invalid attempts, and I didn't upload those either: one where the Planner produced a summary instead of the worker, and an early research attempt.

    ## Terminal verification

    - `HARNESS_AUTH=auto` with `SERVER_HOST=0.0.0.0`, `::` or `192.168.1.2`: startup exits 1 with "HARNESS_AUTH requires a loopback SERVER_HOST". `HARNESS=bogus` also exits 1. - `HARNESS_AUTH=auto` on `127.0.0.1:8787`: the server starts and serves HTTP 200.

    ## E2E note

    During development, the unflagged parallel E2E suite (`AGENT=off`) passed 226/230 and 227/230 because of UI timing failures. `--workers=1` passed 230/230, and the CI profile passed with 3 specs that recovered on retry. The final deterministic gate below passed.

    Assistant-model: GPT-6 Sol, Claude Opus 5.5 Assistant-workflow: harness-stack (runs e1e96553, 917fbe31, and this run; run ids in the workflow status) Assistant-verification: bun run ci passed: Found 1 warning and 0 errors. | Finished in 68ms on 497 files with 121 rules using 16 threads. | tokens ok — 227 defined across 21 stylesheets Assistant-verification: bun run types passed: @chopin/visuals types: Exited with code 0 | @chopin/editor types: Exited with code 0 | @chopin/server types: Exited with code 0 | @chopin/web types: Exited with code 0 Assistant-verification: bun test passed: 2 skip | 0 fail | 6591 expect() calls | Ran 1583 tests across 179 files. [10.35s] Assistant-verification: mise exec -- bun run e2e passed: Container chopin-harness-4-db-e2e-fixtures-1 Removing | Container chopin-harness-4-db-e2e-1 Removing | Container chopin-harness-4-db-e2e-fixtures-1 Removed | Container chopin-harness-4-db-e2e-1 Removed Assistant-verification: Bun focused regression passed: one 64-credit native session across two parsed turns failed before repair (two sessions) and passed after; adapter tests cover drift, abort, error and unlimited behavior Assistant-verification: terminal startup passed: HARNESS_AUTH=auto refused on 0.0.0.0, ::, 192.168.1.2 and HARNESS=bogus refused; served HTTP 200 on 127.0.0.1:8787 Assistant-verification: agent-browser Edge manual scenario passed: summaryAgent replaced "Empty document" with a generated description; 25,336-byte document described through one limited native session; /research published a sourced child document Assistant-verification: exact-case link check passed: documentation links and anchors resolve Assistant-verification: ffmpeg freezedetect failed: summary and research recordings are mostly frozen, so no video is attached Co-authored-by: @lavaman131

  2. lavaman131mentionedlast month on GitHub

  3. lavaman131subscribedlast month on GitHub

  4. lavaman131added-to-stacklast month on GitHub

  5. lavaman131referenced this in Add the Pi harness adapter and E2E harness coverage #195last month on GitHub

  6. lavaman131ready-for-reviewlast month on GitHub

  7. MA

    Not blocking but some duplication: summary worker [session setup](https://github.com/githubnext/chopin/blob/c4e9fe60bf2683cfa533ffc86288191410f88910/apps/server/src/jobs/document-summary.ts#L223-L245) and [cleanup](https://github.com/githubnext/chopin/blob/c4e9fe60bf2683cfa533ffc86288191410f88910/apps/server/src/jobs/document-summary.ts#L306-L315) is the same as research worker’s [setup](https://github.com/githubnext/chopin/blob/c4e9fe60bf2683cfa533ffc86288191410f88910/apps/server/src/jobs/research-workspace.ts#L892-L907) and [cleanup](https://github.com/githubnext/chopin/blob/c4e9fe60bf2683cfa533ffc86288191410f88910/apps/server/src/jobs/research-workspace.ts#L947-L959)

    could extract into a helper.

  8. MA

    No review summary.

  9. LA
    Author

    @MaggieAppleton Agreed. The summary and research workers repeat the same session open (the abort race plus destroy-on-late-open) and the same destroy → sandbox → release cleanup chain. I'll pull that into a shared worker-session helper in a follow-up PR. That way this approved PR and the stacked #195/#196 don't need another restack.

    Assistant-workflow: inline

  10. MaggieAppletonmentionedlast month on GitHub

  11. MaggieAppletonsubscribedlast month on GitHub

  12. lavaman131referenced this in Share worker session setup and teardown between background jobs #198last month on GitHub

  13. LA
    Author

    Follow-up is up as #198, stacked on #196: `openWorkerSession` in `jobs/worker-session.ts` now handles setup and teardown for both workers.

    Assistant-workflow: inline

  14. lavaman131removed-from-stacklast month on GitHub

  15. lavaman131base-ref-force-pushedlast month on GitHub

  16. lavaman131head-ref-force-pushed 7dcc88d34281last month on GitHub

  17. lavaman131added-to-stacklast month on GitHub

  18. lavaman131merged 377402d237eblast month on GitHub

  19. lavaman131closedlast month on GitHub

  20. lavaman131head-ref-deletedlast month on GitHub