## Summary

- add a code-registered, PostgreSQL-backed background job service with fenced claims, retries, cancellation, owner revocation, and immutable artifacts - run document summaries and explicitly assigned web research in isolated disposable Copilot sessions with bounded capabilities and no automatic document edits - expose reconnect-safe job status through the protocol, inline research controls, Tasks & Progress, rollout flags, deployment documentation, and browser coverage

## Verification

- `bun test` (995 passed, 2 skipped) - `bun run types` - `bun run ci` - PostgreSQL storage contracts (38 passed) - `bun run build` - local `bun run e2e` (190 passed) - final GitHub validation, E2E, and container jobs passed

## Preview

Manual report: https://github.com/githubnext/chopin/pull/113#issuecomment-5380481594

The live summary worker completed successfully. Live public research could not complete because the dedicated preview account’s connected Copilot endpoint advertised no `web_search`; the worker failed closed before a search call. Completion still needs verification in an entitled environment that offers that capability.

## Preview verification report

**Preview:** https://113-chopin.githubnext.com **PR head:** `8df031c2d515f32862ab0375d344f6ee867937ee` **Outcome:** Passed with one live-provider limitation described below.

### Environment

- Authenticated as the dedicated preview account; Playwright redacted the identity. - Repository picker contained exactly one option, `CHOPIN_PREVIEW_REPOSITORY`, with **Create and edit channels**. - Coolify reported the validated PR host while the PR head was the SHA above. The ready comment does not independently attest the deployed commit. - Final CI: validation passed, E2E passed, container build passed.

### Passed

- Created the disposable `PR 113 background jobs preview` channel and inserted an inline Research Question. - Verified the public-search disclosure, assignment, `owner-unavailable` pause, active-owner recovery after a Planner turn, retry controls, and Tasks status projection. - Verified a fresh document-summary job completed in an isolated worker and its result was readable in Tasks. - Verified Tasks & Progress on desktop at 1440x900 and compact mobile at 390x844. Mobile navigation had four reachable 44px destinations and no horizontal overflow (`scrollWidth === innerWidth === 390`). - Verified the inline and Tasks surfaces remained readable after reloads/redeploys. - Current deployment console: 0 errors. Current HTTP API requests: successful. Historical WebSocket 404/502 errors were limited to expected reconnects across repeated preview redeploys.

### Live provider limitation

- Public research did not complete because this preview account/runtime endpoint did not advertise `web_search`. - The isolated `github-mcp-server` connected, but its live `tools/list` was empty. The exact session filter, metadata audit, and permission gate therefore failed closed before any web-search model call. - Resulting UI state was bounded and recoverable: `failed · attempts-exhausted:attempt-error`, with **Research again** available. - I did not widen the worker to all GitHub MCP tools or a read-write endpoint. A live research completion still needs verification with an account/environment whose Copilot endpoint offers `web_search`.

### Preview-driven fixes

- Corrected custom-tool metadata auditing and the CLI minimum AI-credit limit. - Added bounded MCP readiness and diagnostics. - Switched to the isolated built-in GitHub MCP provider, enabled it only in an empty temporary runtime, blocked ambient plugin discovery, and used the exact `mcp:github-mcp-server-web_search` filter.

### Persisted test data

- One disposable channel, the harmless Bun-version question, scoped Planner acknowledgement messages, and `Preview summary worker retest.` remain in the sandbox.

## Normal-account follow-up

A normal account with GitHub web search enabled reproduced the failure on the previous head. The account reached the built-in MCP server, but Chopin filtered on the pre-normalization name (`mcp:github-mcp-server-web_search`) after Copilot had already hoisted the effective tool to `web_search`.

Commit `a8b7d68` now:

- filters the effective tool as `mcp:web_search` - requires the connected server’s live raw catalog to contain `web_search` - tolerates unrelated raw FIDES offers while still requiring effective metadata to contain exactly the terminal result and `github-mcp-server/web_search` - retains the exact read-only permission gate and citation-source validation

Validation passed: 995 unit tests, types, lint/format/token checks, browser E2E, and container build. Preview head: `a8b7d68f9c3a1d0c200dd3dcd76b4333fcdac66d`.

Live completion now needs a retry with the normal account. If its endpoint still omits the capability, the server log will explicitly report `does not offer web_search; offered …` rather than a generic capability-audit failure.

## Tasks progress follow-up

Commit `c14b215` replaces the unhelpful standalone `running` state with a durable stage timeline in Tasks & Progress.

Research now reports these code-owned stages:

- Public web research - Private document analysis - Research report synthesis

Each stage is shown as **In progress**, **Completed**, or **Interrupted**, and retries are labeled by attempt. The capped 32-entry log survives reload/reconnect and never stores prompts, model prose, document content, URLs, errors, or credentials. Progress writes are claim-generation fenced; cancellation is now atomic and cannot lose a race with progress or heartbeat revisions.

Verification on head `c14b2159e96140f8806e43045aae23e80eca061d`:

- 1000 unit tests passed, 2 skipped - 39 PostgreSQL contract/migration tests passed - types, formatting, lint, and token checks passed - production build passed - all 191 browser tests passed - running-progress browser test verifies stage display and persistence after reload - GitHub validation, E2E, and container jobs passed

## Retry-loop fix

Commit `0422acf` fixes the repeated `Public web research / Attempt N / Interrupted` cycle.

Root cause: the first periodic heartbeat was scheduled before credential resolution. The credential-bound renewal extended the claim farther than that already-scheduled heartbeat could extend it, so storage rejected the heartbeat as non-increasing. The runner detached locally without incrementing failures, then reclaimed the expired job indefinitely.

The fix:

- resets heartbeat timing after credential-bound renewal - uses one strictly increasing renewal horizon - renews before bounded remote owner revalidation - waits for actual execution cleanup before retrying heartbeat loss - fails terminally rather than overlapping an abort-ignoring execution - preserves heartbeat recovery during shutdown

Regression coverage crosses delayed owner resolution, persisted progress, multiple heartbeats, signal-aware retry, abort-ignoring failure, and shutdown during recovery.

Verification on `0422acf53248857527a620499a9ec8f43a3c357e`: 1003 unit tests passed, types/CI passed, and GitHub validation, E2E, and container jobs passed.

## Interruption diagnostics

Commit `e1b1f6d` adds durable, safe explanations to interrupted research stages. New runs now show messages such as:

- `Interrupted · Copilot web search is unavailable` - `Interrupted · Public web research failed` - `Interrupted · Attempt timed out` - `Interrupted · Runner heartbeat was lost` - `Interrupted · Planner owner became unavailable` - `Interrupted · Research sources could not be verified`

Only fixed reason categories are persisted. Raw SDK/provider errors, prompts, document content, URLs, and credentials remain server-only. Retry, pause, timeout, credential rotation, heartbeat recovery, MCP classification, and citation-validation ordering have direct regression coverage.

Verification on `e1b1f6d9c8302e06f39098a54f4fbd68889c10dc`: 1008 unit tests, 39 PostgreSQL tests, build, targeted Chromium progress tests, and all GitHub validation/E2E/container jobs passed.

Existing interruption rows remain generic because their causes were not stored. Cancel the old job and choose **Research again** to get reasoned entries.

## Actionable public-stage diagnostics

Commit `f1f97a8` replaces the remaining umbrella `public-research-failed` path with precise, safe causes:

- web search unavailable, denied, not invoked, or failed - Copilot session failed - structured result missing or invalid - submitted sources mismatched search results - MCP source metadata missing - hosted-search sources unverifiable

The public worker now has exact stage-specific result schemas and explicit tool-only instructions. Genuine successful searches with empty findings/sources are accepted; non-empty evidence remains provenance-checked. Both MCP tool events and provider-hosted search/citation events are tracked. Provenance is snapshotted when the model submits its result, preventing later events from retroactively validating it.

Server diagnostics contain only stage/phase, call/success/failure counts, citation count, result booleans, hosted counters, and permission-denied boolean. They never contain prompts, page content, URLs, provider messages/codes, or credentials.

Verification on `f1f97a8356aafa618cd544364bdaaff178d6f6f5`: 1010 unit tests, types/CI, production build, targeted Chromium progress tests, and all GitHub validation/E2E/container jobs passed.

Cancel the previous terminal job and choose **Research again**. If it still fails, the new log line’s `progressReason` and `diagnostic={...}` identify the exact failing contract.

## Web-search provenance serialization fix

Commit `c85c149` addresses the live `Search results had no verifiable source metadata` failure. Copilot CLI 1.0.80 derives citation sources internally but omits `citableSources` from the serialized `tool.execution_complete` event. The source URLs remain in the successful tool result’s rich citation annotations.

Chopin now recovers provenance from only:

- SDK `citableSources` when present - structured `url_citation` annotations - whole-result JSON carrying those annotations - typed resource/resource-link URIs - provider-hosted assistant citations

It ignores free-form prose, Markdown-like text, MCP metadata, icons, and URLs appearing only in the model submission. Extraction is bounded by text bytes, node count, and 64 stage-wide URLs; every URL passes the existing public-HTTPS policy. No fetch, DNS lookup, redirect, or URL resolution is performed.

Verification on `c85c1490403d06507ad75755fae77b7cfd58f21f`: 1013 unit tests, types/CI, and all GitHub validation/E2E/container jobs passed.

Cancel the prior failed job and choose **Research again** to test this path.

## Background Work tab

Commit `0f1a6e4` restores **Tasks & Progress** as its original disabled desktop placeholder and moves all background-job status, progress, results, and cancellation into a separate active **Background Work** tab.

- Desktop document controls show both concepts separately. - Compact navigation shows Background Work only; the reserved placeholder is not a mobile destination. - `BACKGROUND_JOBS=off` hides Background Work but preserves Tasks & Progress. - Existing saved `tasks` preferences migrate to `background-work`. - Background Work waits for server capability confirmation, avoiding a blank view during connection.

Verification on `0f1a6e46836d462269d25295a29bc0301e29eea2`: 1018 unit tests, types/CI, production build, 19 affected browser tests, and all GitHub validation/E2E/container jobs passed.

## Rebased onto latest main

PR #113 is now based on `59ede82` and integrates with the redesigned global NavigationShell. Background Work remains separate from the reserved Tasks & Progress placeholder.

Post-rebase integration includes a horizontally reachable desktop document tab strip at constrained split widths, container-responsive Background Work gutters, and a compact `Background` visual label with the full `Background Work` accessible name.

Verification on `6f7c5b453aa73cc472d0f43db6b0a571774ac846`: 1039 unit tests, types/CI, production build, all 175 browser tests, and all GitHub validation/E2E/container jobs passed. The local pre-rebase backup remains at `backup/background-agents-runtime-20260823`.