[{"_1":2,"_3":-5,"_4":-5},"loaderData",{"_5":6,"_7":8,"_9":10,"_11":12},"actionData","errors","root",{"_228":-5,"_231":232,"_233":234},"routes/shell",{"_40":41,"_227":-5,"_228":-5,"_229":230},"routes/repo",{"_33":-7,"_34":35,"_36":37,"_38":39,"_40":41,"_42":31,"_43":31,"_44":31,"_45":31,"_46":31,"_47":31,"_48":49,"_50":49,"_51":31,"_52":-7,"_53":-7,"_54":55,"_56":57,"_58":59,"_60":-5,"_61":62,"_63":64,"_65":66,"_67":68},"routes/repo.blob",{"_13":14,"_15":16,"_17":18,"_19":20,"_21":14,"_22":23,"_24":-5,"_25":26,"_27":28,"_28":29,"_30":31,"_32":14},"ref","9d68d2829c382691c1bcb4dc1cd38b8a01109bea","path","docs/background-jobs.md","oid","8ee2bfc2ae9e5956e5b45068c4123006377d5b05","size",35561,"commit","mounts",[],"source","plain",{"_13":14,"_15":16},"kind","text","# Background jobs and workers\n\nChopin's background system runs durable, versioned work outside the shared\nPlanner conversation. A job may execute ordinary code or open one or more\nisolated Copilot worker sessions. The code calls the durable unit a **background\njob**; this document uses **worker** for the disposable execution session. A\nworker is not a child Planner turn, coding agent, or runtime plugin.\n\n> [!IMPORTANT]\n> The registry is a closed, code-owned allowlist. Persisted rows name a\n> registered job type and version; they never contain executable code. Adding a\n> job requires a source change and deployment.\n\nThis guide covers the generic job framework, how to register a new definition,\nand the worker boundaries used by generated document descriptions and research requests.\nSee [Hosted agent](hosted-agent.md) for the Planner conversation and\n[Storage](storage.md) for the broader durable model.\n\n## Vocabulary\n\n| Term | Meaning |\n| ------------------------------- | -------------------------------------------------------------------------------------------------- |\n| Definition | The code-owned `JobDefinition` for one `type@version`. |\n| Job | One durable request, including normalized input and lifecycle state. |\n| Target | The semantic resource a job updates, such as one document or one internal research attempt. |\n| Target generation | The newest request for a target. A newer generation supersedes older active work. |\n| Attempt | One job claim. Credential or capacity failure may prevent its executor from starting. |\n| Failure | An attempt that consumes retry budget. `failures`, not `attempts`, is compared with `maxAttempts`. |\n| Claim generation | A fencing counter that prevents an expired or cancelled worker from publishing late output. |\n| Artifact | The immutable, validated JSON result written only when a job completes. |\n| Worker | A disposable execution context, optionally backed by an isolated Copilot SDK session. |\n| Background-job channel revision | The invalidation counter for job mutations in one channel, separate from each job's revision. |\n\nKeep these counters separate from the Yjs epoch, document sequence, plan\nrevision, storage revision, research request revision, and implementation\ngraph counters.\n\n## Architecture\n\n```mermaid\nflowchart LR\n\tE[Scheduler, user route, or domain service] --> S[JobService]\n\tS -->|validate, normalize, fingerprint| P[(PostgreSQL)]\n\tP -->|fenced claim| R[JobRunner]\n\tR --> C[Credential resolver]\n\tC --> X[Registered executor]\n\tX -->|optional disposable session| A[Copilot worker]\n\tX -->|progress| P\n\tX -->|candidate artifact| S\n\tS -->|validate and settle| P\n\tP --> I[job:changed invalidation]\n\tP --> D[Optional domain reconciliation]\n```\n\nThe main boundaries are:\n\n- `JobRegistry` stores every supported definition version and selects the\n highest version for new work.\n- `JobService` owns origin checks, canonical JSON, byte limits, fingerprints,\n enqueueing, control mutations, artifact validation, and publication hooks.\n- `BackgroundJobStore` owns durable state transitions, target generations,\n claims, progress, and artifacts.\n- `JobRunner` polls and wakes for work, resolves credentials, enforces\n concurrency, heartbeats claims, executes definitions, and handles retries and\n shutdown.\n- A definition's executor owns the actual work and any worker-specific\n capability boundary.\n\nProduct code should enqueue through `JobService`, not call\n`storage.jobs.enqueue()` directly. The raw storage port enforces persistence\ninvariants but does not know registry versions, origins, codecs, or byte limits.\n\n## Registration checklist\n\n1. Choose a stable lowercase type and start at version 1.\n2. Define strict, deterministic input and artifact codecs.\n3. Select only the origins an actual caller needs; enforce authorization at the\n route, socket, scheduler, or tool boundary.\n4. Select `active-planner` or `none` deliberately. For a user-triggered\n `active-planner` job, establish or validate ownership before enqueueing.\n5. Set timeout, failure budget, aggregate AI-credit metadata, and byte limits.\n6. Declare fixed progress stages without private or model-generated text.\n7. Implement execution with cancellation, deadlines, and bounded cleanup.\n8. For model work, use fresh audited workers and structured result tools.\n9. Add a publication hook only when completion needs a freshness gate.\n10. Register every supported version in `apps/server/src/main.ts`.\n11. Add an explicit scheduler, route, domain service, or Planner tool that\n enqueues through the matching `JobService.enqueue*()` method.\n12. Choose stable target and idempotency keys; do not pre-prefix the target.\n13. Add idempotent domain reconciliation if completion creates related state.\n14. Add explicit browser or Planner projection only when required.\n15. For new domain persistence, add and register a migration, update both\n adapters, and extend the shared storage contract.\n16. Test codecs, origins, authorization, retries, cancellation, credentials,\n publication, storage contracts, and every exposed UI path.\n\n## Definition contract\n\nA registered definition has this shape:\n\n```ts\ntype JobDefinition = {\n\ttype: string;\n\tversion: number;\n\tlabel: string;\n\tdescription: string;\n\torigins: readonly (\"scheduler\" | \"planner\" | \"user\")[];\n\tcredential: \"active-planner\" | \"none\";\n\tlimits: {\n\t\ttimeoutMs: number;\n\t\tmaxAttempts: number;\n\t\tmaxAiCredits: number;\n\t\tmaxInputBytes: number;\n\t\tmaxArtifactBytes: number;\n\t};\n\tprogress?: Readonly>;\n\tinput: { parse(value: JsonValue): Input };\n\tartifact: { parse(value: JsonValue): Artifact };\n\texecute(execution: JobExecution): Promise;\n\tpublish?(publication: JobPublication): Promise;\n};\n```\n\n`JobRegistry` validates definitions when the process starts:\n\n- types and progress stages use lowercase hyphenated identifiers;\n- versions and limits are positive safe integers;\n- labels and descriptions are nonblank;\n- at least one origin is declared, with distinct members of `scheduler`,\n `planner`, and `user`;\n- credential mode is `active-planner` or `none`;\n- progress, when present, has 1 through 32 code-owned labels, each at most 160\n characters;\n- only one definition may claim a particular `type@version`.\n\n`label` and `description` are catalog metadata. They do not create or label a\nbrowser surface automatically.\n\n### Strict codecs\n\nInput and artifact codecs are security and compatibility boundaries. They\nshould:\n\n- accept exact fields and reject unknown ones;\n- normalize deterministically;\n- enforce semantic, collection, and string bounds;\n- return plain JSON only;\n- reject `undefined`, dates, class instances, accessors, sparse arrays,\n non-finite numbers, and cyclic values;\n- validate provenance and relationships rather than trusting model output.\n\n`JobService` canonicalizes JSON before and after the input codec. Object keys are\nsorted, nesting is bounded, UTF-8 bytes are measured, and the normalized form is\nused in the idempotency fingerprint. Settlement applies the same process to the\nartifact codec.\n\nChanging a persisted contract normally requires a new definition version. Keep\nold versions registered while durable pending, paused, or running rows can still\nreference them. New enqueue requests use the highest registered version; the\nrunner and settlement always resolve the exact persisted version.\n\n## Minimal definition\n\nThe following skeleton shows a non-model job. Real codecs should usually be more\nspecific than these examples.\n\n```ts\nimport { JobExecutionError } from \"./registry\";\n\nimport type { JsonValue } from \"../storage/model\";\nimport type { JobDefinition } from \"./registry\";\n\ntype ExampleInput = {\n\tresourceId: string;\n};\n\ntype ExampleArtifact = {\n\tresourceId: string;\n\tresult: string;\n};\n\nasync function analyze(\n\tresourceId: string,\n\tsignal: AbortSignal,\n\tdeadline: Date,\n): Promise {\n\tif (signal.aborted || Date.now() >= deadline.getTime()) {\n\t\tthrow new Error(\"analysis stopped\");\n\t}\n\treturn `Analyzed ${resourceId}`;\n}\n\nfunction record(value: JsonValue): Record {\n\tif (!value || typeof value !== \"object\" || Array.isArray(value)) {\n\t\tthrow new Error(\"expected object\");\n\t}\n\treturn value;\n}\n\nfunction input(value: JsonValue): ExampleInput {\n\tlet item = record(value);\n\tlet resourceId = typeof item.resourceId === \"string\" ? item.resourceId.trim() : \"\";\n\tif (Object.keys(item).length !== 1 || !resourceId || resourceId.length > 128) {\n\t\tthrow new Error(\"invalid example input\");\n\t}\n\treturn { resourceId };\n}\n\nfunction artifact(value: JsonValue): ExampleArtifact {\n\tlet item = record(value);\n\tif (\n\t\tObject.keys(item).length !== 2\n\t\t|| typeof item.resourceId !== \"string\"\n\t\t|| typeof item.result !== \"string\"\n\t\t|| item.result.length > 4_000\n\t) throw new Error(\"invalid example artifact\");\n\treturn { resourceId: item.resourceId, result: item.result };\n}\n\nexport function exampleDefinition(): JobDefinition {\n\treturn {\n\t\ttype: \"example-analysis\",\n\t\tversion: 1,\n\t\tlabel: \"Example analysis\",\n\t\tdescription: \"Analyzes one example resource.\",\n\t\torigins: [\"user\"],\n\t\tcredential: \"none\",\n\t\tprogress: { analyze: \"Analyzing resource\" },\n\t\tlimits: {\n\t\t\ttimeoutMs: 60_000,\n\t\t\tmaxAttempts: 2,\n\t\t\tmaxAiCredits: 1,\n\t\t\tmaxInputBytes: 8 * 1024,\n\t\t\tmaxArtifactBytes: 16 * 1024,\n\t\t},\n\t\tinput: { parse: input },\n\t\tartifact: { parse: artifact },\n\t\tasync execute(execution) {\n\t\t\tawait execution.progress(\"analyze\", \"started\");\n\t\t\tif (execution.signal.aborted) {\n\t\t\t\tthrow new JobExecutionError(\"analysis-cancelled\");\n\t\t\t}\n\t\t\tlet result = await analyze(\n\t\t\t\texecution.input.resourceId,\n\t\t\t\texecution.signal,\n\t\t\t\texecution.deadline,\n\t\t\t);\n\t\t\tawait execution.progress(\"analyze\", \"completed\");\n\t\t\treturn artifact({ resourceId: execution.input.resourceId, result });\n\t\t},\n\t};\n}\n```\n\nEven a credential-free definition currently needs positive `maxAiCredits`.\nThat field is catalog metadata, not proof that AI is used.\n\n## Register a definition\n\nDefinitions are assembled explicitly in `apps/server/src/main.ts`:\n\n```ts\nlet definitions: JobDefinition[] = [];\nif (config.backgroundJobs) {\n\tdefinitions.push(exampleDefinition());\n}\nlet jobRegistry = new JobRegistry(definitions);\n```\n\nThere is no directory scan or plugin discovery. Register every version that may\nstill have durable work:\n\n```ts\ndefinitions.push(exampleDefinitionV1(), exampleDefinitionV2());\n```\n\nRegistration alone does not schedule work or create an API. Add an explicit\nenqueue path using the intended origin method:\n\n```ts\nlet result = await jobs.enqueueUser({\n\tchannelId,\n\ttype: \"example-analysis\",\n\ttargetKey: resourceId,\n\tidempotencyKey: `example-analysis:${requestId}`,\n\tinput: { resourceId },\n});\n```\n\nChoose the origin method deliberately:\n\n| Method | Intended caller |\n| -------------------- | ----------------------------------------------- |\n| `enqueueScheduler()` | A code-owned scheduler or coordinator. |\n| `enqueuePlanner()` | A Planner tool or Planner-origin domain action. |\n| `enqueueUser()` | An explicit authenticated user action. |\n\nCallers do not supply an origin field. The service stamps it and checks that the\ndefinition permits it.\n\nOrigins record how work entered the system; they are not authentication or\nauthorization. A browser route must still authenticate the session, enforce\nOrigin, admission, repository node identity, and role. Planner tools and\nschedulers must enforce their own trusted ingress before calling `JobService`.\n\nThe caller supplies the semantic target without the job type prefix.\n`JobService` changes `resource-id` into\n`example-analysis:resource-id`. Supplying an already-prefixed target would\nproduce `example-analysis:example-analysis:resource-id`.\n\n### Idempotency and supersession\n\nAn idempotency key is unique within a channel. Reusing it with the same\nfingerprint returns the original job with `repeated: true`; reusing it with\ndifferent normalized input, origin, target, version, or availability conflicts.\n\nA new idempotency key for the same target increments the target generation and\nsupersedes older pending, paused, or running generations. Older terminal rows\nremain durable, but consumers must reject artifacts whose target generation is\nno longer current.\n\nUse request identity for idempotency and resource identity for the target. Do\nnot generate both from the current time.\n\n## Durable lifecycle\n\n```text\npending -> running\npending/running -> paused\npaused -> pending\nrunning -> pending retry or process recovery\nrunning -> completed\nrunning -> failed\npending/paused/running -> cancelled\npending/paused/running -> superseded\n```\n\nThe lifecycle is:\n\n1. `JobService` resolves the current definition, validates origin and input,\n computes the fingerprint, and enqueues under the writer fence.\n2. Storage advances the target generation, supersedes older active work, and\n inserts a pending row before `job:changed` is published.\n3. `JobRunner` polls globally and wakes immediately for local pending changes.\n4. Storage claims current pending work with `FOR UPDATE ... SKIP LOCKED`, or\n reclaims an expired running claim.\n5. The runner resolves the exact persisted definition version and parses the\n persisted input again.\n6. The runner resolves any required credential, stores only token-free binding\n metadata, and applies global and per-owner concurrency.\n7. The executor receives cloned input, credential, abort signal, deadline, and\n progress callback.\n8. `JobService` validates the candidate artifact and runs any publication hook.\n9. Fenced storage atomically marks the job completed and inserts its immutable\n artifact.\n10. Generic and domain-specific invalidations happen only after persistence.\n\nThe default runner uses global concurrency 2, per-owner concurrency 1, a\n30-second claim TTL, a 10-second heartbeat, one-second polling, exponential\nretry up to five minutes, and a 10-second shutdown grace period.\n\n### Claims and late output\n\nEvery claimed mutation must match the channel, job ID, process claim owner,\nclaim generation, unexpired claim, and current target generation. Cancellation,\nsupersession, owner revocation, timeout, or shutdown clears or invalidates that\nclaim. A worker may continue briefly if it ignores cancellation, but its late\nprogress and artifact cannot commit.\n\n`claim_owner` identifies the runner process, not the Planner owner or an\nauthorization principal.\n\n### Attempts and retries\n\n`attempts` increments on every claim. `failures` increments only when a retry\nconsumes failure budget. Despite its name, `JobLimits.maxAttempts` is currently\ncompared with `failures`.\n\nCredential rotation, owner capacity, shutdown, and similar recovery paths may\nrequeue without increasing `failures`. A `JobExecutionError` supplies a safe,\ncode-owned durable reason; an unclassified executor failure becomes\n`attempt-error`.\n\nDefinitions must honor both `execution.signal` and `execution.deadline`, and\nmust clean up disposable resources in `finally`. Fencing protects persistence;\nit does not stop leaked external work.\n\n### Progress and diagnostics\n\nProgress stages must be declared by the definition. The executor can append\n`started` and `completed`. The runner makes a best-effort `interrupted` append\nwhile it still owns the claim; cancellation, supersession, or claim loss may\nleave no interruption entry. Storage retains at most 32 entries.\n\nStage IDs, labels, and reason codes are durable and browser-visible. Structured\ndiagnostics are bounded but log-only. Neither category may contain:\n\n- prompts or model prose;\n- query text or private document content;\n- source URLs or provider payloads;\n- credentials, paths, or raw errors.\n\nUse `JobExecutionError` for a bounded durable machine reason and optional\noperator diagnostic. It accepts at most 16 boolean, nonnegative integer, or\ntoken-like string diagnostic fields. Unknown browser reasons are deliberately\nrendered as a generic interruption.\n\n## Publication hooks\n\nA definition may delay completion with `publish()`:\n\n```ts\nasync publish({ job, artifact, commit }) {\n\tif (!await stillCurrent(job, artifact)) {\n\t\tthrow new Error(\"target changed before publication\");\n\t}\n\tawait commit();\n}\n```\n\nThe hook runs after artifact validation but before settlement. It must call and\nnormally await `commit()` exactly once while the hook is active. A hook that\ndoes not commit leaves the attempt uncompleted; a late or second commit is\nrejected. If the hook commits and then throws, the durable completion remains\nauthoritative and the service logs the hook error.\n\nUse a publication hook only when completion needs an external freshness or\nserialization guard. `document-summary@1` uses one to hold the document mutation\ngate and recheck revision and source hash. Ordinary jobs settle directly.\n\nThe generic service publication callback sends only a background-job channel\nrevision invalidation. A separate domain service may observe a completed job and\npersist its own projection. That reconciliation must be idempotent and\nrecoverable on a later read because observer delivery is not a durable queue.\n\n## Credentials and ownership\n\n`credential: \"active-planner\"` means the job uses the channel's existing\nPlanner owner and Copilot entitlement. The runner resolves ownership through\n`ActiveOwnerBindings`; it never claims ownership itself. An explicit product\naction may establish ownership before enqueueing. Without an available owner,\nthe job pauses as `owner-unavailable`.\n\nResolution and later revalidation check the process-local session, credential\nrevision, ownership generation, instance admission, repository node identity,\nGitHub App installation, repository write role, and expiry. Definitions must\nnever copy the GitHub token into input, artifacts, progress, diagnostics, or\nPostgreSQL. A durable claim binding contains only the owner session ID, owner\ngeneration, credential revision, and repository ID.\n\n`credential: \"none\"` skips owner resolution. The current server still disables\nthe entire runner when `AGENT=off`, so a credential-free definition does not run\nunder that flag without a deliberate change to the main runner gate.\n\n## Model-backed workers\n\nDo not reuse the Planner conversation session. Every worker stage opens a fresh,\ndisposable SDK session. Most stages submit one bounded structured result;\ngenerated document descriptions reuse one disposable session for multiple\nbounded chunk and reduction turns.\n\n### Private worker\n\nUse `Agent.openWorker()` for private document or repository-derived material.\nThe effective capability set must contain only the definition's terminal result\ntool. Private workers receive no MCP server, web search, URL fetch, repository\ntool, shell, filesystem, host Git, skill, or plugin.\n\n### Public research worker\n\nUse `Agent.openPublicResearchWorker()` only for an explicitly disclosed public\nquery. Its fail-closed capability audit requires exactly:\n\n```text\ncustom:submit_research_result\ngithub-mcp-server/web_search\n```\n\nNever give one session both private context and public web capability. The\npublic evidence worker receives only the submitted brief. An isolated no-web\ndocument-analysis worker receives the brief and parent-document snapshot. A\nseparate isolated no-web report-synthesis worker receives the brief, normalized\npublic evidence, and the private findings from document analysis.\n\nAccepted public sources must be canonical public HTTPS URLs observed in\nsuccessful web-search citation or typed-resource metadata. Bare URLs in model\nprose do not establish provenance. Chopin never follows, resolves, or fetches a\nsource URL, and generated reports may cite only URLs retained in normalized\nevidence. This validates provenance, not factual accuracy or page safety.\n\n### Executor-owned limits\n\n`JobLimits.maxAiCredits` is not centrally enforced by `JobRunner`. It describes\nthe maximum aggregate credits for one job attempt. A model-backed executor must\nset per-session limits whose possible total does not exceed it; for example, a\n60-credit two-stage job can allocate 30 credits to each worker. The current\nworker helper requires at least 30 credits per session. Keep definition metadata\nand actual worker construction synchronized.\n\nAlways recheck `credential.authorize`, observe credential and job abort signals,\nand discard the SDK session in `finally`.\n\n## Current definitions\n\n| Definition | Production trigger | Persisted input | Worker boundary |\n| --------------------- | --------------------------------------- | ------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------- |\n| `document-summary@1` | Open, edit, restore, or MCP persistence | Revision, source hash, generator version, and `output:\"description\"`; not the source | Private worker; source loaded at execution and publication rechecks current revision/hash |\n| `research-evidence@1` | Immediate research request | Internal workspace, initial turn, exact submitted brief | Public worker with only result tool and audited `web_search` |\n| `research-answer@1` | Completed evidence | Parent document snapshot, evidence, and internal compatibility history | Two private workers with only result tools for analysis and complete report synthesis |\n\n`document-summary@1` remains the only durable definition version; there is no\n`document-summary@2`. New V1 requests carry the output marker and use the\nidempotency key `description-v1::`. The distinct\nrequest identity means an unchanged document whose existing V1 artifact is a\nlegacy summary is eligible for lazy description generation.\n\nThe V1 codecs still read markerless legacy inputs and their `summary` artifacts.\nThose artifacts remain available through Background Work, but the catalogue\nprojector ignores them. A new marked artifact instead carries\n`output:\"description\"` and one physical line identifying the document's type,\npurpose, and subject, for example `PRD for ...`, `RFC about ...`, or `Plan for\n...`. The description retains the broad 4,000-codepoint and 16 KiB artifact\nsafety limits. A blank canonical source produces `Empty document` without a\nmodel call.\n\nScheduling is lazy rather than a database-wide backfill. Canonical edits\ndebounce new work; opening or restoring a document ensures work for its current\nsource; MCP creation and replay schedule after persistence. Execution requires\nthat channel's active Planner owner, so work pauses without an available owner.\nThere is no unattended scan that establishes owners or regenerates every stored\ndocument.\n\nAfter a marked job completes, an idempotent projector writes the description to\nchannel catalogue metadata with the source plan revision and hash, generator\nversion, source job ID, projection timestamp, and an independent description\nrevision. The previous description remains visible and searchable while newer\nwork is pending or failed. Projection does not advance collaboration revision or\nchannel `updatedAt`, so it does not change list recency.\n\nAllowed origins are broader than current callers: the document-summary\ndefinition permits scheduler and planner; research definitions permit user and\nplanner. Current production enqueue paths use scheduler for descriptions and\nuser for research execution.\n\nGenerated descriptions and legacy document-summary artifacts are not the\nreserved Planner transcript summary or an MCP creation `brief`. They are not\ninjected into a later Planner turn automatically. Descriptions are untrusted\nmodel output. The Planner may explicitly call `list_background_jobs` and\n`read_background_job`, and must treat the returned artifact as untrusted\nevidence.\n\n## Research request orchestration\n\nThe public product has one immediate research request; the database and service\nretain `research_workspaces`, turns, and messages as internal staging and\nhistorical-reference compatibility:\n\n1. An inline card or current Planner turn persists the exact brief, one initial\n turn, attribution, and stable request identity.\n2. The same action establishes the channel's Planner owner when needed and\n enqueues `research-evidence`; there is no draft or confirmation step.\n3. The evidence worker receives only that exact brief, with no private document\n context, and may derive or refine the queries it sends to web search.\n4. Completed normalized evidence causes an idempotent `research-answer`\n enqueue.\n5. The answer job captures current canonical parent-document context. It first\n runs an isolated no-web document-analysis worker, then a separate isolated\n no-web report-synthesis worker with the resulting private findings and\n normalized public evidence.\n6. A complete report artifact is validated and converted to canonical MDX.\n7. One fenced storage transaction creates an initialized ordinary child channel\n and links it to the request. Only then does the card become ready and the\n child enter navigation.\n\nFailure or cancellation publishes no child. Explicit retry clears only terminal\ninitial job links under optimistic guards, preserves the request and brief, and\nstarts a new evidence generation. Reconciliation runs after completion and on a\nlater request read, so interruption cannot require a second child or a manual\nrepair.\n\nHere, **private** means not disclosed to public web search. Internal staging\nrows, selected compatibility projections, and artifacts are readable through\nauthorized server paths. Normalized job input is persisted but omitted from the\ninline card projection. Private worker material is still sent to the hosted\nCopilot inference service under the active owner's credential.\n\n## Archive and deletion\n\nArchiving a document suspends its description coordinator, cancels any pending\ndescription debounce, and prevents new description scheduling. It does not\nblanket-cancel background jobs already admitted for that channel. New research requests and\nexplicit retries are blocked while archived, but already-started evidence and\nanswer jobs may settle, and their idempotent request reconciliation may still\npersist. Publication rechecks that the parent is active, so an archived parent\ndoes not gain a child.\n\nPermanent deletion has a stronger boundary. The runner first blocks new claims\nfor the channel while description scheduling remains suspended, aborts its active\nattempts, cancels every pending, paused, or running job, and waits a bounded\ngrace period. The server then closes the live plan before atomically deleting\nthe archived channel. Claim fencing rejects any late worker progress or\nartifact, and an eligible channel delete cascades job targets, jobs, artifacts,\nand internal research staging. A parent with a child and a child linked from a\npublished request are deliberately not eligible for deletion.\n\n## Storage and retention\n\nThe generic schema supports a new standalone job type without a migration:\n\n- `background_job_channels` stores the independent background-job channel\n revision;\n- `background_job_targets` stores current target generations;\n- `background_jobs` stores requests, state, claims, normalized input, progress,\n and sanitized failure reason;\n- `background_job_artifacts` stores one immutable completed result per job.\n\nA migration is needed only when the job adds domain-specific durable records or\nlinks, as research request staging does. When changing `StorageAdapter`, update the\nmodel, port, memory adapter, PostgreSQL adapter, migration, migration registry in\n`apps/server/src/storage/postgres/migrations.ts`, migration tests, and shared\ncontract suite together. Never edit an applied migration; checksums make that a\nstartup error. See [Storage](storage.md#postgresql-schema) for the table map.\n\nJob summaries omit input, fingerprint, idempotency key, and claim binding. This\ndoes not make input ephemeral: normalized input is stored in PostgreSQL. For\nexample, document-description input omits document source, while research-answer\ninput deliberately persists the captured source snapshot. Store only material\nthe workflow needs and document its retention boundary.\n\nThere is no current pruning policy for jobs, artifacts, or internal research\nstaging while their parent document remains stored. Cancellation prevents child\npublication; it does not erase input or history. Full eligible document deletion\nis the exception and removes that channel-owned state.\n\n## Browser and Planner integration\n\nThe current browser has no generic Background Work destination. Research status\nis deliberately projected through the inline request card: queued, searching,\nanalyzing, writing, publishing, ready, failed, or cancelled, plus discovered\nsources and safe errors where available. The card uses request create, detail,\ncancel, and retry routes and refreshes after `research:changed`.\n\nThe server retains `job:list`, `job:get`, `job:cancel`, and `job:changed`\nfor internal and compatibility consumers. Repository readers may list and read.\nCancellation requires current write access and is restricted to active,\nuser-origin research evidence and answer jobs.\n\nA newly registered type receives no browser surface automatically. Add or update\nthese when a domain projection is needed:\n\n- subject extraction in `apps/server/src/jobs/service.ts` and both job storage\n adapters;\n- safe browser serialization and cancellation rules in\n `apps/server/src/jobs/browser.ts`;\n- protocol declarations in `packages/protocol/job.d.ts`;\n- domain HTTP state and invalidations when the job belongs to a larger workflow.\n\nThe Planner can list and read jobs, but has no generic enqueue, pause, resume,\ncancel, or artifact-trust capability. A domain-specific Planner tool must still\nrespect the job origin, authorization, disclosure, and persistence boundaries.\n\n## Configuration\n\nOnly the exact value `off` disables these flags:\n\n| `AGENT` | `BACKGROUND_JOBS` | `WEB_RESEARCH` | Result |\n| ------- | ----------------- | -------------- | ------------------------------------------------------------------------------------------------------- |\n| on | on | on | Document description, research evidence, and research answer definitions; runner and coordinator active |\n| on | on | off | Document description and private research answer definitions; no new public evidence |\n| off | on | forced off | Persisted jobs and descriptions remain stored; the runner and description coordinator are off |\n| any | off | forced off | No definitions execute |\n\nConfiguration is read at startup. Restoring a disabled capability requires a\nrestart. Existing durable rows and artifacts are not deleted by a flag.\n\n`AGENT=off` disables the entire runner, including `credential: \"none\"` jobs.\nGating a definition itself can leave durable rows paused as `unregistered-type`.\nPrefer keeping old definitions registered while gating new enqueue paths. If a\ndefinition must be removed conditionally, provide and test an explicit resume\npath; automatic owner recovery only revives compatible `active-planner` work.\n\nIf a new definition needs its own flag, update configuration parsing and tests,\n`.env.example`, Compose metadata and tests, self-hosting documentation, and the\nE2E server environment where relevant.\n\n## Common failure traps\n\n- Registering a definition without adding an enqueue path.\n- Removing an old version while durable work still references it.\n- Treating caller input as the job origin.\n- Treating a stamped origin as authentication or authorization.\n- Pre-prefixing the target key before calling `JobService`.\n- Reusing an idempotency key with changed semantic input.\n- Persisting tokens, raw provider events, prompts, or private prose in progress\n and diagnostics.\n- Assuming `maxAiCredits` is runner-enforced.\n- Assuming `maxAttempts` is compared with claim count rather than failures.\n- Ignoring the abort signal because claim fencing rejects late output.\n- Reusing a Planner session for background work.\n- Giving a private worker ambient MCP, URL, repository, shell, or filesystem\n capability.\n- Giving one worker both private context and public web search.\n- Calling an undeclared progress stage.\n- Calling a publication `commit()` twice, too late, or not at all.\n- Broadcasting or projecting state before its durable commit.\n- Assuming process-local completion callbacks are durable delivery.\n- Bypassing `JobService` and therefore bypassing codecs, origins, versions, and\n byte limits.\n- Assuming Playwright executes live Copilot work; E2E runs with `AGENT=off` and\n seeds validated job state.\n\n## Testing\n\nUse the narrowest tests that exercise the new behavior:\n\n```bash\n# Definition, service, runner, and domain tests\nbun test apps/server/src/jobs apps/server/src/research\n\n# Memory adapter and the rest of the unit/domain suite\nbun test\n\n# Real PostgreSQL transitions, fencing, and migrations\nbun run test:postgres\n\n# Type and static validation\nbun run types\nbun run ci\n\n# Browser integration when the job has a user-facing surface\nbun run e2e\n```\n\nDefinition tests should inject engines rather than call live Copilot. Cover\nstrict codecs, exact context separation, progress, citation or provenance rules,\nand artifact rejection. Runner tests should cover success, retries, timeout,\nowner loss, credential rotation, cancellation, heartbeat loss, late output, and\nshutdown. Storage behavior belongs in the shared adapter contract so memory and\nPostgreSQL run the same cases.\n\n## Main implementation points\n\n- Definition and registry contract: `apps/server/src/jobs/registry.ts`\n- Validation, enqueue, settlement, and publication: `apps/server/src/jobs/service.ts`\n- Claims, execution, retries, and shutdown: `apps/server/src/jobs/runner.ts`\n- Current definitions: `apps/server/src/jobs/document-summary.ts` and\n `apps/server/src/jobs/research-workspace.ts`\n- Document-description projection: `apps/server/src/jobs/document-description.ts`\n- Definition registration and runtime wiring: `apps/server/src/main.ts`\n- Active owner resolution: `apps/server/src/agent/active-owner.ts`\n- Private and public worker construction: `apps/server/src/agent/client.ts`\n- Durable job model and port: `apps/server/src/storage/model.ts` and\n `apps/server/src/storage/port.ts`\n- PostgreSQL job store: `apps/server/src/storage/postgres/jobs.ts`\n- Generic protocol compatibility: `packages/protocol/job.d.ts`\n- Research orchestration: `apps/server/src/research/service.ts`\n","editable",false,"branch","documentNavigation","repo","githubnext/chopin","repoId","7d1a8ac3-35b0-4dea-85d4-06a980f3bada","ownerId","org:89615882","site","nexthub","private","admin","settings","maintainer","triage","starred","stars",0,"watchers","watching","pinned","notify","views",[183,184,185,186,187,188,189,190,191,192,193,194,195],"navigation",{"_54":55,"_139":140,"_141":142,"_143":144,"_145":146,"_147":148,"_149":150},"sources",[],"viewer","meta",{"_131":132,"_133":134,"_135":136,"_42":31,"_137":126,"_138":31,"_48":49,"_50":49},"branches",[73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130],"tags",[69,70,71,72],"cloneUrl","https://git.nexthub.sh/githubnext/chopin.git","color-controls-slice-1","color-controls-slice-2","color-controls-slice-3","color-controls-slice-4","Krzysztof-Cieslak-patch-1","Krzysztof-Cieslak-patch-2","Maggie/app-preview-toolkit","Maggie/ci-playwright-install-timeout","Maggie/collaborative-visual-decision","Maggie/design-audit","Maggie/generic-visual-decision-runtime","Maggie/impeccable-chopin-metadata","Maggie/jev-chat-product","Maggie/jev-composed-assessment","Maggie/native-document-layouts","Maggie/openui-composition-product","Maggie/openui-composition-trial","Maggie/planner-visual-authoring","Maggie/planner-visual-requests","Maggie/visual-selection-eval","artifacts/pr-419-live-preview-20261009","automation/pr-maintenance-state","automation/use-effect/callout-disabled-picker-8cf9e49ba5c09471","automation/use-effect/collaboration-options-ref-4e3a5a08a879a5ae","automation/use-effect/room-workspace-effective-access-a57d40866c71bed0","chore/bun-1.4.2","color-controls","color-controls-plan","comments","design-pr-assets","design/archived-decisions-badge","design/channel-route-remount","design/chat-decision-entry","design/conversation-analysis-markers","design/decision-evidence","design/settled-line-icon","dev/integration","diagram-step-controls","feat/liveapp-frontend-pilot","feat/local-experiments","feat/local-launcher-tracer","fix/autolink-mdx-crash","github-pills/2-pills","github-pills/3-hover-card","maggie/app-preview-toolkit","maggie/ci-playwright-install-timeout","maggie/collaborative-visual-decision","maggie/design-audit","maggie/generic-visual-decision-runtime","maggie/impeccable-chopin-metadata","maggie/jev-composed-assessment","maggie/native-document-layouts","maggie/openui-composition-product","maggie/openui-composition-trial","maggie/planner-visual-authoring","maggie/planner-visual-requests","maggie/visual-selection-eval","main","plain-text-fence-chrome","tokens/09-copy","wireframes/2-renderer","wireframes/3-planner","description","Let's compose rich plans together","owner","githubnext","section","","head","empty","pins",[165,166,167],"documents",{"_160":161,"_162":163},"code",{"_156":157,"_158":159},"automations",{"_153":154,"_155":-5},"delivery",{"_151":49,"_152":-5},"tag","04zvKQ8QOrDUF8TO1Ovz0Zy8_TT","releases","applications","workflows",3,"deployments","commits",1000,"changes",13,"counts",{"_164":163},"total",11,"issue",{"_168":158,"_170":181,"_15":158,"_172":182,"_52":174,"_175":31,"_176":174,"_177":159},{"_168":178,"_170":179,"_15":178,"_172":180,"_52":174,"_175":31,"_176":174,"_177":49},{"_168":169,"_170":171,"_15":169,"_172":173,"_52":174,"_175":31,"_176":174,"_177":163},"id","issues","name","Issues","filters",{},true,"saved","pinnable","count","tasks","Tasks",{},"Changes",{},{"_168":158,"_170":181,"_15":158,"_172":226,"_52":174,"_175":31,"_176":174},{"_168":178,"_170":179,"_15":178,"_172":225,"_52":174,"_175":31,"_176":174},{"_168":169,"_170":171,"_15":169,"_172":224,"_52":174,"_175":31,"_176":174},{"_168":221,"_170":222,"_15":221,"_172":223,"_52":31,"_175":31,"_176":174},{"_168":218,"_170":219,"_15":218,"_172":220,"_52":31,"_175":31,"_176":174},{"_168":153,"_170":216,"_15":145,"_172":217,"_52":31,"_175":31,"_176":174},{"_168":155,"_170":213,"_15":214,"_172":215,"_52":31,"_175":31,"_176":174},{"_168":151,"_170":211,"_15":151,"_172":212,"_52":31,"_175":31,"_176":174},{"_168":207,"_170":208,"_15":209,"_172":210,"_52":31,"_175":31,"_176":174},{"_168":141,"_170":205,"_15":141,"_172":206,"_52":31,"_175":31,"_176":31},{"_168":202,"_170":203,"_15":202,"_172":204,"_52":31,"_175":31,"_176":31},{"_168":199,"_170":200,"_15":199,"_172":201,"_52":31,"_175":31,"_176":31},{"_168":196,"_170":197,"_15":196,"_172":198,"_52":31,"_175":31,"_176":31},"decisions","Decisions",{},"changelog","Changelog",{},"roadmap","Roadmap",{},"Documents",{},"activity","Activity","insights",{},"Releases",{},"Deployments","automations/deployments",{},"Workflows",{},"discussions","Discussions",{},"goals","Goals",{},{},{},{},"groups","user","agents",[],"now",1794596002971,"sidebar","expanded"]