githubnext/chopinPublic

Let's compose rich plans together

Recover frozen development inputs for visual-document evaluation #407

MergedMaggieAppletonmerged 6 commits intomainfromrefs/pull/407/head
  1. MA
    MaggieAppleton

    ## Summary

    Recover the frozen development input manifest from the earlier visual-document corpus, then add a versioned dataset that a fresh clone can load offline. V2 contains 7 attributed, source-grounded discussion adaptations (27 cutoffs), 19 input-only synthetic chats (68 checkpoints), and one pinned original Rust RFC. It keeps expected outcomes out of model input and verifies each committed file by size and SHA-256.

    The historical registry remains byte-identical (`5cdc0868…889cb`). V2 retains its original source hashes, IDs, source clusters and cutoff times rather than changing the earlier experiment record. The small API is `listCases()` / `loadCase(id, checkpointId?)` in `evals/datasets/v2.ts`.

    ## Verification

    - Fresh `git archive` checkout with no ignored files: `bun test evals/datasets/input.test.ts evals/datasets/v2.test.ts` — 5 pass, 0 fail, 0 skip. - `bun run types` and `bun run ci` — pass. - Original Rust RFC and historical registry byte hashes match their pinned values; all 27 committed input files match the V2 manifest.

    ## Source limits

    The GitHub discussions are clearly labeled paraphrases, not republished contributor comments. Each event points to its source and keeps historical timing. Their summaries preserve the discussion structure but do not retain complete original wording or long code examples. The Rust RFC uses its source repository's MIT license option and carries the notice. The unlicensed ESLint RFC supplement remains historical metadata, not a V2 text input. These are development candidates, not approved gold or model-quality results. Validation and unopened holdout files are excluded.

    ## Data flow

    ![Diagram of the frozen historical registry, three kinds of committed V2 inputs, and the hash-checked offline loader](https://github.com/user-attachments/assets/b8b0deb3-1600-4842-ab13-2d075824bddf)

  2. MAeval: register frozen development source manifest7e9925c
  3. MAeval: restore bounded development input accessb02cf1c
  4. MAtest: enforce frozen registry hash in clean checkoutsb749a9b

  5. MaggieAppletonconvert-to-draftlast month on GitHub

  6. MAMerge remote-tracking branch 'origin/main' into maggie/visual-document-development-corpusfe81cdb
  7. MAeval: commit versioned offline development inputs680b4ca
  8. MAMerge remote-tracking branch 'origin/main' into maggie/visual-document-development-corpus20cf5d3

  9. MaggieAppletonready-for-reviewlast month on GitHub

  10. github-actions[bot]botadded label maintenance:readylast month on GitHub

  11. github-actions[bot]bot

    <!-- chopin:pr-maintenance:v1 --> **PR maintenance: ready**

    Inspected head: 20cf5d3ff18717b869dedc2c07516cbe23e3c6ad

    Base head: 97e17cf82dd9b5b4b56c2cb41660ce3dfba98e8c

    [PR readiness runs](https://github.com/githubnext/chopin/actions/workflows/pr-readiness.yml) · [CI run](https://github.com/githubnext/chopin/actions/runs/37912725634) <!-- chopin:pr-maintenance-auth:eyJwYXlsb2FkIjp7InNjaGVtYVZlcnNpb24iOjEsInJlcG9zaXRvcnkiOiJnaXRodWJuZXh0L2Nob3BpbiIsInJldmlzaW9uIjowLCJwcnMiOnsiNDA3Ijp7ImJvZHkiOiI8IS0tIGNob3Bpbjpwci1tYWludGVuYW5jZTp2MSAtLT5cbioqUFIgbWFpbnRlbmFuY2U6IHJlYWR5KipcblxuSW5zcGVjdGVkIGhlYWQ6IDIwY2Y1ZDNmZjE4NzE3Yjg2OWRlZGMyYzA3NTE2Y2JlMjNlM2M2YWRcblxuQmFzZSBoZWFkOiA5N2UxN2NmODJkZDliNWI0YjU2YzJjYjQxNjYwY2UzZGZiYTk4ZThjXG5cbltQUiByZWFkaW5lc3MgcnVuc10oaHR0cHM6Ly9naXRodWIuY29tL2dpdGh1Ym5leHQvY2hvcGluL2FjdGlvbnMvd29ya2Zsb3dzL3ByLXJlYWRpbmVzcy55bWwpIMK3IFtDSSBydW5dKGh0dHBzOi8vZ2l0aHViLmNvbS9naXRodWJuZXh0L2Nob3Bpbi9hY3Rpb25zL3J1bnMvMzc5MTI3MjU2MzQpIiwiYmxvY2tlcklkIjpudWxsfX19LCJzaWduYXR1cmUiOiJmN2JiM2M3ZmNjZWFjMDM1YjcwMTMyYjg0ODE5N2RjNWYxNDk2NzFjMTNkMWMxZWFhMGJjM2ZjYzNlMWQxNWFlIn0= -->

  12. MaggieAppletonmerged 7c690cbcfdf2last month on GitHub

  13. MaggieAppletonclosedlast month on GitHub

  14. MaggieAppletonhead-ref-deletedlast month on GitHub