iamnbutler/shelfgoblinPublic

Shelf Goblin — discover LitRPG audiobooks, follow series, and track your library.

Add private quality research index and calibration tools #4

Mergediamnbutlermerged 1 commit intomainfromrefs/pull/4/head
  1. IA

    The catalog has no auditable quality signal independent of star ratings. This adds a private research pipeline for book, series, and author craft evidence, with resumable inference, retained paid responses, versioned SQLite snapshots, and explicit unknown/coverage states. It does not change the public catalog or its sort order.

    Craft, audiobook performance, content preferences, recognition, and corroborated publication risk stay separate. Ratings are excluded from selection, prompts, and arithmetic. Audio release batching cannot trigger a writing-quality penalty, and author aggregates never become evidence about an individual book.

    The benchmark registry includes user-proposed reference ranges and synthetic controls. Real trials exposed sparse craft evidence and classification errors; the current metric is selected-review craft sentiment, not a validated overall literary-quality score. `publicRankingEnabled` is explicitly false. Separate OpenAI span extraction and Jev semantic verification tools support further calibration without overwriting the first experiment or repurchasing its answers.

    Validation: 2,727 tests pass with API credentials empty; backend typecheck and Svelte check pass. A bounded live trial assessed 323 retained reviews. The stronger extraction model passed 16 invented regression examples, but a real-comment audit still found errors, so those results are not promoted into rankings. Paid receipts and private source evidence are backed up in the private data archive.

    See `docs/quality-index.md` for commands, model experiments, limitations, and the optional local PostgreSQL adapter.

  2. iamnbutlerready-for-reviewlast month on GitHub

  3. iamnbutlermerged 60f8a62beb7dlast month on GitHub

  4. iamnbutlerclosedlast month on GitHub