Chopin · What the interface checks tested and found

\n\t\t\t\t

\n\t\t\t\t\t

\n\t\t\t\t\t\t

01 / the tests

\n\t\t\t\t\t\t

What did I actually ask the browser to do?

\n\t\t\t\t\t

\n\t\t\t\t\t\tThe tests use a dedicated example page built mostly from the same components as Chopin.\n\t\t\t\t\t\tThis lets us examine particular states, such as an open menu or a failed save, without\n\t\t\t\t\t\tlogging in or creating a real document.\n\t\t\t\t\t

\n\t\t\t\t\t\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\n\t\t\t\t\t

Question	What the test does	Result
Has the appearance changed?	\n\t\t\t\t\t\t\t\t\tTakes pictures of text, status labels, buttons, fields, conversation, note boxes,\n\t\t\t\t\t\t\t\t\tcode and tables, plus open menus and pop-up windows. Later runs compare new\n\t\t\t\t\t\t\t\t\tpictures with the reviewed references and flag differences for inspection.\n\t\t\t\t\t\t\t\t	\n\t\t\t\t\t\t\t\t\tAll 32 reference pictures have been refreshed from the current interface; changed\n\t\t\t\t\t\t\t\t\tpictures were visually reviewed. GitHub will compare against these saved\n\t\t\t\t\t\t\t\t\treferences.\n\t\t\t\t\t\t\t\t
Are there common accessibility problems?	\n\t\t\t\t\t\t\t\t\tChecks each pictured state for problems such as text that is too faint against its\n\t\t\t\t\t\t\t\t\tbackground, controls without a name that assistive software can announce, and\n\t\t\t\t\t\t\t\t\tmisleading structural labels.\n\t\t\t\t\t\t\t\t	\n\t\t\t\t\t\t\t\t\tApproved contrast shortfalls remain visible in the results. The revised checks\n\t\t\t\t\t\t\t\t\tpassed, including tests proving that unexpected findings still fail.\n\t\t\t\t\t\t\t\t
Can you use the controls without a mouse?	\n\t\t\t\t\t\t\t\t\tOpens the document menu, moves between options with arrow keys, opens the rename\n\t\t\t\t\t\t\t\t\twindow, moves between its controls with Tab and closes it with Escape. Checks that\n\t\t\t\t\t\t\t\t\tthe button that opened it becomes the active keyboard control again. Also chooses\n\t\t\t\t\t\t\t\t\tan item in a dropdown.\n\t\t\t\t\t\t\t\t	These keyboard sequences passed at both widths.
Do empty and failed saves behave as expected?	\n\t\t\t\t\t\t\t\t\tClears the title to check that Save becomes unavailable, then simulates a failed\n\t\t\t\t\t\t\t\t\trename to check that an error appears and the field is marked as having a problem.\n\t\t\t\t\t\t\t\t	\n\t\t\t\t\t\t\t\t\tBoth states behaved as expected. This tests the local example, not an actual\n\t\t\t\t\t\t\t\t\tserver save.\n\t\t\t\t\t\t\t\t
Can content still be read when space is limited?	\n\t\t\t\t\t\t\t\t\tDoubles the page’s base text size and checks that selected fields, note boxes and\n\t\t\t\t\t\t\t\t\tdocument-action controls fit their containers. Separately checks that a wide table\n\t\t\t\t\t\t\t\t\tand narrow code preview can scroll using the keyboard.\n\t\t\t\t\t\t\t\t	\n\t\t\t\t\t\t\t\t\tThose checks passed. Wide code and tables deliberately scroll sideways; showing\n\t\t\t\t\t\t\t\t\tonly part of them at once is expected.\n\t\t\t\t\t\t\t\t
Does the reduced-motion preference work?	\n\t\t\t\t\t\t\t\t\tEnables the browser preference for less motion, then opens the menu and rename\n\t\t\t\t\t\t\t\t\twindow and checks that their transitions are effectively immediate.\n\t\t\t\t\t\t\t\t	The tested controls respected that preference.

\n\t\t\t\t\t

\n\t\t\t\t\t\tWhat the numbers mean: the same 19 test cases run at two widths, giving\n\t\t\t\t\t\t38 checks. Eight further checks deliberately introduce mistakes to prove the colour\n\t\t\t\t\t\texceptions do not hide them, bringing the total to 46. The interface checks produce 16\n\t\t\t\t\t\treference pictures at each width, giving 32 pictures. These are counts of test cases and\n\t\t\t\t\t\tpictures, not counts of screens covered or bugs found.\n\t\t\t\t\t

\n\t\t\t\t

\n\t\t\t\t\t

\n\t\t\t\t\t\t

02 / findings

\n\t\t\t\t\t\t

What needed repair, and what was a design choice

\n\t\t\t\t\t

Problem found	Why it matters	What changed
Some colours fell below the automated contrast target	\n\t\t\t\t\t\t\t\t\tLabels on red action buttons, conversation timestamps and queued messages, and\n\t\t\t\t\t\t\t\t\tseveral colours in code previews did not meet the contrast check. That can make\n\t\t\t\t\t\t\t\t\tthem harder to read.\n\t\t\t\t\t\t\t\t	\n\t\t\t\t\t\t\t\t\tThe original colours and quieter secondary text are restored, as you requested.\n\t\t\t\t\t\t\t\t\tTheir contrast warnings remain recorded as deliberate design choices. The tests\n\t\t\t\t\t\t\t\t\tstill reject new or changed findings; they do not treat all faint text as\n\t\t\t\t\t\t\t\t\tacceptable.\n\t\t\t\t\t\t\t\t
Code previews needed keyboard access	\n\t\t\t\t\t\t\t\t\tA narrow code block can contain text beyond the right-hand edge. A keyboard user\n\t\t\t\t\t\t\t\t\tneeds to be able to reach the preview and scroll it.\n\t\t\t\t\t\t\t\t	\n\t\t\t\t\t\t\t\t\tThe preview can become the active keyboard control, shows a visible outline, and\n\t\t\t\t\t\t\t\t\tpassed the narrow-width scrolling check.\n\t\t\t\t\t\t\t\t
Some accessibility labels were misleading	\n\t\t\t\t\t\t\t\t\tThe example page described several static illustrations as active pop-up windows.\n\t\t\t\t\t\t\t\t\tAn initial code-preview fix also labelled several code boxes as major page regions\n\t\t\t\t\t\t\t\t\twith identical names.\n\t\t\t\t\t\t\t\t	\n\t\t\t\t\t\t\t\t\tThose labels were corrected so they describe the elements accurately. Neither\n\t\t\t\t\t\t\t\t\tfinding was left as an ignored test failure.\n\t\t\t\t\t\t\t\t

\n\t\t\t\t\t

\n\t\t\t\t\t\tThe keyboard, focus, scrolling and layout repairs remain. The colour restoration is\n\t\t\t\t\t\tcoordinated with the separate interface-repair work. The report will distinguish\n\t\t\t\t\t\taccepted contrast tradeoffs from unexpected test failures.\n\t\t\t\t\t

\n\t\t\t\t\t

\n\t\t\t\t\t\t

\n\t\t\t\t\t\t\tRetained by choice: the normal and focused buttons measure about 4.20:1 with white\n\t\t\t\t\t\t\tlabels. Hover and pressed states remain distinct. This is an acknowledged contrast\n\t\t\t\t\t\t\tshortfall.\n\t\t\t\t\t\t

\n\t\t\t\t\t\n\t\t\t\t\t

\n\t\t\t\t\t\tWe also found problems in the tests themselves. An early narrow\n\t\t\t\t\t\tscreenshot cut off part of a button example, and content still loading above a table\n\t\t\t\t\t\tmoved it during capture. I split oversized examples into smaller pictures and made the\n\t\t\t\t\t\tbrowser wait for content to finish loading. Those were capture problems, not evidence\n\t\t\t\t\t\tthat the app’s table was broken.\n\t\t\t\t\t

\n\t\t\t\t\t

\n\t\t\t\t\t\tWhat we learned: looking at the rendered interface revealed readability\n\t\t\t\t\t\tand keyboard problems that source-code checks alone had not settled. But the test setup\n\t\t\t\t\t\talso needs scrutiny: a bad screenshot can create a misleading result.\n\t\t\t\t\t

\n\t\t\t\t

\n\t\t\t\t\t

\n\t\t\t\t\t\t

03 / the evidence

\n\t\t\t\t\t\t

Examples of what was inspected

\n\t\t\t\t\t

\n\t\t\t\t\t\tThese pictures show the states covered by the tests, not before-and-after comparisons.\n\t\t\t\t\t\tThe conversation and code pictures now show the restored colours. All 46 local browser\n\t\t\t\t\t\tchecks passed after the restoration.\n\t\t\t\t\t

\n\t\t\t\t\t

\n\t\t\t\t\t\t

\n\t\t\t\t\t\t\t

\n\t\t\t\t\t\t\t\tRestored colours. Conversation: the checks include timestamps and the queued\n\t\t\t\t\t\t\t\tmessage. The message-entry area is an illustration, so this picture does not prove\n\t\t\t\t\t\t\t\tsending a message works.\n\t\t\t\t\t\t\t

\n\t\t\t\t\t\t\n\t\t\t\t\t\t

\n\t\t\t\t\t\t\t

\n\t\t\t\t\t\t\t\tRestored original code colours. The measured contrast shortfalls are accepted by\n\t\t\t\t\t\t\t\tchoice. A separate keyboard test checks that a narrow preview can scroll.\n\t\t\t\t\t\t\t

\n\t\t\t\t\t\t\n\t\t\t\t\t\t

\n\t\t\t\t\t\t\t

\n\t\t\t\t\t\t\t\tRename window: a simulated failure shows an error and marks the title field as\n\t\t\t\t\t\t\t\tinvalid. Keyboard tests also check entering and leaving this window.\n\t\t\t\t\t\t\t

\n\t\t\t\t\t\t\n\t\t\t\t\t\t

\n\t\t\t\t\t\t\t

\n\t\t\t\t\t\t\t\tDocument menu: the outline shows which action will respond to the keyboard. The test\n\t\t\t\t\t\t\t\tmoves between actions and closes the menu with Escape.\n\t\t\t\t\t\t\t

\n\t\t\t\t\t\t\n\t\t\t\t\t

\n\t\t\t\t

\n\t\t\t\t\t

\n\t\t\t\t\t\t

04 / checking the checks

\n\t\t\t\t\t\t

Can the tests actually catch mistakes?

\n\t\t\t\t\t

\n\t\t\t\t\t\tA test that always passes is not useful. To check that these tests could catch a\n\t\t\t\t\t\tproblem, I temporarily changed a menu’s colour and removed the name from a menu action.\n\t\t\t\t\t

\n\t\t\t\t\t

\n\t\t\t\t\t\t

\"Original — \n\t\t\t\t\t\t\t\t1. The original picture\n\t\t\t\t\t\t\t

\n\t\t\t\t\t\t

\n\t\t\t\t\t\t\t

2. The deliberate colour change

\n\t\t\t\t\t\t\n\t\t\t\t\t\t

\n\t\t\t\t\t\t\t

3. The test highlighted the difference and failed

\n\t\t\t\t\t\t\n\t\t\t\t\t

\n\t\t\t\t\t

\n\t\t\t\t\t\tThe missing action name also caused the accessibility check to fail. Both deliberate\n\t\t\t\t\t\tdefects were then removed. This shows that the checks caught those two specific\n\t\t\t\t\t\tmistakes; it does not prove they will catch every possible design or accessibility\n\t\t\t\t\t\tproblem.\n\t\t\t\t\t

\n\t\t\t\t\t

\n\t\t\t\t\t\tAfter adding the colour exceptions, I ran four more deliberate mistakes at each width:\n\t\t\t\t\t\tfaint new primary text, a changed code colour, an approved finding disappearing, and an\n\t\t\t\t\t\tunnamed button. All eight were rejected as intended. These checks are now part of the\n\t\t\t\t\t\tregular suite, so preserving quiet secondary text does not mean accepting any new\n\t\t\t\t\t\tproblem.\n\t\t\t\t\t

\n\t\t\t\t

\n\t\t\t\t\t

\n\t\t\t\t\t\t

05 / outcome and limits

\n\t\t\t\t\t\t

What have we achieved—and what remains?

\n\t\t\t\t\t

\n\t\t\t\t\t\tBuilt: reusable browser checks for the selected examples, reviewed\n\t\t\t\t\t\treference pictures, and a way to show a reviewer what changed when a test fails. The\n\t\t\t\t\t\tproposed change adds these checks to GitHub’s automatic checks on future code changes.\n\t\t\t\t\t

\n\t\t\t\t\t

\n\t\t\t\t\t\tRun successfully: all 46 browser checks passed locally after the colour\n\t\t\t\t\t\trestoration. GitHub reruns them against the saved pictures and also runs the existing\n\t\t\t\t\t\tapplication tests. Open Latest checks for the current proposed change’s results.\n\t\t\t\t\t

\n\t\t\t\t\t

\n\t\t\t\t\t\tStill awaiting integration: change #209 is open. Its proposed testing\n\t\t\t\t\t\tworkflow is not yet part of the main branch through this change. It also needs to be\n\t\t\t\t\t\tcombined with the related work and tested again after that combination.\n\t\t\t\t\t

\n\t\t\t\t\t

\n\t\t\t\t\t\t\tCoverage is selective. These examples do not cover every screen,\n\t\t\t\t\t\t\tstate or user journey. Much of the page uses real components; the message composer and\n\t\t\t\t\t\t\ttable-toolbar illustrations do not test real editing behaviour.\n\t\t\t\t\t\t
\n\t\t\t\t\t\t\tA narrow browser window is not a phone test. The checks used one\n\t\t\t\t\t\t\tbrowser engine at desktop-like and phone-like widths. They do not establish\n\t\t\t\t\t\t\tcompatibility with Safari, Firefox, touch interaction or actual mobile devices.\n\t\t\t\t\t\t
\n\t\t\t\t\t\t\tAccessibility still needs human testing. No screen-reader user study\n\t\t\t\t\t\t\tor complete accessibility audit was performed. A clean automatic scan covers only the\n\t\t\t\t\t\t\trules that tool can check in the states we showed it.\n\t\t\t\t\t\t
\n\t\t\t\t\t\t\tPictures cannot judge design quality. A comparison can flag changed\n\t\t\t\t\t\t\tspacing or colours. A person must decide whether a change is intentional, attractive\n\t\t\t\t\t\t\tand useful before accepting a new reference picture.\n\t\t\t\t\t\t

\n\t\t\t\t\t

\n\t\t\t\t\t\tThe practical gain: we now have a repeatable way to catch certain\n\t\t\t\t\t\tinterface mistakes and make unexpected visual changes visible during review. We still\n\t\t\t\t\t\tneed design judgement, broader coverage and human accessibility testing.\n\t\t\t\t\t

\n\t\t\t\t

\n\t\t\t\t\t

\n\t\t\t\t\t\t

06 / optional detail

\n\t\t\t\t\t\t

Technical evidence and next steps

\n\t\t\t\t\t

\n\t\t\t\t\t\t

Open implementation details, test results and source links

\n\t\t\t\t\t\t

\n\t\t\t\t\t\t\tRecorded contrast: Delete labels 4.20:1; timestamps/tool details 4.43:1; queued text\n\t\t\t\t\t\t\t2.21–2.64:1; measured code tokens 2.14–3.27:1; supplementary audit labels 4.12:1. The\n\t\t\t\t\t\t\tscan target is 4.5:1. Higher numbers mean a stronger difference between text and\n\t\t\t\t\t\t\tbackground. These are accepted shortfalls, not claims that the target is met. See the measured decision record and rejection tests.\n\t\t\t\t\t\t

\n\t\t\t\t\t\t

\n\t\t\t\t\t\t\tThe browser automation tool is Playwright; the automatic accessibility checker is axe.\n\t\t\t\t\t\t\tScreenshots are taken in a fixed Linux environment, with the same browser and fonts\n\t\t\t\t\t\t\tlocally and on GitHub, to reduce differences caused by the testing machine.\n\t\t\t\t\t\t

\n\t\t\t\t\t\t

\n\t\t\t\t\t\t\tVersions: Bun 1.4.2, Playwright 1.62.1, axe 4.13.0. Viewports: 1440 × 1000 and 390 ×\n\t\t\t\t\t\t\t844. The revised policy accepts only explicitly approved colour findings, including\n\t\t\t\t\t\t\ttheir measured colours, text size, markup and count. Other findings, changed\n\t\t\t\t\t\t\tmeasurements and stale exceptions fail. No screenshot masks are used. The\n\t\t\t\t\t\t\ttext-enlargement test changes the base text size; it is not a full browser-zoom test.\n\t\t\t\t\t\t

\n\t\t\t\t\t\t

\n\t\t\t\t\t\t\tThe original 28 September GitHub run, before the colour decision, tested source plus\n\t\t\t\t\t\t\treport commit faaa6563: 38 design checks passed in 2.6 minutes; their\n\t\t\t\t\t\t\tfull step, including setup, took 3 minutes 34 seconds. The 240 existing system tests\n\t\t\t\t\t\t\tpassed in 4.2 minutes. The separate validation and production-container jobs passed\n\t\t\t\t\t\t\ttoo.\n\t\t\t\t\t\t

\n\t\t\t\t\t\t

\n\t\t\t\t\t\t\tLocally, 1,658 unit tests passed, with two database-dependent suites skipped; database\n\t\t\t\t\t\t\tcontract tests passed on GitHub. The production build check confirmed that this\n\t\t\t\t\t\t\tdevelopment-only example page is not bundled into the shipped app.\n\t\t\t\t\t\t

\n\t\t\t\t\t\t

\n\t\t\t\t\t\t\tReview/integration note: the planned order is #207, #208, then #209. These changes\n\t\t\t\t\t\t\tshare some fixes. After combining them, the reviewed source-file acknowledgement needs\n\t\t\t\t\t\t\tupdating and the combined checks need another run. Passing each branch separately is\n\t\t\t\t\t\t\tnot evidence that their combination passes.\n\t\t\t\t\t\t

\n\t\t\t\t\t\t

\n\t\t\t\t\t\t\t\tProposed change #209 ·\n\t\t\t\t\t\t\t\tCompleted GitHub test run · Latest checks\n\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\tList of covered examples · How to run the tests and review pictures\n\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\tAll wide reference pictures · All narrow reference pictures\n\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\tDetailed findings · Recorded missing-name failure\n\t\t\t\t\t\t\t

\n\t\t\t\t\t

\n\t\t\t\t

\n\t\t\t