githubnext/ace2Public

Ace: a collaborative coding environment on the team's own machines

Let native agents inspect image files by path

IA

iamnbutler opened ·

The installed pi text reader rejects images with `unsupported_image`. Composer uploads and `desktop_inspect` already deliver images, but an agent cannot directly inspect a screenshot or render written to disk. Opening an image in a native viewer is an indirect workaround.

## Acceptance

- Read supported image files by path on the execution host and return bounded model-visible images. - Show the same images in tool results and preserve them when reopening the channel through pi's existing history. - Report missing, unsupported, and oversized files clearly; preserve local/hosted execution ownership. - Generate a screenshot/render in a lane, inspect it with a real native model, correct a visible defect, and reopen its result.

This is separate from incoming non-image uploads (#26) and live desktop captures (#8).

Dogfooding core/QOL: #5. [Review evidence](https://github.com/githubnext/ace2/issues/5#issuecomment-5986485157).

  1. iamnbutlerparent-issue-addedlast month on GitHub

  2. iamnbutlerreferenced this in [Meta] Dogfood Ace end to end #5last month on GitHub

  3. iamnbutlerreferenced this in fix(app): stream the reply draft inside the timeline instead of over it #114last month on GitHub

  4. iamnbutlerreferenced this in feat(app): fold channels idle for 6 hours into an Inactive group #126last month on GitHub

  5. IA
    iamnbutler
    Author

    Another occurrence while validating #157 in Ace. The channel agent saved Playwright screenshots of the real app under `/tmp/2fc3e21eed858614-team-channel-sections/shots/` and tried to inspect them with its `read` tool. The call failed with: `…/shots/00-empty.png is an image (image/png); reading images is not supported`. `desktop_inspect` wasn't an alternative because the macOS session was locked.

    Workaround used: the agent checked each state through the browser's ARIA snapshots, page text, and the host's own catalog. A Codex agent outside Ace viewed the PNGs to review them visually. Screenshot review is the part this issue should let an Ace agent do itself.

  6. iamnbutlerreferenced this in feat(app): group projects by repository with teammate channel sections #157last month on GitHub

  7. iamnbutlerreferenced this in fix(ui): keep unsent composer drafts across restart #165last month on GitHub

  8. iamnbutlerreferenced this in Desktop tools unavailable when a source CLI starts a worker in an installed app's data directory #128last month on GitHub

  9. iamnbutlerreferenced this in feat(app): rebuild Settings and top-anchor every dialog #171last month on GitHub

  10. iamnbutlerreferenced this in fix(ui): render timeline text with the line breaks it was measured with #175last month on GitHub