Convert pasted plain-text Markdown into document blocks #360
- MA
## Why Plain-text Markdown pasted from a terminal or writing tool stayed as literal markers such as `##`, `**`, and fences. Structured HTML paste already converted.
## What changed - Detect clear Markdown in ordinary plain-text paste, parse it through the restricted dialect, and insert validated document blocks at the selection. - Keep protected components, raw HTML, imports, expressions, and ambiguous prose literal. Rejected content falls back to ordinary paste. - Preserve literal paste inside code, math, inline code, and table cells, as well as the explicit paste-as-plain-text shortcut. Structured HTML still takes precedence.
## Verification 17 focused paste tests, `bun run types`, `bun run ci`, and `bun run build` pass on the restacked branch. GitHub's browser suite will run the Markdown paste scenarios on the final published head.
## Visuals 

- MA
**Hold — please don't merge yet.** This adds a paste path through the restricted dialect parser; a security-focused review is in progress. I'll post the verdict here.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
- MA
**Review:** Changes needed
The feature works in a real browser. Headings, lists, code, tables, quotes and tasks land as blocks and sync to the second client. Component tags, `<img onerror>`, `{expr}` and `import` stay escaped literal text (`\<Questionnaire …`), so a paste can't create a projection. The new `parse()` options default to the old behaviour, and existing callers are unchanged. Unit tests pass (621) and CI is green. Two problems need fixing before merge.
### 1. A paste can trigger `plan:reset` for the whole room (blocking)
`pastedMarkdown` (`packages/editor/src/markdown-paste.ts`, the `validate(tree)` check) only runs the dialect `validate()`. The server's batch check in `room.apply` also runs `assertIntroducedUrls` (hidden or control characters, whitespace-hidden schemes, `//host` and `\\host`). Links are covered by the existing `link-guard.ts` transform. **Images are not.**
Reproduced on a fake-GitHub server with two clients:
``` paste: Look **here**  server: [plan] rejected batch: bad-image A recv: {"kind":"plan:reset","reason":"rebuilt"} B recv: {"kind":"plan:reset","reason":"rebuilt"} ```
Everyone in the room is rebuilt under a new epoch and loses undo history and cursors, and an empty list item is left behind. Before this PR, a plain-text paste could not create an image node at all. A zero-width character in a copied URL is a realistic accident, not only an attack.
In a headless run, `//evil.example`, `\\evil.example`, `<\u0001javascript:…>` and `<java script:…>` links also pass `pastedMarkdown`, and the server would reject them. In the browser `link-guard` unwraps those links, so today the risk comes only from images. That protection depends on a separate plugin being mounted.
**Fix:** in `pastedMarkdown`, also call `assertIntroducedUrls([], tree.children)` inside the try block and fall back to a literal paste when it throws. Add tests for a hidden-character image and a `//host` link.
Smaller variants of the same problem: `validate` sees only the pasted tree, not the document. A paste that pushes the document past `MAX_IMAGES` (100), or past `MAX_DEPTH` once nested at the caret, is accepted by the client and rejected by the server. Consider passing the current image count and depth, or skipping conversion when the paste contains images and the document is near the limit.
### 2. Ordered lists lose their numbers (UX bug)
`"1986. A great year.\n1987. Another one."` is detected as Markdown and stored as `1. A great year.\n2. Another one.`, so the years are lost. This affects any pasted ordered list that doesn't start at 1 (`5. … 6. …`). Either stop counting `\d+[.)]` items whose first number isn't `1` (tighten `LIST_ITEM`), or keep the `start` through `$createPlanNodes` and export. Add `1986. …` to the "leaves ordinary prose alone" test.
### Minor
- **⇧⌘V detection** (`registerMarkdownPaste`, the KEY_DOWN handler). `event.key.toLowerCase() === "v"` misses non-Latin layouts (Russian `м`), and Option combinations, where `key` becomes a symbol. Safari's and macOS's "Paste and Match Style" is ⌥⇧⌘V. Check `event.code === "KeyV" || event.key.toLowerCase() === "v"`. The Edit-menu "Paste and Match Style" fires no keydown at all, so it will still convert. Browsers expose no `inputType` for match-style (`beforeinput` reports `insertFromPaste` for both), so the keydown flag is the practical route. Note the menu limitation in the module comment. - **Quadratic regexes.** `TABLE` (`^.*\|.*\n`) and `LINK` (`\]\([^)\s]+`) backtrack quadratically. A 128 KB single line of `a|` takes about 1.5 s, and 128 KB of `[x](` about 3 s, on the main thread inside the paste handler. Skip detection above `MAX_SOURCE_BYTES`, or tighten the patterns: `^[^\n|]*\|[^\n]*\n`, and `[^()\s]` with a length cap for the URL. - **No size check.** A 368 KB list paste converts in about 1.1 s. Pass `{ bytes }` to `validate` so a paste larger than the document limit stays literal. A literal paste that size is also rejected by the server, so this is not new. - **Lazy continuation.** Email-style text (`Remember:\n- tickets\n- hotel\nThanks`) becomes a list, and `Thanks` is folded into the last item as a continuation line. That is standard Markdown, but surprising in prose.
- MA
Addressed in da4f20e8. CI is green.
1. **Server rejection (blocker).** `pastedMarkdown` now runs `assertIntroducedUrls([], tree.children)` and `validate(tree, { bytes })`. It also checks the document as it stands: the images already in it plus the pasted ones must stay within `MAX_IMAGES`, and the nesting at the caret plus the pasted tree's depth must stay within `MAX_DEPTH`. The depth estimate leans deep, so borderline cases paste as plain text. Any failure falls back to a literal paste. New tests cover a zero-width-character image, `//host` and `\\host` links, the image limit, the depth limit, and pastes over the byte limit (ASCII and multibyte). 2. **Ordered lists.** Detection counts numbered lines only once a list starts at 1, so `1986. … 1987. …` on its own stays prose. The dialect writes every ordered list from 1, so a list that starts elsewhere in a converted paste becomes numbered lines of text (`1986\. A **great** year.`) instead of being renumbered. Tested. 3. **⇧⌘V.** `isPlainPasteKey` accepts `code === "KeyV"` or `key` `v`, so ⌥⇧⌘V and non-Latin layouts work. The module comment notes that the Edit-menu "Paste and Match Style" still converts. Tested. 4. **Performance.** Detection is skipped above `MAX_SOURCE_BYTES`. Fences, tables and lists are now checked in one pass over the lines, and the inline patterns exclude their own delimiters, so matching takes linear time. A test runs 128 KB of `a|`, `[x](`, `**a`, `~~a`, `[[[` and unclosed fences, each under 200 ms. 5. **Lazy continuation.** An unindented line that isn't a list item now ends the list, so `Thanks` becomes its own paragraph. Indented continuation lines are kept. Tested.
- MA
**Review:** Changes needed
I re-reviewed da4f20e8 in a real browser on a fake-GitHub server, with two clients watching the socket for `plan:reset`. The blocker is fixed. One issue remains: a crafted paste of a few KB can freeze the tab.
### Verified (no `plan:reset` in any case; the server logged no `rejected batch`) - **Unsafe URLs paste as plain text:** a zero-width space in an image URL or in an `https` link, `//host`, `\\host`, a `\u0001javascript:` link, `<java script:…>`, and an image URL with leading whitespace. `[site](https://ok.example)` still becomes a link. - **Image limit:** pasting 99 images converts. Then pasting 2 more stays plain text (escaped `!\[i100]…`). Then pasting 1 more converts, for exactly 100 in the stored source. - **Nesting at the caret:** with the caret inside a 6-level list, nested lists of 6, 4 or 3 levels stay plain text, and 2 or 1 levels convert. A 9-level list at the root stays plain text; the estimate is one level cautious, as you said. - **Ordered lists:** `1986. … 1987. …` stays prose. `5. five / 6. six` stays prose. With bold, `1986. A **great** year.` becomes a numbered paragraph. `1. 2.` becomes a list. A nested `3. 4.` inside a bullet item becomes lines. - **Lists end at prose:** `Thanks` becomes its own paragraph. An indented continuation line stays inside its item. - **⇧⌘V:** the `isPlainPasteKey` tests cover `KeyV` with `м` and `◊`. Headless Chromium does not run a paste for real Shift and Option shortcuts, so I couldn't press the actual keys. - **Realistic large pastes are fast:** a 22 KB, 300-item list with bold, code and links takes 82 ms. A 20 KB document of 200 sections takes 107 ms. - 627 unit tests pass and CI is green.
### Remaining: a short crafted paste freezes the tab for seconds to minutes Detection is linear now, but the conversion still runs on the main thread. Lexical's insertion slows down quadratically when one paragraph holds thousands of formatted runs. Timings in the browser for paste plus first frame, offline:
| Paste | Markdown | Literal (⇧⌘V) | |---|---|---| | `"**a".repeat(2000)` (6 KB) | 1.5 s | 55 ms | | `"**a".repeat(4000)` (12 KB) | 10.9 s | 170 ms | | `"**a".repeat(8000)` (24 KB) | 85 s | 0.6 s |
The 128 KB case from your perf test never finished: the tab hung for more than 10 minutes. That test only times `looksLikeMarkdown`, not the full paste. `parse()` also slows down quadratically on runs of `*`: 32 KB of `## h\n` plus `*a` repeated takes 1.3 s on its own. Only the person pasting is affected, but their tab hangs and the paste never reaches the room.
**Fix:** after parsing, fall back to a plain-text paste when any parent holds more than about 200 phrasing children, or the tree has more than a few thousand nodes. Optionally also cap conversion at about 64 KB so `parse()` stays well under a second. Add a test that times `$pasteMarkdown` end to end on `"**a".repeat(4000)`.
### Not this PR (for the record) - A plain-text paste that takes the document past the open-question expiry reserve (`fitsExpiry`, `source-too-large`) is still refused by the server. It behaves the same on `main`. - A 128 KB literal paste of `a|` repeated took 7.7 s in Lexical. On my fake server the resulting batch was rejected and the server then lost its writer lease and shut down. Both behave the same as `main`, but are worth a separate issue.
- MA
Fixed in 7c97950b. CI is green.
- **Before anything is inserted**, `pastedMarkdown` checks the parsed tree and pastes literally if any of these hold: - the text is over 64K characters (checked before parsing), - the tree has more than 2,000 nodes, - any block holds more than 200 children. - **The root isn't capped by child count**, only by the 2,000-node total. Inserting whole blocks isn't the quadratic path: 450 bold paragraphs convert in about 30 ms headless. - **New timing test** (`a fragmented paste falls back to literal text quickly`) runs the full `$pasteMarkdown` path in a headless editor: - Must stay literal, each under 200 ms: `**a`×4000, 128 KB of `**a`, 1,000 inline code runs, and a 1,500-item link list. - Must still convert under 200 ms: the 450-paragraph paste.
- MA
**Review:** Changes needed
I checked 7c97950b in a real browser (fake-GitHub server) and in headless Lexical. The pathological pastes no longer freeze the tab. But the new 2,000-node cap is now too tight for ordinary documents, which defeats the purpose of the feature. Both fixes are small.
### Fixed | Paste | Before (da4f20e8) | Now | |---|---|---| | `"**a".repeat(4000)` (12 KB) | 10.9 s | 130–210 ms, literal | | `"**a".repeat(8000)` (24 KB) | 85 s | 0.5–0.8 s, literal | | 9 paragraphs × 198 children (just under the caps) | not tested | 40–60 ms |
`"**a"` at 128 KB is skipped by the character cap. It then takes about 14 s in Lexical's own literal paste, which is the same on `main`. No regressions: - Basic Markdown still converts. - A zero-width-space image URL and a `//host` link stay literal. - `1986.` stays prose. - `Thanks` is split out of the list. - `<Questionnaire>` stays literal. - No `plan:reset` from any of these.
### 1. Ordinary documents of about 12 KB now paste as raw Markdown (blocking) `MAX_PASTE_NODES = 2_000` counts every MDAST node. A typical section (`## Section`, a paragraph with **bold** and *emphasis*, a 2-item list) is about 17 nodes. In the browser:
| Paste | Result | |---|---| | 120 sections (12 KB) | **literal** (`## Section 119` left as text) | | 200 sections (20 KB) | **literal** | | 300-item list with bold, code and links (22 KB) | **literal** | | 200-item list (15 KB) | converts |
A long Claude answer or a README is exactly what this PR is for, and it now comes in as raw `##` and `**` without any explanation. The freeze was caused by too many children in one block, and the 200-children cap already prevents that. Before this commit, a 2,800-node, 20 KB document converted in about 100 ms. **Suggest raising `MAX_PASTE_NODES` to about 20,000**, or dropping it and keeping only the per-block cap plus the character cap. Add a test showing a 20 KB multi-section document converts.
### 2. `parse()` still runs before the size checks (should fix) The node count only runs after `parse()`, and micromark slows down quadratically on runs of single `*`. Below the 64K-character cap, `"## h\n" + "*a".repeat(N)` takes 0.8–2 s at 32 KB and 3–9 s at 64 KB in the browser, and 5.2 s for `parse` alone in Bun. Any paste with thousands of `*` or `_` delimiters would go over the node cap anyway, so a cheap check before parsing loses nothing. For example, take the literal paste when `(text.match(/[*_]/g)?.length ?? 0) > 4_000`.
### 3. The 200 ms timing test risks flaking on CI On an M-series Mac the slowest case takes 77–85 ms (`"**a".repeat(4000)`), and the 450-paragraph case 38–45 ms. That leaves only about 2.3× headroom. Shared Linux runners and coverage or GC pauses commonly use that up. The structural assertions (`handled === false`) already prove which path ran. Keep the timing only as a coarse guard, around 1,000 ms, or compare against a literal paste of the same text (for example, no more than 10× slower).
Unit tests pass and CI is green.
- MA
Fixed in c932cc1d. CI is green.
1. **Long documents:** `MAX_PASTE_NODES` is now 20,000. The 200-children cap on any non-root node still prevents the freeze. New test: a 200-section document of about 21 KB, each section a heading, a bold-and-link paragraph and a two-item list, converts. It takes about 70 ms headless. 2. **Parser guard:** a paste with more than 4,000 `*`/`_` characters takes the literal path before `parse()` runs. The timing test adds `"a *"` × 5000 behind a heading. 3. **Timing assertions:** loosened to 1,000 ms. The `handled === false` checks still prove the literal path.
- MA
**Hold lifted.** Security review passed (no paste can trigger a server batch rejection; projections stay literal), and the final perf items are addressed in c932cc1d (20k node cap, pre-parse emphasis guard, looser timing bounds). CI fully green.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
MaggieAppletonhead-ref-force-pushed 3d4855cb523flast month on GitHub
MaggieAppletonauto-rebase-enabledlast month on GitHub
MaggieAppletonmerged 7f95d854c9cclast month on GitHub
MaggieAppletonhead-ref-deletedlast month on GitHub