Skip to content

Commit f113027

Browse files
AminDhouibclaude
andauthored
feat(upload): Ctrl+V clipboard paste with preview, confirm, and progress on the web page (#72)
Pasting an image on the upload page now stages it instead of doing nothing: a preview panel shows the image, its size, and which project it will be sent to, and nothing uploads until the user confirms with Send. Send stays disabled until a project pill is picked (the destination line says so), picking a pill live-updates the target, and the upload itself runs over XHR so a real progress bar + percentage track it, ending in a confirmed "Pasted into <project> ✓" state (or a Retry-able error state). Cancel discards the staged image; pastes are ignored mid-flight so an in-progress upload can't be swapped out from under itself. Verified live in a real Chrome (DevTools protocol) against the built page: staging, gating, destination update, progress mid-states, success state, auto-clear, error/Retry path, and Cancel all behave as specified. Tests: a unit pin for the paste-UI page contract (panel, dest, bar, Send/Cancel, paste listener), and a new browser-tier Playwright test that synthesizes a ClipboardEvent with a real PNG and proves the full flow ends with the byte-identical paste-*.png on disk plus live tmux injection. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 8a3d2eb commit f113027

5 files changed

Lines changed: 242 additions & 3 deletions

File tree

CLAUDE.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ Open every project in its own terminal, launch an AI agent in it, and auto-tile
1212
- The `needs_ssh` tier is CI-only by design: `tests/e2e/test_ssh.py` (dry-run pins) + `tests/e2e/test_ssh_real.py` (REAL wire — every test traverses the loopback sshd that `.github/actions/setup-ssh-server` provisions, exporting `MDTEST_SSH_PORT`/`MDTEST_SSH_KEY`/`MDTEST_SSH_HOST` and an `mdssh` alias in the runner's real `~/.ssh/config`). Locally these skip because the env vars are absent — **never install/enable an SSH server on a dev machine to run them**. Tests that seed the ssh user's real HOME (the Windows attach flagship — attach can't inject `--config` remotely) are additionally gated on `GITHUB_ACTIONS=true` / `MDTEST_ALLOW_REAL_HOME=1`. macOS window-over-SSH legs are a loud `::warning` skip (see DESIGN.md known debt).
1313
- dist (packaged-install) locally: `pip install build` (not a dev extra; CI installs it per-job) then `pytest tests/dist/ -m dist` — builds a wheel, installs it into a pristine no-extras venv, and drives the real installed `multideck` entry point (proves the exact `pip install` user env; import-sweep catches dev/optional-dep leaks). Window-spawn leg is win32-only and skips elsewhere; wheel build + import sweep + virgin first-run + socket-real serve run on every OS.
1414
- Real-PTY menu tier (`tests/e2e/test_pty_menu.py`, marker `pty`, rides the existing `end-to-end` CI job on all 3 OSes): drives the installed `python -m multideck` under a genuine pseudo-terminal — pexpect on POSIX, pywinpty/ConPTY on Windows (`pywinpty>=2.0; sys_platform=='win32'` in the `dev` extra) — via the uniform driver `tests/e2e/_pty.py`, asserting on the plain on-screen text (ANSI stripped; children run `NO_COLOR=1`) and, for first-run, the valid config written to disk. This is the only test of the real `sys.stdin.isatty()` interactive path (`CliRunner` fakes the tty). Locally it runs anywhere pexpect/pywinpty resolve.
15-
- Browser-upload tier (`tests/e2e/test_upload_browser.py`, marker `browser`, dedicated `browser-upload` ubuntu job, CI-only, gated on `MDTEST_BROWSER=1`): a real headless Chromium (Playwright — the `browser` extra `["playwright>=1.40"]`, NOT in `dev`, provisioned only by that job via `playwright install --with-deps chromium`) drives the real mobile upload page against a real `multideck serve` on loopback and asserts the on-disk upload is byte-identical + the page title/form contract. Locally the module skips cleanly (no `MDTEST_BROWSER`, no Playwright). Honest gap: it symlinks real `tmux` as `psmux` and stands up a real tmux session as the multiplexer shim (Linux has no `psmux` binary) — see DESIGN.md.
15+
- Browser-upload tier (`tests/e2e/test_upload_browser.py`, marker `browser`, dedicated `browser-upload` ubuntu job, CI-only, gated on `MDTEST_BROWSER=1`): a real headless Chromium (Playwright — the `browser` extra `["playwright>=1.40"]`, NOT in `dev`, provisioned only by that job via `playwright install --with-deps chromium`) drives the real mobile upload page against a real `multideck serve` on loopback and asserts the on-disk upload is byte-identical + the page title/form contract + the Ctrl+V clipboard-paste confirm flow (synthetic ClipboardEvent → staged preview with Send gated on project selection → XHR upload → byte-identical `paste-*.png` on disk). Locally the module skips cleanly (no `MDTEST_BROWSER`, no Playwright). Honest gap: it symlinks real `tmux` as `psmux` and stands up a real tmux session as the multiplexer shim (Linux has no `psmux` binary) — see DESIGN.md.
1616
- The `needs_tailscale` tier is CI-only by design (`tests/e2e/test_tailnet_real.py` + the non-required `tailnet` ubuntu job): it joins a REAL ephemeral, tag-scoped Tailscale node (`tailscale/github-action`, SHA-pinned, OAuth via `TS_OAUTH_CLIENT_ID`/`TS_OAUTH_SECRET` + `tag:ci`) and exercises `tailnet.py`'s live `tailscale` probes (`ip4`/`magicdns_host`/`probe`) plus `multideck serve`'s Tailscale-facing default bind (`/health` answers on the real Tailscale IP; the LAN wildcard is proven unbound). Gated on `MDTEST_TAILSCALE=1` — locally it skips because the var is absent; **never install/join Tailscale on a dev machine to run it** (same posture as `needs_ssh`). The OAuth secrets don't exist yet: the job detects their absence and skips LOUDLY (`::warning`, job stays green, forks never fail) — once the user adds them it goes fully live with zero code change (one-time setup steps in the job comment). Excluded from the `end-to-end` job's SSH-less run exactly like `needs_ssh`.
1717
- The `monitor_lab` tier is CI-only by design (`tests/platform/test_monitor_lab_tiling.py` + the `monitor_lab` CI job, windows-latest): it INSTALLS the parsec-vdd virtual-display driver to fabricate a mixed-DPI multi-monitor topology, then drives real `--go` tiling across the virtual monitors and asserts each window rect lands in its `compute_grid` cell (physical pixels). Gated on `MDTEST_MONITOR_LAB=1` — locally it skips because the var is absent; **never install the driver / set the var on a dev machine to run it** (same posture as `needs_ssh`). The lab engine is `tests/platform/monitor_lab.py` (stdlib ctypes, win32-only, import-harmless on POSIX). The offline half — `tests/unit/test_monitor_lab_topologies.py` feeding committed golden topologies (`tests/platform/fixtures/topologies/*.json`) into `compute_grid` — runs everywhere in the normal unit gate. Two more CI-only Windows interaction tiers ride in the platform-integration windows leg (same never-on-a-dev-box posture): the `wt_cold` tier (`tests/platform/test_wt_cold_start.py`, gated `MDTEST_WT_COLD=1`) pins the first-ever `wt` launch on the fresh runner against the default settle+retry tiling budget — coldness is guaranteed by CI step ordering, so it must stay the first step that can spawn wt; and the `interaction` tier (`tests/platform/test_real_hotkey.py`, gated `MDTEST_INTERACTION=1`) drives the REAL Alt+V chain — product-spawned listener, real `SendInput` chord, real CF_DIB clipboard, real `serve` + psmux inject — and would stomp a dev machine's clipboard/keystrokes, so never set the var locally. The windows leg also installs pinned psmux (v3.3.6 release zip, sha256-verified, into `%LOCALAPPDATA%\psmux`) so `tests/platform/test_real_psmux.py` (lifecycle + full attach chain in a real wt window) runs instead of skipping. Riding in the same CI job (shared `lab` fixture + driver install via `tests/platform/conftest.py`): the doctor-replay tier — `multideck doctor --json` emits monitor topology under a `monitors` key, `tests/platform/doctor_replay.py` plans an achievable lab topology from a bug-report blob, `tests/platform/test_doctor_replay.py` replays committed sample reports (`tests/platform/fixtures/doctor_reports/*.json`) live, and `tests/unit/test_doctor_replay_offline.py` pins the parse/plan/slot math (incl. negative origins) everywhere.
1818
- Real-writer contract tier (`tests/e2e/test_state_sink_contract.py`, marker `node_contract`, rides the existing `end-to-end` job on all 3 OSes — node is preinstalled on every hosted runner, so no CI change): `npm install`s the pinned real agent-state writer package (`ai-agent-notifier@1.0.6` — see `AINS_VERSION`) into a tmp prefix, drives its REAL hook (`src/notify.mjs`) under real node with realistic Claude `Stop`/`Notification`/`SessionStart` events and a fully-redirected HOME, then checks multideck's real `agent_state` reader against it. It PINS a headline finding (F-NC-001): the package writes ZERO `~/.multideck/state/` records (it is a pure notifier; there is no `state-sink.mjs`) — so the tier currently asserts the store stays empty and flips RED if a wired hook ever starts writing records. Skips cleanly if node/npm or the npm registry are unavailable; tmp-dir only, never touches real `~/.multideck`/`~/.claude`.

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -142,7 +142,7 @@ Send screenshots from your phone straight into a project's agent session:
142142
"settings": { "psmux": true, "uploadServer": true, "uploadPort": 8033 }
143143
```
144144

145-
`multideck serve` (or `uploadServer: true` during launch) starts a small HTTP server; `multideck mobile` prints the phone URL + a QR code you can install as a home-screen app (the QR code needs the optional `qr` extra: `pip install multideck[qr]`). Pick a project on the phone, upload an image, and its path is pasted into that project's session. The Alt+V hotkey (Windows) does the same for whatever `md:` session is focused.
145+
`multideck serve` (or `uploadServer: true` during launch) starts a small HTTP server; `multideck mobile` prints the phone URL + a QR code you can install as a home-screen app (the QR code needs the optional `qr` extra: `pip install multideck[qr]`). Pick a project on the phone, upload an image, and its path is pasted into that project's session. On a desktop browser you can also **Ctrl+V** an image from the clipboard: the page stages it with a preview showing which project it will go to, waits for you to confirm with **Send**, and shows live upload progress until the "pasted into …" confirmation. The Alt+V hotkey (Windows) does the same for whatever `md:` session is focused.
146146

147147
This works **over Tailscale**: the server binds only the loopback and your machine's Tailscale IP — never the LAN wildcard — and `attach`/`mobile`/`termius` shell out to the `tailscale` CLI to resolve hosts. Devices must be on your tailnet; there is deliberately no auth token, since the bind set is the access control. To bind something else (e.g. LAN-wide), use the escape hatch: `multideck serve --host 0.0.0.0`.
148148

src/multideck/upload_server.py

Lines changed: 160 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -188,6 +188,26 @@ def _flash(
188188
.drop.ok{border-color:#a6e3a1;color:#a6e3a1}
189189
.drop.err{border-color:#f38ba8;color:#f38ba8}
190190
.drop input{position:absolute;inset:0;opacity:0;cursor:pointer;font-size:0}
191+
.paste{display:none;margin-bottom:8px;border:1.5px solid #45475a;border-radius:10px;
192+
padding:10px;background:#181825}
193+
.paste.show{display:block}
194+
.paste img{display:block;max-width:100%;max-height:40vh;border-radius:6px;
195+
margin:0 auto 8px;background:#11111b}
196+
.paste-meta{display:flex;justify-content:space-between;gap:8px;font-size:.75rem;
197+
color:#9399b2;margin-bottom:8px}
198+
#paste-dest{color:#89b4fa;font-weight:600}
199+
.bar{display:none;height:6px;border-radius:3px;background:#313244;overflow:hidden;
200+
margin-bottom:8px}
201+
.bar.show{display:block}
202+
#bar-fill{height:100%;width:0%;background:#89b4fa;transition:width .15s}
203+
.paste-actions{display:flex;gap:8px}
204+
.paste-actions button{flex:1;padding:9px;border:none;border-radius:8px;
205+
font-weight:700;font-size:.8rem;cursor:pointer}
206+
#paste-send{background:#a6e3a1;color:#1e1e2e}
207+
#paste-send:disabled{background:#45475a;color:#6c7086;cursor:not-allowed}
208+
#paste-send.ok{background:#a6e3a1}
209+
#paste-send.err{background:#f38ba8;color:#1e1e2e}
210+
#paste-cancel{background:#313244;color:#bac2de}
191211
.toast{font-size:.75rem;color:#6c7086;text-align:center;min-height:1.1em;
192212
transition:color .2s}
193213
.toast.ok{color:#a6e3a1}
@@ -205,7 +225,7 @@ def _flash(
205225
<body>
206226
<div class="head">
207227
<h1>MD</h1>
208-
<span>tap project &rsaquo; tap file &rsaquo; done</span>
228+
<span>tap project &rsaquo; tap file or Ctrl+V &rsaquo; done</span>
209229
</div>
210230
211231
<div class="pills" id="pills">PROJECTS_PLACEHOLDER</div>
@@ -214,6 +234,19 @@ def _flash(
214234
<span id="drop-label">select a project first</span>
215235
<input type="file" id="file" accept="image/*,video/*,.pdf,.txt,.json,.csv,.log" disabled>
216236
</div>
237+
238+
<div class="paste" id="paste-box">
239+
<img id="paste-img" alt="pasted image">
240+
<div class="paste-meta">
241+
<span id="paste-dest"></span>
242+
<span id="paste-size"></span>
243+
</div>
244+
<div class="bar" id="paste-bar"><div id="bar-fill"></div></div>
245+
<div class="paste-actions">
246+
<button id="paste-send" disabled>Send</button>
247+
<button id="paste-cancel">Cancel</button>
248+
</div>
249+
</div>
217250
<div class="toast" id="toast">&nbsp;</div>
218251
219252
<div class="install" id="install">
@@ -287,6 +320,132 @@ def _flash(
287320
}
288321
}, 2000);
289322
});
323+
324+
// Ctrl+V clipboard upload: stage the pasted image (preview + target project),
325+
// send only on explicit confirm, and show live upload progress. XHR instead of
326+
// fetch because only XHR exposes upload-progress events.
327+
const pbox = document.getElementById('paste-box');
328+
const pimg = document.getElementById('paste-img');
329+
const pdest = document.getElementById('paste-dest');
330+
const psize = document.getElementById('paste-size');
331+
const pbar = document.getElementById('paste-bar');
332+
const pfill = document.getElementById('bar-fill');
333+
const psend = document.getElementById('paste-send');
334+
const pcancel = document.getElementById('paste-cancel');
335+
let staged = null;
336+
let sending = false;
337+
338+
function fmtSize(b) {
339+
if (b < 1024) return b + ' B';
340+
if (b < 1048576) return (b / 1024).toFixed(1) + ' KB';
341+
return (b / 1048576).toFixed(1) + ' MB';
342+
}
343+
344+
function refreshPaste() {
345+
if (!staged) return;
346+
pdest.textContent = proj ? '→ ' + proj : 'select a project above';
347+
psend.disabled = sending || !proj;
348+
}
349+
pills.forEach(p => p.addEventListener('click', refreshPaste));
350+
351+
function clearStage() {
352+
if (staged) URL.revokeObjectURL(staged.url);
353+
staged = null;
354+
sending = false;
355+
pbox.className = 'paste';
356+
pbar.className = 'bar';
357+
pfill.style.width = '0%';
358+
psend.className = '';
359+
psend.textContent = 'Send';
360+
psend.disabled = true;
361+
pcancel.disabled = false;
362+
}
363+
364+
window.addEventListener('paste', e => {
365+
if (sending) return; // never swap the image out from under an upload
366+
const items = (e.clipboardData || {}).items || [];
367+
for (const it of items) {
368+
if (it.kind === 'file' && it.type.startsWith('image/')) {
369+
e.preventDefault();
370+
stageFile(it.getAsFile());
371+
return;
372+
}
373+
}
374+
});
375+
376+
function stageFile(file) {
377+
if (staged) URL.revokeObjectURL(staged.url);
378+
const ext = (file.type.split('/')[1] || 'png').replace('jpeg', 'jpg');
379+
const ts = new Date().toISOString().replace(/[-:]/g, '').slice(0, 15);
380+
staged = {file: file, url: URL.createObjectURL(file),
381+
name: 'paste-' + ts + '.' + ext};
382+
pimg.src = staged.url;
383+
psize.textContent = fmtSize(file.size);
384+
pfill.style.width = '0%';
385+
pbar.className = 'bar';
386+
psend.className = '';
387+
psend.textContent = 'Send';
388+
pbox.className = 'paste show';
389+
toast.textContent = '\u00a0';
390+
toast.className = 'toast';
391+
refreshPaste();
392+
}
393+
394+
function pasteFail(msg) {
395+
sending = false;
396+
psend.className = 'err';
397+
psend.textContent = 'Retry';
398+
psend.disabled = false;
399+
pcancel.disabled = false;
400+
toast.textContent = msg;
401+
toast.className = 'toast err';
402+
}
403+
404+
psend.addEventListener('click', () => {
405+
if (!staged || !proj || sending) return;
406+
sending = true;
407+
psend.className = '';
408+
psend.disabled = true;
409+
pcancel.disabled = true;
410+
psend.textContent = 'Sending 0%';
411+
pbar.className = 'bar show';
412+
413+
const form = new FormData();
414+
form.append('file', staged.file, staged.name);
415+
form.append('project', proj);
416+
form.append('inject', '1');
417+
418+
const xhr = new XMLHttpRequest();
419+
xhr.open('POST', '/upload');
420+
xhr.upload.addEventListener('progress', ev => {
421+
if (!ev.lengthComputable) return;
422+
const pct = Math.round(ev.loaded / ev.total * 100);
423+
pfill.style.width = pct + '%';
424+
psend.textContent = 'Sending ' + pct + '%';
425+
});
426+
xhr.addEventListener('load', () => {
427+
let d = {};
428+
try { d = JSON.parse(xhr.responseText); } catch (e) {}
429+
if (xhr.status === 200 && d.ok) {
430+
pfill.style.width = '100%';
431+
psend.className = 'ok';
432+
psend.textContent = (d.injected ? 'Pasted into ' + proj : 'Sent') + ' ✓';
433+
toast.textContent = staged.name
434+
+ (d.injected ? ' pasted into ' + proj : ' sent');
435+
toast.className = 'toast ok';
436+
setTimeout(clearStage, 2500);
437+
} else {
438+
pasteFail(d.error || 'upload failed');
439+
}
440+
});
441+
xhr.addEventListener('error', () => pasteFail('network error'));
442+
xhr.send(form);
443+
});
444+
445+
pcancel.addEventListener('click', () => {
446+
if (sending) return;
447+
clearStage();
448+
});
290449
</script>
291450
292451
<script>

tests/e2e/test_upload_browser.py

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@
2626

2727
from __future__ import annotations
2828

29+
import base64
2930
import http.client
3031
import json
3132
import os
@@ -307,3 +308,64 @@ def test_real_browser_upload_lands_byte_identical_file(serve, page, tmp_path):
307308
assert len(landed) == 1, f"expected exactly one upload, got {landed}"
308309
assert landed[0].read_bytes() == expected, "uploaded bytes differ on disk"
309310
assert landed[0].name.endswith("shot.png"), landed[0].name
311+
312+
313+
def test_clipboard_paste_upload_confirms_and_lands_byte_identical(
314+
serve, page, tmp_path
315+
):
316+
"""The Ctrl+V flow in a real browser: a paste event stages the image with
317+
a visible preview, the confirm step holds the upload back until a project
318+
is picked (Send disabled, destination line says so), picking the project
319+
names it as the target, and confirming uploads — ending in the confirmed
320+
success state with the exact pasted bytes on disk.
321+
322+
The paste itself is a synthesized ClipboardEvent (headless Chromium has no
323+
OS clipboard to press Ctrl+V against); everything downstream of the event
324+
— staging, preview, confirm gating, XHR POST, tmux injection, file write —
325+
is the real product path."""
326+
png_path = tmp_path / "clip.png"
327+
expected = _make_png(png_path)
328+
329+
page.goto(serve.url)
330+
expect(page).to_have_title("md upload")
331+
332+
page.evaluate(
333+
"""(b64) => {
334+
const bytes = Uint8Array.from(atob(b64), c => c.charCodeAt(0));
335+
const dt = new DataTransfer();
336+
dt.items.add(new File([bytes], 'clip.png', {type: 'image/png'}));
337+
window.dispatchEvent(new ClipboardEvent('paste', {clipboardData: dt}));
338+
}""",
339+
base64.b64encode(expected).decode(),
340+
)
341+
342+
# Staged: preview panel up, image shown, but NOT sent — no project yet.
343+
expect(page.locator("#paste-box")).to_be_visible()
344+
expect(page.locator("#paste-img")).to_be_visible()
345+
expect(page.locator("#paste-send")).to_be_disabled()
346+
expect(page.locator("#paste-dest")).to_contain_text("select a project")
347+
348+
# Picking the project flips the destination line and arms Send.
349+
page.locator(".pill", has_text=serve.TITLE).click()
350+
expect(page.locator("#paste-dest")).to_contain_text(serve.TITLE)
351+
expect(page.locator("#paste-send")).to_be_enabled()
352+
353+
page.locator("#paste-send").click()
354+
355+
# Confirmed: the button lands in the success state and the toast names the
356+
# live injection target (a real tmux session behind the psmux shim).
357+
expect(page.locator("#paste-send")).to_contain_text("✓")
358+
expect(page.locator("#toast")).to_contain_text("pasted into " + serve.TITLE)
359+
360+
landed = _wait_until(
361+
lambda: (
362+
sorted(serve.uploads_dir.glob("*")) if serve.uploads_dir.is_dir() else []
363+
),
364+
timeout=10,
365+
)
366+
assert landed, f"no file landed in {serve.uploads_dir}"
367+
assert len(landed) == 1, f"expected exactly one upload, got {landed}"
368+
assert landed[0].read_bytes() == expected, "pasted bytes differ on disk"
369+
assert "paste-" in landed[0].name and landed[0].name.endswith(".png"), landed[
370+
0
371+
].name

tests/unit/test_upload_server.py

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,24 @@ def test_pill_wire_value_is_session_id(self):
3737
html = _build_html([{"name": "my.api", "session": "my-api", "path": "x"}])
3838
assert 'data-name="my-api"' in html
3939

40+
def test_clipboard_paste_ui_ships_on_the_page(self):
41+
# Ctrl+V flow contract: the staged-image confirm panel (preview img,
42+
# destination-project line, progress bar, explicit Send/Cancel) and the
43+
# window paste listener must all be present in the served page. The
44+
# real-browser behavioural proof lives in the `browser` e2e tier.
45+
html = _build_html([{"name": "p", "path": "x"}])
46+
for anchor in (
47+
'id="paste-box"',
48+
'id="paste-img"',
49+
'id="paste-dest"',
50+
'id="paste-bar"',
51+
'id="paste-send"',
52+
'id="paste-cancel"',
53+
"addEventListener('paste'",
54+
"XMLHttpRequest",
55+
):
56+
assert anchor in html, f"paste-upload UI anchor missing: {anchor}"
57+
4058

4159
class TestConfigSessions:
4260
def test_carries_display_name_and_sanitized_session(self, tmp_path):

0 commit comments

Comments
 (0)