Interactive UI Linker (VS Code Extension)

Interactive UI Linker opens your running web app inside VS Code, lets you click UI elements, and produces structured context that you can insert into prompts or editor files while iterating with an agent.

What This Extension Does

• Opens a live web app preview in a VS Code webview panel.
• Injects a picker script into the page through a local proxy endpoint.
• Lets you click elements and captures:
  • CSS selector
  • XPath
  • tag name
  • text snippet
  • key attributes (id, class, role, data-testid, etc.)
  • bounds (x, y, width, height)
• Opens an inline comment box next to the selected element.
• Sends the request to Copilot when you press Enter in that inline box.
• Copies selection context as JSON or as a prompt-ready block.
• Inserts the latest selection block directly into the active editor.

Commands

• Interactive UI Linker: Open Picker Panel
• Interactive UI Linker: Insert Latest Selection into Editor
• Interactive UI Linker: Copy Latest Selection as JSON
• Interactive UI Linker: Copy Latest Selection Prompt

Quick Start

1. Install dependencies.

   npm install

2. Compile extension.

   npm run compile

3. Open this folder in VS Code.

4. Press F5 to launch an Extension Development Host.

5. Run the command Interactive UI Linker: Open Picker Panel.

6. Enter your app URL (for example http://localhost:3000) and click Load URL.

7. Click any UI element.

8. In the inline box that appears next to it, write your request and press Enter to send to Copilot.

9. Optionally use Copy Prompt or Insert for manual workflows.

Settings

• interactiveUiLinker.defaultUrl: default URL used when opening the panel.

Architecture

• Extension host (src/extension.ts):
  • registers commands
  • manages latest selection state
  • hosts local HTTP proxy (/preview?target=...)
  • injects picker script into HTML
• Webview UI:
  • loads proxied page in an iframe
  • toggles pick mode
  • shows latest selection JSON
  • sends MCP-style message envelopes (intent, prompt, notify) back to the extension

Notes and Current Limitations

• Works best with local dev servers and pages that allow iframe embedding.
• If a target page blocks iframe embedding via X-Frame-Options or strict CSP, preview may fail.
• The current proxy injects picker script into HTML responses from /preview. It does not rewrite all deep navigations across full-page reloads.

Future Improvements

• Add a navigation-aware proxy to preserve picker injection on cross-page navigation.
• Add history panel for multi-element selection batches.
• Add a command to push selected context directly into a chosen agent workflow.