Skip to content

Latest commit

History

History
35 lines (28 loc) 路 1.51 KB

File metadata and controls

35 lines (28 loc) 路 1.51 KB

Prompt Injection to Tool Misuse

A malicious prompt causes the agent to misuse a tool, breaching intended boundaries.

An attacker hides directives inside ordinary user input so the agent treats them as legitimate goals, calls a tool with attacker-chosen parameters, and reaches the downstream system before any policy check has a chance to fire.

sequenceDiagram
    participant U as Untrusted input
    participant A as Agent
    participant T as Tool
    participant D as Downstream system
    U->>A: Embedded instruction<br/>(Influence)
    note right of A: Intent reinterpreted,<br/>no policy check
    A->>T: Tool call without<br/>schema or scope check
    T->>D: Unsafe operation<br/>(Authority)
    D-->>A: Impact realised
Loading



Defence source-labels every input, separates instructions from data, forces a policy decision before any tool is invoked, and routes the call through a tool broker so that even a successful injection cannot reach the downstream system unchecked.

block-beta
columns 1
  L1["Source labelling<br/>on every input"]
  L2["Instruction-data<br/>separation"]
  L3["Policy decision<br/>before action"]
  L4["Tool broker:<br/>allowlist and schema validation"]
  L5["Outcome control<br/>and end-to-end trace"]
Loading