Skip to content

[Question]: External tool suspend still emits a synthetic tool result [Awaiting external execution] #2798

Description

@dragondyt

中文

问题

外部工具挂起时,运行时除了发出 RequireExternalExecutionEvent,为什么还会自己合成一条工具结果,内容是 [Awaiting external execution]

外部工具此时还没有真正执行。发出 ToolResultBlock / ToolResult* 事件序列,看起来像工具已经产出了 output,这和 HITL 约定冲突:真正的结果应该由外部回传。

当前行为

ToolExecutor.executeCore 在可用性与 schema 校验通过后,对 SchemaOnlyTool / @Tool(externalTool=true) 短路:

if (tool instanceof ToolBase tb && tb.isExternalTool()) {
    return Mono.just(ToolResultBlock.suspended(toolCall));
}

ToolResultBlock.suspended()[Awaiting external execution](或 ToolSuspendException.getReason())写入 output,并设置 metadata.agentscope_suspended = truestate 默认是 RUNNING

ReActAgent.runToolBatch 随后把它当成普通工具结果发出:

  1. ToolResultStartEvent
  2. ToolResultTextDeltaEvent,内容为 [Awaiting external execution]
  3. ToolResultEndEvent(RUNNING)
  4. 然后才是 RequireExternalExecutionEvent

buildSuspendedMsg() 还会把 ToolUseBlock 和这条合成的 ToolResultBlock 一起放进返回的 MsgGenerateReason.TOOL_SUSPENDED)。

为什么这看起来不对

  1. 两套信号叠在一起。 挂起已经由 RequireExternalExecutionEvent + GenerateReason.TOOL_SUSPENDED 表达。再伪造一条工具结果,等于同时说「这是 output」和「请到外面执行」。
  2. 占位文案和真实 output 共用同一字段。 消费方如果直接渲染 ToolResultBlock.output / ToolResultTextDeltaEvent,而不检查 isSuspended(),就会把 [Awaiting external execution] 当成工具成功返回。协议适配器必须特判(AG-UI 已在 ToolResultEndEvent(RUNNING) 时丢掉 buffer、不发 ToolCallResult;其它适配器未必如此)。
  3. 和权限 HITL 不一致。 RequireUserConfirmEvent 暂停时不会伪造工具结果。外部执行应该一样:暂停并等待,不要发明 output。
  4. 恢复时同一个 toolCallId 会出现第二次结果。 占位结果先被流式发出,调用方稍后注入真正的 ToolResultBlock。UI / trace / chat-completions 适配器可能对同一次调用看到两条结果。

期望行为(建议)

短路本身没问题:不要调用 callAsync,不要注入 preset,不要在本地执行。

需要改的是挂起的表达方式:

  • 不要把占位文案写入 ToolResultBlock.output
  • 不要对尚未执行的工具发出 ToolResultStart / TextDelta / End
  • 只通过 RequireExternalExecutionEvent 暴露待执行调用(返回 MsgGenerateReason.TOOL_SUSPENDED,带 ToolUseBlock,不要带假结果)。
  • 若 acting 循环仍需要内部标记,放在 metadata 或专用 state 里,不要放进 output

或者,如果必须保留 ToolResultBlock 做配对:output 留空 + 明确的 ToolResultState(例如专用 SUSPENDED,不要复用 RUNNING)+ 在外部结果到达前跳过 ToolResult* 事件。

想请维护者确认

  1. 这条合成的 [Awaiting external execution] 结果,是为了 LLM 上下文 / 协议兼容而有意设计的,还是把挂起编码成 ToolResultBlock 时留下的副作用?
  2. runToolBatchresult.isSuspended() 时,是否应该跳过 emitToolResultDelta / ToolResultEndEvent
  3. ToolResultBlock.suspended() 是否应该停止把占位文案写入 output

English

Question

When an external tool is suspended, why does the runtime itself emit a tool result with the placeholder [Awaiting external execution], in addition to RequireExternalExecutionEvent?

An external tool has not executed yet. Emitting a ToolResultBlock / ToolResult* event sequence makes it look like the tool already produced output. That conflicts with the HITL contract: the real result should come from outside the agent.

Current behavior

ToolExecutor.executeCore short-circuits SchemaOnlyTool / @Tool(externalTool=true) after availability + schema validation:

if (tool instanceof ToolBase tb && tb.isExternalTool()) {
    return Mono.just(ToolResultBlock.suspended(toolCall));
}

ToolResultBlock.suspended() fills output with [Awaiting external execution] (or ToolSuspendException.getReason()), and sets metadata.agentscope_suspended = true. state defaults to RUNNING.

ReActAgent.runToolBatch then treats this like a normal tool result:

  1. ToolResultStartEvent
  2. ToolResultTextDeltaEvent with [Awaiting external execution]
  3. ToolResultEndEvent(RUNNING)
  4. RequireExternalExecutionEvent

buildSuspendedMsg() also puts both the ToolUseBlock and this synthetic ToolResultBlock into the returned Msg (GenerateReason.TOOL_SUSPENDED).

Why this looks wrong

  1. Two overlapping signals. Suspend is already represented by RequireExternalExecutionEvent + GenerateReason.TOOL_SUSPENDED. A fabricated tool result is a second, conflicting signal: "here is the output" vs "please execute this outside".
  2. Placeholder lives in the same field as real output. Consumers that render ToolResultBlock.output / ToolResultTextDeltaEvent without checking isSuspended() will show [Awaiting external execution] as if the tool succeeded. Protocol adapters have to special-case this (AG-UI already drops the buffer on ToolResultEndEvent(RUNNING) and does not emit ToolCallResult; other adapters may not).
  3. Inconsistent with permission HITL. RequireUserConfirmEvent pauses without synthesizing a tool result. External execution should be the same: pause and wait, do not invent output.
  4. Resume produces a second result for the same toolCallId. The placeholder is streamed first; the caller later injects the real ToolResultBlock. UIs / traces / chat-completions adapters can see two results for one call.

Expected behavior (proposal)

The short-circuit itself is fine: do not invoke callAsync, do not inject presets, do not run locally.

What should change is how suspend is represented:

  • Do not put placeholder text into ToolResultBlock.output.
  • Do not emit ToolResultStart / TextDelta / End for a tool that has not executed.
  • Surface the pending call only via RequireExternalExecutionEvent (and GenerateReason.TOOL_SUSPENDED on the returned Msg, with ToolUseBlocks but no fake result).
  • If an internal marker is still needed for the acting loop, keep it in metadata / a dedicated state, not in output.

Alternatively, if a ToolResultBlock must exist for pairing: empty output + explicit ToolResultState (e.g. a dedicated SUSPENDED, not reused RUNNING) + skip ToolResult* events until the external result arrives.

Questions for maintainers

  1. Is the synthetic [Awaiting external execution] result intentional for LLM context / protocol compatibility, or leftover from encoding suspend as a ToolResultBlock?
  2. Should runToolBatch skip emitToolResultDelta / ToolResultEndEvent when result.isSuspended()?
  3. Should ToolResultBlock.suspended() stop writing placeholder text into output?

Environment

  • Module: agentscope-core
  • Related: ToolExecutor.executeCore, ToolResultBlock.suspended(), ReActAgent.runToolBatch, RequireExternalExecutionEvent

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions