Nightly E2E #140
threepointonenightly.yml
on: schedule
E2E: ai-chat (Playwright, deterministic)
3m 8s
E2E: ai-chat (Playwright, Workers AI)
5m 8s
E2E: think (chat recovery)
12m 4s
E2E: ai-chat (SIGKILL recovery)
5m 34s
E2E: agents (fiber eviction)
5m 9s
E2E: shared-engine genericity (pi + tanstack)
4m 57s
E2E: codemode (Playwright)
2m 33s
E2E: codemode (Playwright, Workers AI)
3m 1s
E2E: agents (browser connector)
2m 56s
E2E (deployed): ai-chat recovery on real edge
3m 36s
E2E (deployed): Think recovery probe on real edge
4m 56s
Annotations
26 errors, 11 warnings, and 3 notices
|
E2E: agents (browser connector)
Process completed with exit code 1.
|
|
e2e/codemode.spec.ts:73:3 › codemode e2e (Workers AI) › LLM generates and executes code that calls listProjects tool:
packages/codemode/e2e/codemode.spec.ts#L85
1) e2e/codemode.spec.ts:73:3 › codemode e2e (Workers AI) › LLM generates and executes code that calls listProjects tool
Error: expect(received).toBe(expected) // Object.is equality
Expected: true
Received: false
83 | // listProjects returns Alpha and Beta
84 | const lower = response.toLowerCase();
> 85 | expect(lower.includes("alpha") || lower.includes("beta")).toBe(true);
| ^
86 | });
87 |
88 | test("LLM generates code with multiple tool calls", async ({
at /home/runner/work/agents/agents/packages/codemode/e2e/codemode.spec.ts:85:63
|
|
E2E: shared-engine genericity (pi + tanstack)
Process completed with exit code 1.
|
|
[tanstack-recovery-e2e] e2e/workers-ai.test.ts > tanstack-recovery e2e (REAL Workers AI provider, non-deterministic) > continues a real-model turn from its survived partial after a SIGKILL:
experimental/tanstack-recovery/e2e/workers-ai.test.ts#L248
Error: turn committed before it could be killed mid-stream
❯ waitForBufferedContent e2e/workers-ai.test.ts:248:15
❯ e2e/workers-ai.test.ts:327:22
|
|
[tanstack-recovery-e2e] e2e/workers-ai.test.ts > tanstack-recovery e2e (REAL Workers AI provider, non-deterministic) > continues a real-model turn from its survived partial after a SIGKILL:
experimental/tanstack-recovery/e2e/workers-ai.test.ts#L248
Error: turn committed before it could be killed mid-stream
❯ waitForBufferedContent e2e/workers-ai.test.ts:248:15
❯ e2e/workers-ai.test.ts:327:22
|
|
[tanstack-recovery-e2e] e2e/workers-ai.test.ts > tanstack-recovery e2e (REAL Workers AI provider, non-deterministic) > continues a real-model turn from its survived partial after a SIGKILL:
experimental/tanstack-recovery/e2e/workers-ai.test.ts#L248
Error: turn committed before it could be killed mid-stream
❯ waitForBufferedContent e2e/workers-ai.test.ts:248:15
❯ e2e/workers-ai.test.ts:327:22
|
|
[tanstack-recovery-e2e] e2e/workers-ai.test.ts > tanstack-recovery e2e (REAL Workers AI provider, non-deterministic) > continues a real-model turn from its survived partial after a SIGKILL:
experimental/tanstack-recovery/e2e/workers-ai.test.ts#L248
Error: turn committed before it could be killed mid-stream
❯ waitForBufferedContent e2e/workers-ai.test.ts:248:15
❯ e2e/workers-ai.test.ts:327:22
|
|
e2e/llm.spec.ts:153:3 › LLM e2e (Workers AI) › server-side tool call: model calls getWeather and returns result:
packages/ai-chat/e2e/llm.spec.ts#L107
3) e2e/llm.spec.ts:153:3 › LLM e2e (Workers AI) › server-side tool call: model calls getWeather and returns result
Retry #3 ───────────────────────────────────────────────────────────────────────────────────────
SyntaxError: Unexpected token 'A', "An error occurred." is not valid JSON
105 | (m.body as string).trim()
106 | )
> 107 | .map((m) => JSON.parse(m.body as string));
| ^
108 | }
109 |
110 | test.describe("LLM e2e (Workers AI)", () => {
at map (/home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:107:22)
at extractChunkBodies (/home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:107:6)
at /home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:165:20
|
|
e2e/llm.spec.ts:153:3 › LLM e2e (Workers AI) › server-side tool call: model calls getWeather and returns result:
packages/ai-chat/e2e/llm.spec.ts#L107
3) e2e/llm.spec.ts:153:3 › LLM e2e (Workers AI) › server-side tool call: model calls getWeather and returns result
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
SyntaxError: Unexpected token 'A', "An error occurred." is not valid JSON
105 | (m.body as string).trim()
106 | )
> 107 | .map((m) => JSON.parse(m.body as string));
| ^
108 | }
109 |
110 | test.describe("LLM e2e (Workers AI)", () => {
at map (/home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:107:22)
at extractChunkBodies (/home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:107:6)
at /home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:165:20
|
|
e2e/llm.spec.ts:153:3 › LLM e2e (Workers AI) › server-side tool call: model calls getWeather and returns result:
packages/ai-chat/e2e/llm.spec.ts#L107
3) e2e/llm.spec.ts:153:3 › LLM e2e (Workers AI) › server-side tool call: model calls getWeather and returns result
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
SyntaxError: Unexpected token 'A', "An error occurred." is not valid JSON
105 | (m.body as string).trim()
106 | )
> 107 | .map((m) => JSON.parse(m.body as string));
| ^
108 | }
109 |
110 | test.describe("LLM e2e (Workers AI)", () => {
at map (/home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:107:22)
at extractChunkBodies (/home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:107:6)
at /home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:165:20
|
|
e2e/llm.spec.ts:153:3 › LLM e2e (Workers AI) › server-side tool call: model calls getWeather and returns result:
packages/ai-chat/e2e/llm.spec.ts#L107
3) e2e/llm.spec.ts:153:3 › LLM e2e (Workers AI) › server-side tool call: model calls getWeather and returns result
SyntaxError: Unexpected token 'A', "An error occurred." is not valid JSON
105 | (m.body as string).trim()
106 | )
> 107 | .map((m) => JSON.parse(m.body as string));
| ^
108 | }
109 |
110 | test.describe("LLM e2e (Workers AI)", () => {
at map (/home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:107:22)
at extractChunkBodies (/home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:107:6)
at /home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:165:20
|
|
e2e/llm.spec.ts:118:3 › LLM e2e (Workers AI) › SSE streaming: receives text-start:
packages/ai-chat/e2e/llm.spec.ts#L107
2) e2e/llm.spec.ts:118:3 › LLM e2e (Workers AI) › SSE streaming: receives text-start, text-delta(s), text-end from real model
Retry #3 ───────────────────────────────────────────────────────────────────────────────────────
SyntaxError: Unexpected token 'A', "An error occurred." is not valid JSON
105 | (m.body as string).trim()
106 | )
> 107 | .map((m) => JSON.parse(m.body as string));
| ^
108 | }
109 |
110 | test.describe("LLM e2e (Workers AI)", () => {
at map (/home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:107:22)
at extractChunkBodies (/home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:107:6)
at /home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:130:20
|
|
e2e/llm.spec.ts:118:3 › LLM e2e (Workers AI) › SSE streaming: receives text-start:
packages/ai-chat/e2e/llm.spec.ts#L107
2) e2e/llm.spec.ts:118:3 › LLM e2e (Workers AI) › SSE streaming: receives text-start, text-delta(s), text-end from real model
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
SyntaxError: Unexpected token 'A', "An error occurred." is not valid JSON
105 | (m.body as string).trim()
106 | )
> 107 | .map((m) => JSON.parse(m.body as string));
| ^
108 | }
109 |
110 | test.describe("LLM e2e (Workers AI)", () => {
at map (/home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:107:22)
at extractChunkBodies (/home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:107:6)
at /home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:130:20
|
|
e2e/llm.spec.ts:118:3 › LLM e2e (Workers AI) › SSE streaming: receives text-start:
packages/ai-chat/e2e/llm.spec.ts#L107
2) e2e/llm.spec.ts:118:3 › LLM e2e (Workers AI) › SSE streaming: receives text-start, text-delta(s), text-end from real model
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
SyntaxError: Unexpected token 'A', "An error occurred." is not valid JSON
105 | (m.body as string).trim()
106 | )
> 107 | .map((m) => JSON.parse(m.body as string));
| ^
108 | }
109 |
110 | test.describe("LLM e2e (Workers AI)", () => {
at map (/home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:107:22)
at extractChunkBodies (/home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:107:6)
at /home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:130:20
|
|
e2e/llm.spec.ts:118:3 › LLM e2e (Workers AI) › SSE streaming: receives text-start:
packages/ai-chat/e2e/llm.spec.ts#L107
2) e2e/llm.spec.ts:118:3 › LLM e2e (Workers AI) › SSE streaming: receives text-start, text-delta(s), text-end from real model
SyntaxError: Unexpected token 'A', "An error occurred." is not valid JSON
105 | (m.body as string).trim()
106 | )
> 107 | .map((m) => JSON.parse(m.body as string));
| ^
108 | }
109 |
110 | test.describe("LLM e2e (Workers AI)", () => {
at map (/home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:107:22)
at extractChunkBodies (/home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:107:6)
at /home/runner/work/agents/agents/packages/ai-chat/e2e/llm.spec.ts:130:20
|
|
e2e/advanced-client-tools.spec.ts:420:3 › Turn coalescing e2e › single tool result + autoContinue produces exactly one continuation:
packages/ai-chat/e2e/advanced-client-tools.spec.ts#L538
1) e2e/advanced-client-tools.spec.ts:420:3 › Turn coalescing e2e › single tool result + autoContinue produces exactly one continuation
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(received).toBeTruthy()
Received: null
536 | );
537 |
> 538 | expect(result.toolCallId).toBeTruthy();
| ^
539 | // Should have exactly 1 STREAM_RESUMING (one continuation)
540 | expect(result.resumingCount).toBe(1);
541 | // Exactly 2 done signals: original stream + continuation
at /home/runner/work/agents/agents/packages/ai-chat/e2e/advanced-client-tools.spec.ts:538:31
|
|
e2e/advanced-client-tools.spec.ts:420:3 › Turn coalescing e2e › single tool result + autoContinue produces exactly one continuation:
packages/ai-chat/e2e/advanced-client-tools.spec.ts#L538
1) e2e/advanced-client-tools.spec.ts:420:3 › Turn coalescing e2e › single tool result + autoContinue produces exactly one continuation
Error: expect(received).toBeTruthy()
Received: null
536 | );
537 |
> 538 | expect(result.toolCallId).toBeTruthy();
| ^
539 | // Should have exactly 1 STREAM_RESUMING (one continuation)
540 | expect(result.resumingCount).toBe(1);
541 | // Exactly 2 done signals: original stream + continuation
at /home/runner/work/agents/agents/packages/ai-chat/e2e/advanced-client-tools.spec.ts:538:31
|
|
E2E: think (chat recovery)
Process completed with exit code 1.
|
|
[think-e2e] src/e2e-tests/assistant-e2e.test.ts > think e2e — real LLM > uses workspace tools to write and read a file:
packages/think/src/e2e-tests/assistant-e2e.test.ts#L470
AssertionError: expected '' to contain 'hello'
- Expected
+ Received
- hello
❯ src/e2e-tests/assistant-e2e.test.ts:470:35
|
|
[think-e2e] src/e2e-tests/assistant-e2e.test.ts > think e2e — real LLM > uses workspace tools to write and read a file:
packages/think/src/e2e-tests/assistant-e2e.test.ts#L470
AssertionError: expected '' to contain 'hello'
- Expected
+ Received
- hello
❯ src/e2e-tests/assistant-e2e.test.ts:470:35
|
|
[think-e2e] src/e2e-tests/assistant-e2e.test.ts > think e2e — real LLM > uses workspace tools to write and read a file:
packages/think/src/e2e-tests/assistant-e2e.test.ts#L470
AssertionError: expected '' to contain 'hello'
- Expected
+ Received
- hello
❯ src/e2e-tests/assistant-e2e.test.ts:470:35
|
|
[think-e2e] src/e2e-tests/assistant-e2e.test.ts > think e2e — real LLM > uses workspace tools to write and read a file:
packages/think/src/e2e-tests/assistant-e2e.test.ts#L470
AssertionError: expected '' to contain 'hello'
- Expected
+ Received
- hello
❯ src/e2e-tests/assistant-e2e.test.ts:470:35
|
|
[think-e2e] src/e2e-tests/assistant-e2e.test.ts > think e2e — real LLM > streams a real LLM response:
packages/think/src/e2e-tests/assistant-e2e.test.ts#L406
AssertionError: expected false to be true // Object.is equality
- Expected
+ Received
- true
+ false
❯ src/e2e-tests/assistant-e2e.test.ts:406:26
|
|
[think-e2e] src/e2e-tests/assistant-e2e.test.ts > think e2e — real LLM > streams a real LLM response:
packages/think/src/e2e-tests/assistant-e2e.test.ts#L406
AssertionError: expected false to be true // Object.is equality
- Expected
+ Received
- true
+ false
❯ src/e2e-tests/assistant-e2e.test.ts:406:26
|
|
[think-e2e] src/e2e-tests/assistant-e2e.test.ts > think e2e — real LLM > streams a real LLM response:
packages/think/src/e2e-tests/assistant-e2e.test.ts#L406
AssertionError: expected false to be true // Object.is equality
- Expected
+ Received
- true
+ false
❯ src/e2e-tests/assistant-e2e.test.ts:406:26
|
|
[think-e2e] src/e2e-tests/assistant-e2e.test.ts > think e2e — real LLM > streams a real LLM response:
packages/think/src/e2e-tests/assistant-e2e.test.ts#L406
AssertionError: expected false to be true // Object.is equality
- Expected
+ Received
- true
+ false
❯ src/e2e-tests/assistant-e2e.test.ts:406:26
|
|
E2E: codemode (Playwright)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: pnpm/action-setup@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
E2E: agents (browser connector)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: pnpm/action-setup@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
E2E: codemode (Playwright, Workers AI)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: pnpm/action-setup@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
E2E: ai-chat (Playwright, deterministic)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: pnpm/action-setup@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
E2E (deployed): ai-chat recovery on real edge
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: pnpm/action-setup@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
E2E: shared-engine genericity (pi + tanstack)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: pnpm/action-setup@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
E2E (deployed): Think recovery probe on real edge
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: pnpm/action-setup@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
E2E: agents (fiber eviction)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: pnpm/action-setup@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
E2E: ai-chat (Playwright, Workers AI)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: pnpm/action-setup@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
E2E: ai-chat (SIGKILL recovery)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: pnpm/action-setup@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
E2E: think (chat recovery)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: pnpm/action-setup@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
🎭 Playwright Run Summary
1 flaky
e2e/codemode.spec.ts:73:3 › codemode e2e (Workers AI) › LLM generates and executes code that calls listProjects tool
7 passed (19.7s)
|
|
🎭 Playwright Run Summary
30 passed (22.2s)
|
|
🎭 Playwright Run Summary
8 failed
e2e/llm.spec.ts:118:3 › LLM e2e (Workers AI) › SSE streaming: receives text-start, text-delta(s), text-end from real model
e2e/llm.spec.ts:153:3 › LLM e2e (Workers AI) › server-side tool call: model calls getWeather and returns result
e2e/llm.spec.ts:201:3 › LLM e2e (Workers AI) › message persistence with real LLM: messages survive reconnection
e2e/llm.spec.ts:245:3 › LLM e2e (Workers AI) › multi-step tool use: model uses addNumbers tool ─
e2e/llm.spec.ts:283:3 › LLM e2e (Workers AI) › step boundaries: multi-step response has start-step and finish-step
e2e/llm.spec.ts:312:3 › LLM e2e (Workers AI) › clear history works after LLM conversation ──────
e2e/llm.spec.ts:354:3 › LLM e2e (Workers AI) › multi-turn context: LLM remembers previous messages
e2e/llm.spec.ts:452:3 › LLM e2e (Workers AI) › OpenAI metadata sanitization: no itemId in persisted messages
2 flaky
e2e/advanced-client-tools.spec.ts:420:3 › Turn coalescing e2e › single tool result + autoContinue produces exactly one continuation
e2e/llm.spec.ts:497:3 › LLM e2e (Workers AI) › multi-tab tool streaming: second connection sees tool-input-start and tool-output-available
9 passed (2.2m)
|