Skip to content

fix: use reference outputs for example creation [closes LSE-2783] - #3403

Draft
Zhen (yumuzi234) wants to merge 2 commits into
mainfrom
open-swe/lse-2783-sdk-reference-outputs
Draft

fix: use reference outputs for example creation [closes LSE-2783]#3403
Zhen (yumuzi234) wants to merge 2 commits into
mainfrom
open-swe/lse-2783-sdk-reference-outputs

Conversation

@yumuzi234

@yumuzi234 Zhen (yumuzi234) commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Description

Python and TypeScript example creation APIs now prefer reference-output terminology while retaining deprecated outputs aliases. Canonical JSON payloads use reference_outputs; multipart storage names remain unchanged. Merge and deploy the API compatibility PR before releasing this change.

Linear: https://linear.app/langchain/issue/LSE-2783/update-dataset-api-and-sdk-to-use-reference-outputs-instead-of-outputs
Depends on: https://github.com/langchain-ai/langchainplus/pull/34107 · UI: https://github.com/langchain-ai/langchainplus/pull/34106

Test Plan

  • Verify canonical, legacy, and conflicting values in Python and TypeScript client tests.
  • Run both SDK unit suites.

Made by Open SWE

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
@github-actions

github-actions Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

JS perf benchmark

Lower is better. Noisy on shared runners — treat as a signal, not a gate.

Base64-heavy payload

Single large base64 string per message — the shape the worker-offload path is optimized for.
Payload: 2511.2 KB in / 5.2 KB out, 100 runs.

metric main this PR delta
Wall time (ms) 1590.67 1812.92 +14.0%
createRun total (ms) 94.13 100.43 +6.7%
createRun p50 (ms) 0.83 0.83 +0.8%
createRun p95 (ms) 1.11 1.86 +67.6%
createRun p99 (ms) 7.72 7.76 +0.6%
createRun max (ms) 7.72 7.76 +0.6%
updateRun total (ms) 31.49 29.71 -5.7%
updateRun p95 (ms) 0.79 0.55 -29.9%
loop lag total (ms) 826.50 978.42 +18.4%
loop lag p50 (ms) 0.11 0.11 -3.2%
loop lag p95 (ms) 4.14 4.98 +20.3%
loop lag p99 (ms) 18.99 11.84 -37.6%
loop lag max (ms) 77.99 74.10 -5.0%

Structural payload

Many small strings across a wide/nested object graph. Should bypass worker offload and use sync flush.
Payload: 1239.5 KB in / 13.3 KB out, 100 runs.

metric main this PR delta
Wall time (ms) 2023.34 1815.18 -10.3%
createRun total (ms) 679.82 630.49 -7.3%
createRun p50 (ms) 6.02 5.71 -5.3%
createRun p95 (ms) 14.10 11.92 -15.5%
createRun p99 (ms) 21.94 12.28 -44.0%
createRun max (ms) 21.94 12.28 -44.0%
updateRun total (ms) 36.15 34.31 -5.1%
updateRun p95 (ms) 0.50 0.52 +4.3%
loop lag total (ms) 1587.03 1414.63 -10.9%
loop lag p50 (ms) 0.09 0.10 +4.2%
loop lag p95 (ms) 13.42 11.22 -16.4%
loop lag p99 (ms) 101.42 135.13 +33.2%
loop lag max (ms) 201.87 155.75 -22.8%

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant