Syntactically invalid code blocks in answers - #672
Draft
langsmith-engine-dev[bot] wants to merge 1 commit into
Draft
Syntactically invalid code blocks in answers#672langsmith-engine-dev[bot] wants to merge 1 commit into
langsmith-engine-dev[bot] wants to merge 1 commit into
Conversation
…parse before answers are sent Links had a real validation tool while code blocks were only checked for markdown cosmetics, so python blocks with JS comments and langgraph.json snippets with mismatched brackets shipped to users as copy-pasteable code.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The agent emits fenced code and configuration snippets that do not parse in the language the fence declares — a
pythonblock using a JavaScript//comment token, and a generatedlanggraph.jsonwhose array is opened with[and closed with). The pre-send checklist in the docs agent prompt validates links with a realcheck_linkstool and validates only markdown cosmetics for code (that a block is wrapped in a language fence), so nothing ever verifies that the code inside the fence is parseable or that config snippets are reproduced verbatim from retrieved docs. Every affected answer reads as authoritative and is copy-pasted straight into user code or config, where it fails immediately with a syntax error.