feat: upgrade to substrait 0.63 and migrate extension URIs to URNs - #16
Open
EpsilonPrime wants to merge 1 commit into
Open
feat: upgrade to substrait 0.63 and migrate extension URIs to URNs#16EpsilonPrime wants to merge 1 commit into
EpsilonPrime wants to merge 1 commit into
Conversation
Bump the substrait crate 0.61 -> 0.63 and the proto submodule to the matching spec commit (spec 0.85.0 / 2aaae7c0). The 0.63 protos remove the deprecated SimpleExtensionURI message and the extension_uris / extension_uri_reference fields in favor of SimpleExtensionURN / extension_urns / extension_urn_reference, so migrate all URI handling to URNs: - Converter/printer/parser switched from extension_uris to extension_urns and from SimpleExtensionUri to SimpleExtensionUrn; FunctionData's extension_uri_reference renamed to extension_urn_reference. - Grammar: the extension_space token now accepts URN strings (extension:<owner>:<id>) which contain ':' characters. - Visitor generator: disambiguate leaf names that now collide because NamedTable, PrecisionTimestamp, PrecisionTime and IntervalCompound each exist in two message scopes, via NAME_OVERRIDES. - Printer: add not-yet-implemented arms for the new Lambda / LambdaInvocation expressions and Func type (full support lands later in the stack). - Migrate all TPC-H roundtrip fixtures and the q6 converter fixture from yaml URIs (/functions_X.yaml) to URNs (extension:io.substrait:functions_X). All 48 tests pass.
Member
Author
Missing / stubbed features introduced in this PR (for later decisions)The URN bump pulled in new 0.63 proto types that this PR only stubs (they compile and round-trip, but have no textplan syntax and emit
These need a text-syntax design + fixtures to implement fully. Deferred per the stack plan; PR #17 lists the full consolidated gap list. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR 1 of a stack upgrading substrait-textplan to the latest substrait protos, then switching to substrait-packaging.
What
Bump the
substraitcrate 0.61 → 0.63 and the proto submodule to the matching spec commit (spec 0.85.0 /2aaae7c0). The 0.63 protos remove the deprecatedSimpleExtensionURImessage and theextension_uris/extension_uri_referencefields in favor ofSimpleExtensionURN/extension_urns/extension_urn_reference, so this migrates all URI handling to URNs.Changes
extension_uris→extension_urnsandSimpleExtensionUri→SimpleExtensionUrn;FunctionData.extension_uri_reference→extension_urn_reference.extension_spacetoken now accepts URN strings (extension:<owner>:<id>) which contain:— the old fragment-basedURItoken couldn't tokenize them.NamedTable,PrecisionTimestamp,PrecisionTimeandIntervalCompoundeach exist in two message scopes (viaNAME_OVERRIDES).Lambda/LambdaInvocationexpressions andFunctype — full support lands later in the stack./functions_X.yaml) to URNs (extension:io.substrait:functions_X).Tests
All 48 tests pass (
cargo test);cargo fmt --checkclean.