Skip to content

[BugFix] Preserve TopN projection when eliminating sort columns with equality predicates - #78021

Merged
stephen-shelby merged 1 commit into
StarRocks:branch-3.5from
Smith-Cruise:fix-eliminate-sort-drop-topn-projection-3.5
Aug 19, 2026
Merged

[BugFix] Preserve TopN projection when eliminating sort columns with equality predicates#78021
stephen-shelby merged 1 commit into
StarRocks:branch-3.5from
Smith-Cruise:fix-eliminate-sort-drop-topn-projection-3.5

Conversation

@Smith-Cruise

@Smith-Cruise Smith-Cruise commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Why I'm doing:

On branch-3.5 , a query combining a decimal128/largeint divide in the select list with ORDER BY <column pinned by an equality predicate> LIMIT N fails at planning time:

Getting analyzing error. Detail message: Cannot convert ColumnRefOperator to Expr,
please check the input expression: 14: expr.

Minimal repro:

CREATE TABLE t (k1 varchar(60) NOT NULL, v1 decimal(28, 8), v2 decimal(28, 8))
DUPLICATE KEY(k1) DISTRIBUTED BY HASH(k1) BUCKETS 4;

SELECT k1, v1, v2, v1 / v2 FROM t WHERE k1 = '2026-06' ORDER BY k1 LIMIT 10;

Root cause — an interaction of two rules:

  1. HoistHeavyCostExprsUponTopnRule ([Enhancement] Hoist heavy-cost(decimal divide) upon top-n #55417) hoists the heavy divide into a project above the TopN so it is only evaluated for the top N rows; MergeProjectWithChildRule later merges that project into the TopN operator as its projection.
  2. EliminateSortColumnWithEqualityPredicateRule ([Enhancement] Eliminate sort column when all predicates of the column are equivalent predicates #54177) then eliminates the TopN because every ORDER BY column is pinned by an equality predicate — but it drops the TopN's projection on the floor. The hoisted divide loses its only definition, and plan translation fails when it finds the column mapped to itself with no child producing it.

main is not affected: #58345 added the "merge the TopN's projection into the scan's projection" logic to this rule. That PR was backported to branch-3.5-cc (#70879) but never to branch-3.5, and the later #75581 backport of #74983 only carried the global-limit fix (the projection-merge lines were context in that diff, not part of it).

Workaround for affected versions: SET_VAR(cbo_disabled_rules='TF_HOIST_HEAVY_COST_UPON_TOPN').

What I'm doing:

  • Partial cherry-pick of the EliminateSortColumnWithEqualityPredicateRule hunk from [Enhancement] support defer project after top n  #58345 (without the DeferProjectAfterTopN feature itself): before eliminating the sort, merge the TopN's projection into the scan's projection, only adding entries the scan does not already define.
  • Add a regression test that reproduces the customer query shape; it fails with the error above before the fix and passes after. Upstream has no coverage for this path (the [Enhancement] support defer project after top n  #58345 tests only cover DeferProjectAfterTopN scenarios), so the test is new.

Fixes #issue

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 3.5
    • 3.4
    • 3.3

🤖 Generated with Claude Code

…equality predicates

EliminateSortColumnWithEqualityPredicateRule drops the projection carried
by the TopN operator when it eliminates the sort because every ORDER BY
column is pinned by an equality predicate. When
HoistHeavyCostExprsUponTopnRule has hoisted a heavy decimal128/largeint
divide above the TopN (later merged into the TopN operator as its
projection), the hoisted expression loses its definition and plan
translation fails with:

    Cannot convert ColumnRefOperator to Expr, please check the input
    expression: N: expr

Merge the TopN's projection into the scan's projection before eliminating
the sort. This is a partial cherry-pick of StarRocks#58345, which fixed this spot
on main but was never backported to branch-3.5.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Smith Cruise <chendingchao1@126.com>
@stephen-shelby
stephen-shelby merged commit ee65a7e into StarRocks:branch-3.5 Aug 19, 2026
45 of 48 checks passed
@Smith-Cruise
Smith-Cruise deleted the fix-eliminate-sort-drop-topn-projection-3.5 branch August 19, 2026 09:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants