-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[enhance](runtime filter) impl partition pruning in runtime filer #47025
base: master
Are you sure you want to change the base?
[enhance](runtime filter) impl partition pruning in runtime filer #47025
Conversation
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
run buildall |
TPC-H: Total hot run time: 32093 ms
|
TeamCity be ut coverage result: |
TPC-DS: Total hot run time: 195472 ms
|
ClickBench: Total hot run time: 30.52 s
|
run buildall |
TPC-H: Total hot run time: 32424 ms
|
TPC-DS: Total hot run time: 188758 ms
|
ClickBench: Total hot run time: 30.84 s
|
TeamCity be ut coverage result: |
run buildall |
TPC-H: Total hot run time: 32468 ms
|
TPC-DS: Total hot run time: 194362 ms
|
ClickBench: Total hot run time: 30.88 s
|
TeamCity be ut coverage result: |
run buildall |
TPC-H: Total hot run time: 32077 ms
|
TPC-DS: Total hot run time: 193645 ms
|
TeamCity be ut coverage result: |
ClickBench: Total hot run time: 31.03 s
|
run buildall |
TPC-H: Total hot run time: 32312 ms
|
TPC-DS: Total hot run time: 184063 ms
|
ClickBench: Total hot run time: 29.99 s
|
TeamCity be ut coverage result: |
ClickBench: Total hot run time: 31.22 s
|
1e8e7af
to
e19475c
Compare
run buildall |
TeamCity cloud ut coverage result: |
TeamCity be ut coverage result: |
run buildall |
TeamCity cloud ut coverage result: |
TPC-H: Total hot run time: 31560 ms
|
This PR implements partition pruning through runtime filters. When executing a SQL query like:
During execution, the backend (BE) will receive a dynamic runtime filter condition
partition_col = xxx
. Since partition_col is a partitioning column, we can use its value to determine if the partition can be pruned.Additionally, this mechanism also supports filtering queries like:
If func cannot be evaluated at the frontend (FE), the frontend will not perform partition pruning. However, since the backend can compute func, this mechanism allows us to handle pruning scenarios that are not possible at the frontend, providing a more efficient pruning process on the backend side.
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)