Skip to content

Commit 12f2f2f

Browse files
alambnotashesadriangb
authored andcommitted
[branch-52] fix: disable dynamic filter pushdown for non min/max aggregates (apache#20279) (apache#20877)
- Part of apache#20855 - Closes apache#20267 on branch-52 This PR: - Backports apache#20279 from @notashes to the branch-52 line Co-authored-by: notashes <edgerunnergit@riseup.net> Co-authored-by: Adrian Garcia Badaracco <1755071+adriangb@users.noreply.github.com>
1 parent ab55e11 commit 12f2f2f

2 files changed

Lines changed: 20 additions & 1 deletion

File tree

datafusion/physical-plan/src/aggregates/mod.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -979,7 +979,7 @@ impl AggregateExec {
979979
} else if fun_name.eq_ignore_ascii_case("max") {
980980
DynamicFilterAggregateType::Max
981981
} else {
982-
continue;
982+
return;
983983
};
984984

985985
// 2. arg should be only 1 column reference

datafusion/sqllogictest/test_files/dynamic_filter_pushdown_config.slt

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -257,6 +257,25 @@ physical_plan
257257
04)------RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1
258258
05)--------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/scratch/dynamic_filter_pushdown_config/agg_data.parquet]]}, projection=[score], file_type=parquet, predicate=category@0 = alpha AND DynamicFilter [ empty ], pruning_predicate=category_null_count@2 != row_count@3 AND category_min@0 <= alpha AND alpha <= category_max@1, required_guarantees=[category in (alpha)]
259259

260+
# Test 4b: COUNT + MAX — DynamicFilter should NOT appear here in mixed aggregates
261+
262+
query TT
263+
EXPLAIN SELECT COUNT(*), MAX(score) FROM agg_parquet WHERE category = 'alpha';
264+
----
265+
logical_plan
266+
01)Projection: count(Int64(1)) AS count(*), max(agg_parquet.score)
267+
02)--Aggregate: groupBy=[[]], aggr=[[count(Int64(1)), max(agg_parquet.score)]]
268+
03)----Projection: agg_parquet.score
269+
04)------Filter: agg_parquet.category = Utf8View("alpha")
270+
05)--------TableScan: agg_parquet projection=[category, score], partial_filters=[agg_parquet.category = Utf8View("alpha")]
271+
physical_plan
272+
01)ProjectionExec: expr=[count(Int64(1))@0 as count(*), max(agg_parquet.score)@1 as max(agg_parquet.score)]
273+
02)--AggregateExec: mode=Final, gby=[], aggr=[count(Int64(1)), max(agg_parquet.score)]
274+
03)----CoalescePartitionsExec
275+
04)------AggregateExec: mode=Partial, gby=[], aggr=[count(Int64(1)), max(agg_parquet.score)]
276+
05)--------RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1
277+
06)----------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/scratch/dynamic_filter_pushdown_config/agg_data.parquet]]}, projection=[score], file_type=parquet, predicate=category@0 = alpha, pruning_predicate=category_null_count@2 != row_count@3 AND category_min@0 <= alpha AND alpha <= category_max@1, required_guarantees=[category in (alpha)]
278+
260279
# Disable aggregate dynamic filters only
261280
statement ok
262281
SET datafusion.optimizer.enable_aggregate_dynamic_filter_pushdown = false;

0 commit comments

Comments
 (0)