Skip to content

Commit d91dcb7

Browse files
authored
feat(functions-nested): add array_filter higher-order function (apache#21895)
## Which issue does this PR close? Partially addresses apache#14509 — implements `array_filter` / `list_filter`. ## Rationale for this change `array_transform` (apache#21679) added the first `HigherOrderUDF`. `array_filter` is the natural companion: filter array elements with a boolean lambda, matching Spark `filter` / DuckDB `list_filter` semantics. ## What changes are included in this PR? - New `HigherOrderUDF` `ArrayFilter` (`array_filter` / `list_filter` alias) - Boolean lambda per element; `true` keeps, `false`/null drops (matches Spark semantics) - Handles `List`, `LargeList`, sliced arrays, null sublists - Scalar predicate short-circuit (`x -> true` / `x -> false`) - No-copy fast path when nothing is filtered (skips `arrow::compute::filter`) - Shared HOF helpers extracted from `array_transform` into a common module (`value_lambda_pair`, `coerce_single_list_arg`, `single_list_lambda_parameters`, `extract_list_values`) - Shared unit test helpers for higher-order function tests ## Are these changes tested? - Unit tests: basic filter, multiple sublists, sliced arrays, null sublists, all-filtered-out, nothing-filtered (fast path), scalar true/false predicates - SQL logic tests in `array_filter.slt`: filter variants, `array_filter` + `array_transform` combinations, error cases ## Are there any user-facing changes? Yes — `array_filter(array, lambda)` and alias `list_filter(array, lambda)` are now available as SQL functions.
1 parent 707d861 commit d91dcb7

6 files changed

Lines changed: 937 additions & 119 deletions

File tree

0 commit comments

Comments
 (0)