Commit 88326d4
move TABLESAMPLE SQL surface out of core, into RelationPlanner
Background: PR apache#17843 (merged Dec 9 2025) added a `RelationPlanner`
extension API specifically motivated by TABLESAMPLE, and the blog post
"Extending SQL in DataFusion: from ->> to TABLESAMPLE" canonizes that
as the way to add SQL syntax extensions. Two prior in-core attempts
(apache#16505 Spark-style, apache#16325 random()-rewrite) closed without merging
on semantics-fragmentation grounds. alamb's stated reason for keeping
the surface out of core — sampling semantics differ widely across
DBs, plus WASM bloat — applies directly to the parsing we had baked
into `datafusion/sql/src/relation/mod.rs`.
This commit reshapes the PR around that:
- Revert the inline TABLESAMPLE parsing in `relation/mod.rs`.
- Drop the SQL planning unit tests that exercised the inline parsing.
- Replace the hacky inline `Sample` fallback in
`DefaultPhysicalPlanner` with a public `SamplePhysicalPlanner`
`ExtensionPlanner` that callers register alongside their
`RelationPlanner`. Same effect, cleaner shape.
- Move the e2e fixture from a sqllogictest `.slt` (which can't
register a `RelationPlanner`) to a Rust integration test under
`datafusion/core/tests/parquet/tablesample.rs`. The test uses a
small inline `RelationPlanner` to emit the core `Sample` node,
proving the cube-root pushdown works end to end against parquet.
What stays in core (the actual contribution):
- `Sample` logical extension node and `SampleExec` placeholder.
- `SampleSpec` / `SamplePushdownResult` / `FileSourceSampleResult`
types and the `try_push_sample` trait method on `ExecutionPlan` and
`FileSource`.
- Per-node Passthrough overrides (filter, projection, coalesce,
repartition, non-fetch sort).
- `ParquetSource::try_push_sample` cube-root absorption +
`DataSourceExec::try_push_sample` rebuilding `file_groups`.
- `SamplePushdown` optimizer rule registered in the default pipeline.
These are the building blocks any SQL surface (the existing example
under `datafusion-examples/examples/relation_planner/table_sample.rs`
included) can use to push sampling into the source instead of
filtering per-batch with `random() < p`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent d769eb6 commit 88326d4
6 files changed
Lines changed: 387 additions & 292 deletions
File tree
- datafusion
- core
- src
- tests/parquet
- sqllogictest/test_files
- sql
- src/relation
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
237 | 237 | | |
238 | 238 | | |
239 | 239 | | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
240 | 311 | | |
241 | 312 | | |
242 | 313 | | |
| |||
1810 | 1881 | | |
1811 | 1882 | | |
1812 | 1883 | | |
1813 | | - | |
1814 | | - | |
1815 | | - | |
1816 | | - | |
1817 | | - | |
1818 | | - | |
1819 | | - | |
1820 | | - | |
1821 | | - | |
1822 | | - | |
1823 | | - | |
1824 | | - | |
1825 | | - | |
1826 | | - | |
1827 | | - | |
1828 | | - | |
1829 | | - | |
1830 | | - | |
1831 | | - | |
1832 | | - | |
1833 | | - | |
1834 | | - | |
1835 | | - | |
1836 | | - | |
1837 | | - | |
1838 | | - | |
1839 | | - | |
1840 | | - | |
1841 | | - | |
1842 | 1884 | | |
1843 | 1885 | | |
1844 | 1886 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
| 60 | + | |
60 | 61 | | |
61 | 62 | | |
62 | 63 | | |
| |||
0 commit comments