Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion .ai/skills/ffi-capsule-protocol/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,12 @@ one of them, and the failure is a bare `TypeError` from a `call1`. So:
`python/datafusion/user_defined.py`, where the `Protocol` type hints for
these methods live.

Changing what a codec puts *on the wire* is equally breaking, and easier to
miss because no signature moves and nothing fails to compile. Serialized plans
outlive the process that wrote them, so the same checklist applies: upgrade
guide, `api change` label, and a statement of exactly which sessions produce
different bytes.

## Rule 6 — a session keeps one `Arc<SessionContext>` for life

`FFI_TaskContextProvider` holds its provider **weakly**, and every codec handed
Expand Down Expand Up @@ -181,8 +187,11 @@ pins that; changing it should be deliberate.

- `docs/source/contributor-guide/ffi.md` — the protocol, the fork caveat.
- `docs/source/user-guide/upgrade-guides.md` — every past migration.
- `crates/core/src/codec.rs` — the codec chain: the envelope, identity dispatch,
and the two unframed cases from Rule 8.
- `examples/datafusion-ffi-example/src/` — provider, catalog, function, codec
getters, all in current form.
getters, all in current form. `name_only_codec.rs` is the codec that encodes
nothing.
- `examples/datafusion-ffi-query-planner-example/src/planner.rs` — planner
getter.
- `examples/datafusion-ffi-query-planner-example/python/tests/_test_three_library_query_planner.py`
Expand Down
34 changes: 34 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,40 @@ pre-commit run --all-files

Fix any failures before committing.

## Test Coverage

Always prefer Python coverage — a doctest example in a docstring, or a pytest
case. The user-facing Python surface is the first line of defense and the
primary focus, so behavior should be pinned where users actually meet it.

**CI does not run Rust tests.** No workflow invokes `cargo test`; the only
Rust checks are `cargo fmt --check` and
`cargo clippy --no-deps --all-targets`. `--all-targets` compiles
`#[cfg(test)]` code, so a Rust test cannot rot into a non-compiling state, but
it is never executed and a behavioral regression will not fail the build. A
Rust test added today is dead weight.

Adding a `cargo test` job is not a one-line change: `crates/core/Cargo.toml`
enables `pyo3/extension-module` unconditionally, so the test binary fails to
link against `Py_*` symbols on Linux. The feature would have to be gated first.

Write a Rust test only when the behavior is genuinely unreachable from Python,
and wire up CI in the same change so it actually runs. Before concluding it is
unreachable, check the suites that already exist:

- `python/tests/` — the main suite. Run `pytest python/`, **not**
`pytest python/tests/`: `--doctest-modules` is on by default and the
narrower path skips the doctests in `python/datafusion/`.
- `examples/datafusion-ffi-example/python/tests/` and
`examples/datafusion-ffi-query-planner-example/python/tests/` — integration
coverage across a real FFI boundary, for anything involving extension
codecs, table providers, query planners, or capsule export. These need the
example crates built (`maturin build`, then install the wheel).
- `examples/tpch/` — end-to-end query coverage.

Prefer asserting observable behavior over internal accessors. A test that
checks a getter can pass while the path a user actually takes is broken.

## Python Function Docstrings

Every Python function must include a docstring with usage examples.
Expand Down
Loading