Gate the anti-edge chain count rewrite on native storage - #943
Gate the anti-edge chain count rewrite on native storage#943itz-puneet wants to merge 3 commits into
Conversation
tryRewriteExtendChainCount requires both rel groups and node tables to be native (empty storage path and StorageFormat::NONE) before it lets the count fast path iterate the committed node-group grid and read the CSR. tryRewriteAntiEdgeChainCount checked only getScanFunction(), so an icebug-disk graph could take the same CSR-grid arithmetic against tables that have neither structure. Such a graph is constructible: bind_ddl.cpp rejects mixing storage formats but allows an all-icebug-disk graph, and the demo dataset is exactly that shape, with a follows rel from user to user. Hoist the two isNative*Entry predicates out of tryRewriteExtendChainCount into file-scope helpers and apply them in the anti-edge path to the anti rel group, the prefix-chain rel groups, and the node entry. n0, n1 and n2 all bind midTableID, so one node check covers all three. The rewrite is only narrowed, so the fallback is the regular plan. Addresses item 3 of LadybugDB#939.
adsharma
left a comment
There was a problem hiding this comment.
- Needs a regression test
- Suffix-hop node tables beyond n2 aren't explicitly gated — only their rel groups and the mid node are. Today this relies on the binder invariant
(native rel ⇒ native endpoints; bind_ddl.cpp:435-440), whereas the extend-chain path gates every node explicitly. Note there's no equivalent mixing
guard for Arrow tables in bind_ddl.cpp, so this relies on Arrow node tables never being rel endpoints. Consider gating fromNode/toNode entries in the
suffix-hop loop for full parity and defense-in-depth.
Gate the from/to node entries in the suffix-hop loop too, so every node the anti-edge arithmetic touches is checked rather than only the mid node. The binder refuses to mix icebug-disk tables with native ones, but there is no equivalent guard for Arrow-backed tables, so the rel group's format cannot vouch for its endpoints. This brings the path to parity with the extend-chain rewrite. Add two optimizer tests over the same query. The icebug-disk case asserts COUNT_ANTI_EDGE_CHAIN is absent; the native case asserts it is present, so the skip case cannot pass merely because the query stopped matching the rewrite.
|
Both addressed in 96d7485. Suffix-hop endpoints — good catch, and the Arrow point is the part that convinced me: Regression test — two cases in I cannot build locally, so CI is the check on both. |
The first attempt asserted the rewrite fired on a three-extend query, and CI reported RewritesNativeTables failing while SkipsIcebugDiskTables passed. That is the pairing doing its job: the skip assertion was passing vacuously because the query never matched the rewrite in the first place. The rewrite targets the LSQB q9 shape and requires at least four extends: two undirected hops either side of the middle node, an anti-edge between the outer two written as NOT EXISTS, and a directed suffix hop into another table. The query now mirrors q9 from test_files/lsqb/lsqb_queries.test, using the user/follows/city/livesin schema that the demo-db icebug-disk dataset already provides, so the same query runs against both storage formats. Build the DDL by concatenation rather than std::vformat, which needs a non-const lvalue for its format arguments.
|
@adsharma the failing check is deliberate, and I could use a pointer. I wrote the regression test as a pair on one query: That is what is happening. What I have tried for the positive case, both on an empty-DB fixture with a small hand-built graph:
I confirmed the rewrite is not behind a flag — Is this cardinality-dependent in practice, i.e. does it only show up on something like The source changes themselves are unaffected; only this one test is failing. |
Addresses item 3 of #939. Scoped per @adsharma's note on that issue: focus on the native gate, verify icebug-disk manually for now.
Problem
tryRewriteExtendChainCountrequires every participating table to be native before letting the count fast path run —getStorage().empty() && getStorageFormat() == StorageFormat::NONE, applied to rel groups atcount_rel_table_optimizer.cpp:598and to both node entries at:607. The comment there spells out why: the fast path iterates the committed node-group grid of native node tables and reads the CSR of native rel tables, and Arrow-backed and icebug-disk tables have neither.tryRewriteAntiEdgeChainCountchecks onlygetScanFunction().has_value(), at:237for the anti rel group and:408for the prefix-chain extends, and never checks the node entry at all. So the same CSR-grid arithmetic could run against tables that have no such grid.This is reachable rather than theoretical.
bind_ddl.cpp:437rejects mixing storage formats but explicitly permits an all-icebug-disk graph, anddataset/demo-db/icebug-disk/schema.cypheris exactly that shape —usernodes plusfollows(FROM user TO user), the self-referencing pattern this rewrite matches.Change
Hoist the two
isNative*Entrypredicates out oftryRewriteExtendChainCountinto file-scope helpers, following the existing forward-declare-then-define convention in that file, and apply them in the anti-edge path to:n0,n1andn2all bindmidTableID, so one check covers every node the operator's arithmetic touches.This only narrows when the rewrite fires. Anything it now declines falls back to the regular plan, so the result stays correct either way.
Verification
The optimizer translation unit compiles clean under
-Wall -Wextra, matching the unmodified baseline, andclang-format-18reports no diff. No behavioral test is included, per the scoping above.Items 1 and 2 of #939 — node visibility inside the operators, and clipping
getOffsetUpperBound()to the reader snapshot — are untouched and remain open.