Skip to content

PHOENIX-7879 Tests for EXPLAIN text and ExplainPlanAttributes serialization compatibility - #2495

Merged
apurtell merged 5 commits into
apache:masterfrom
apurtell:PHOENIX-7879
Jun 4, 2026
Merged

PHOENIX-7879 Tests for EXPLAIN text and ExplainPlanAttributes serialization compatibility#2495
apurtell merged 5 commits into
apache:masterfrom
apurtell:PHOENIX-7879

Conversation

@apurtell

@apurtell apurtell commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Add a backward compatibility test harness that freezes Phoenix's EXPLAIN output and makes any future change to the grammar an explicit, reviewable act.

ExplainPlanAttributes gains a @JsonPropertyOrder annotation, and RegionLocationsListSerializer and ServerMergeColumnsSerializer handle the two attributes that aren't cleanly serializable.

New test package org.apache.phoenix.query.explain implements compatibility checks. ExplainOracle compiles each query against a connectionless Phoenix driver and compares both the textual and JSON representations of the plan against an expected baseline, producing line-by-line and JSON-pointer diffs on mismatch. Normalizers strip environment-specific noise before comparison. ExplainTextNormalizer collapses parallelism/chunk counts to , removes row/byte stats and region-location lines, and ExplainJsonNormalizer nulls out region locations, lookup counts, split chunks, and row/size estimates and recurses into nested join plans.

ExplainChangeRule supports future PRs that intentionally change the grammar. New rules can be appended that transform the baseline into its new expected shape. Every change is intentional and reviewable. The diff in the rule itself documents exactly what changed.

The golden text in ExplainOracleTest should be periodically regenerated with the ExplainChangeRules dropped afterward. So, ExplainChangeRule is clearly optional. The golden text can simply be updated. However this could be a good practice.

Generated-by: Claude Opus 4.8[1m] noreply@anthropic.com

…zation compatibility

Co-authored-by: Claude <noreply@anthropic.com>
@apurtell

apurtell commented Jun 4, 2026

Copy link
Copy Markdown
Contributor Author

Force pushed a commit message addendum adding Claude as co-author.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a backward-compatibility “golden” test harness to freeze Phoenix EXPLAIN output (both plan-step text and ExplainPlanAttributes JSON) so future grammar changes become explicit and reviewable, while also stabilizing JSON serialization for previously problematic attributes.

Changes:

  • Introduce ExplainOracle + ExplainChangeRule infrastructure to compare normalized current EXPLAIN output against embedded baselines and produce readable diffs on mismatch.
  • Add text/JSON normalizers to strip environment-specific noise (parallelism counts, row/byte estimates, region locations, lookup counts, etc.).
  • Stabilize ExplainPlanAttributes JSON output ordering and add custom Jackson serializers for serverMergeColumns and regionLocations.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
phoenix-core/src/test/java/org/apache/phoenix/query/explain/ExplainTextNormalizer.java Normalizes EXPLAIN text by eliding environment-specific details.
phoenix-core/src/test/java/org/apache/phoenix/query/explain/ExplainJsonNormalizer.java Normalizes EXPLAIN JSON attributes by nulling/zeroing cluster-dependent fields and recursing into nested plans.
phoenix-core/src/test/java/org/apache/phoenix/query/explain/ExplainChangeRule.java Provides rule hooks to transform “golden” expected text/JSON for intentional grammar changes.
phoenix-core/src/test/java/org/apache/phoenix/query/explain/ExplainOracle.java Orchestrates normalization, rule application, and mismatch diff generation for EXPLAIN comparisons.
phoenix-core/src/test/java/org/apache/phoenix/query/explain/ExplainOracleTest.java Corpus of compatibility tests plus serializer/order/normalizer sanity checks.
phoenix-core-client/src/main/java/org/apache/phoenix/compile/ServerMergeColumnsSerializer.java Adds Jackson serialization for Set<PColumn> in plan attributes.
phoenix-core-client/src/main/java/org/apache/phoenix/compile/RegionLocationsListSerializer.java Adds Jackson serialization for List<HRegionLocation> in plan attributes.
phoenix-core-client/src/main/java/org/apache/phoenix/compile/ExplainPlanAttributes.java Freezes JSON property order and wires in the two custom serializers.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@apurtell
apurtell merged commit 29fbbbc into apache:master Jun 4, 2026
@apurtell
apurtell deleted the PHOENIX-7879 branch June 4, 2026 23:55
asf-gitbox-commits pushed a commit that referenced this pull request Jun 4, 2026
@virajjasani

Copy link
Copy Markdown
Contributor

@apurtell @palashc did we have clean build results before merging the PR?

The state of the build does not look good on master https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-mulitbranch/job/master/
Besides this is blocking other PR builds too. I will have to revert the PR.

@virajjasani

Copy link
Copy Markdown
Contributor

PR builds are available with format https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-${PR_number}

e.g. https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-2495 for this PR, since it is merged, it is no longer available to view.

asf-gitbox-commits pushed a commit that referenced this pull request Jun 8, 2026
… serialization compatibility (#2495)"

This reverts commit 29fbbbc.
asf-gitbox-commits pushed a commit that referenced this pull request Jun 8, 2026
… serialization compatibility (#2495)"

This reverts commit cec6a64.
apurtell added a commit to apurtell/phoenix that referenced this pull request Jun 8, 2026
apurtell added a commit to apurtell/phoenix that referenced this pull request Jun 8, 2026
asf-gitbox-commits pushed a commit that referenced this pull request Jun 10, 2026
asf-gitbox-commits pushed a commit that referenced this pull request Jun 10, 2026
…zation compatibility (#2495)

Co-authored-by: Claude Opus 4.8[1m] <noreply@anthropic.com>
apurtell added a commit to apurtell/phoenix that referenced this pull request Jun 17, 2026
…zation compatibility (apache#2495)

Co-authored-by: Claude Opus 4.8[1m] <noreply@anthropic.com>
tkhurana pushed a commit that referenced this pull request Jul 2, 2026
… serialization compatibility (#2495)"

This reverts commit 29fbbbc.
asf-gitbox-commits pushed a commit that referenced this pull request Jul 24, 2026
…zation compatibility (#2495)

Co-authored-by: Claude Opus 4.8[1m] <noreply@anthropic.com>
apurtell added a commit to apurtell/phoenix that referenced this pull request Aug 4, 2026
…zation compatibility (apache#2495)

Co-authored-by: Claude Opus 4.8[1m] <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants