Fixes #31515: include materialized views in PostgreSQL and Greenplum connectors

[zerafachris]

Describe your changes:

Fixes #31515

The PostgreSQL and Greenplum ingestion connectors do not discover materialized views because two things were missing:

1. RELKIND_MAP in common_pg_mappings.py had no 'm' entry, so the relkind value returned for materialized views had no mapping to TableType.MaterializedView.
2. POSTGRES_GET_TABLE_NAMES and GREENPLUM_GET_TABLE_NAMES only filtered on relkind in ('r', 'p', 'f'), which excludes 'm' (materialized view) entirely.

I added "m": TableType.MaterializedView to RELKIND_MAP and extended both queries to include 'm' in the relkind filter. TableType.MaterializedView already exists in the generated schema (used by Snowflake, ClickHouse, BigQuery, and others), so no schema changes are needed.

Type of change:

• Bug fix

High-level design:

N/A — small change.

The pg_class.relkind column uses 'm' for materialized views per the PostgreSQL catalog docs. The existing comment in both query files already listed m = materialized view but the filter never matched it.

Tests:

Use cases covered

• PostgreSQL connector correctly discovers materialized views in a schema during ingestion.
• Greenplum connector correctly discovers materialized views in a schema during ingestion.

Unit tests

• I added unit tests for the new/changed logic.
• Files added/updated: ingestion/tests/unit/topology/database/test_postgres.py
  • test_relkind_map_includes_materialized_view — asserts RELKIND_MAP['m'] == TableType.MaterializedView
  • test_postgres_get_table_names_includes_materialized_view_relkind — asserts POSTGRES_GET_TABLE_NAMES contains 'm'

Backend integration tests

• Not applicable (no backend API changes).

Ingestion integration tests

• Not applicable — covered by unit tests above; connector-level integration test would require a live Postgres instance with materialized views.

Playwright (UI) tests

• Not applicable (no UI changes).

Manual testing performed

Verified by code inspection:

1. Confirmed TableType.MaterializedView exists in metadata.generated.schema.entity.data.table.
2. Confirmed RELKIND_MAP['m'] resolves to TableType.MaterializedView after change.
3. Confirmed POSTGRES_GET_TABLE_NAMES now includes 'm' in the relkind in (...) filter.
4. Confirmed GREENPLUM_GET_TABLE_NAMES now includes 'm' in the relkind in (...) filter.

UI screen recording / screenshots:

Not applicable.

Checklist:

• I have read the CONTRIBUTING document.
• My PR title is Fixes <issue-number>: <short explanation>
• My PR is linked to a GitHub issue via Fixes #31515 above.
• I have commented on my code, particularly in hard-to-understand areas.
• For JSON Schema changes: I updated the migration scripts or explained why it is not needed.
• For UI changes: I attached a screen recording and/or screenshots above.
• I have added tests (unit / integration / Playwright as applicable) and listed them above.

• I have added a test that covers the exact scenario we are fixing.

AI-generated code — reviewed and validated against OSS contribution guidelines. Submitted by zerafachris.

Greptile Summary

This PR routes PostgreSQL and Greenplum materialized views through the existing view-discovery path and classifies them as MaterializedView.

• Adds shared materialized-view discovery for PostgreSQL-compatible connectors.
• Applies the behavior to PostgreSQL and Greenplum while preserving includeViews semantics.
• Adds unit and PostgreSQL integration coverage for discovery, typing, columns, and fallback behavior.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
ingestion/src/metadata/ingestion/source/database/common_pg_mappings.py	Adds the materialized-view relkind mapping and shared view-listing mixin with graceful fallback.
ingestion/src/metadata/ingestion/source/database/postgres/metadata.py	Enables the shared materialized-view discovery path for PostgreSQL.
ingestion/src/metadata/ingestion/source/database/greenplum/metadata.py	Enables PostgreSQL-compatible materialized-view discovery for Greenplum.
ingestion/tests/integration/postgres/test_metadata.py	Verifies a real PostgreSQL materialized view is ingested with its type and columns.
ingestion/tests/unit/topology/database/test_postgres.py	Covers materialized-view listing, failure fallback, and includeViews routing.
ingestion/tests/unit/topology/database/test_greenplum.py	Covers Greenplum materialized-view listing and fallback when reflection is unsupported.

_{Reviews (8): Last reviewed commit: "Fix ci errors" | Re-trigger Greptile}

Context used:

• Knowledge Base — Metadata ingestion
• Knowledge Base — Ingestion connectors

[github-actions]

❌ PR checklist incomplete

This PR cannot be merged until the following are addressed on its linked issue:

• Issue PostgreSQL and Greenplum connectors do not discover materialized views with SQLAlchemy 2.x #31515 is missing Shipping → Source.
• Issue PostgreSQL and Greenplum connectors do not discover materialized views with SQLAlchemy 2.x #31515 is missing Shipping → Priority.

The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically.

Maintainers can bypass this check by adding the skip-pr-checks label.

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[IceS2]

Hey @zerafachris,

Thanks for working on this.
There is one important issue with the current approach: POSTGRES_GET_TABLE_NAMES and GREENPLUM_GET_TABLE_NAMES are called from query_table_names_and_types(), which only runs when includeTables=true. Views are controlled separately through query_view_names_and_types() and includeViews.

This creates inverted behavior:

• includeTables=false, includeViews=true → the materialized view is still missing.
• includeTables=true, includeViews=false → the materialized view is ingested even though views are disabled.

Could you move materialized-view discovery to the view path instead?

• Restore the PostgreSQL and Greenplum table-discovery queries.
• Override query_view_names_and_types() to combine Inspector.get_view_names() as View with Inspector.get_materialized_view_names() as MaterializedView.

The current tests only check that a mapping or SQL string contains 'm', so they can pass while the configuration behavior is wrong. Please add regression coverage for these two cases:

1. includeTables=false, includeViews=true → materialized view is emitted as MaterializedView.
2. includeTables=true, includeViews=false → materialized view is not emitted.

PostgreSQL already has a testcontainer-based integration suite under ingestion/tests/integration/postgres/, so the first case can be covered against a real materialized view.
For Greenplum, a focused behavior test around query_view_names_and_types() would still be valuable.

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[zerafachris]

Thanks for the detailed review @IceS2! You're absolutely right — the table-path approach created inverted behaviour.

I've reworked the fix across 5 commits on this branch:

Core change (moved to view path):

• Reverted POSTGRES_GET_TABLE_NAMES and GREENPLUM_GET_TABLE_NAMES back to ('r', 'p', 'f') — materialized views no longer leak through the table query.
• Added query_view_names_and_types() to both PostgresSource and GreenplumSource: combines inspector.get_view_names() → View + inspector.get_materialized_view_names() → MaterializedView. Both are only called when includeViews=True.

Tests added:

1. test_query_view_names_and_types_includes_materialized_views — calls the method directly with mocked inspector; asserts regular view gets TableType.View and matview gets TableType.MaterializedView.
2. test_matview_not_emitted_when_include_views_false — asserts POSTGRES_GET_TABLE_NAMES does not contain 'm', so includeTables=True, includeViews=False cannot emit a matview through the table path.

The existing test_relkind_map_includes_materialized_view is kept since the 'm' entry in RELKIND_MAP is still correct (provides consistent type resolution).

Prepared with AI assistance (Claude Code, Anthropic), reviewed for correctness before submission.

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[zerafachris]

Good catch — addressed in the latest two commits. Both PostgresSource.query_view_names_and_types and GreenplumSource.query_view_names_and_types now fetch the regular view list first (unconditionally), then attempt get_materialized_view_names() inside an except Exception guard that logs a warning and falls back to an empty list. A failure in the matview API can no longer suppress regular view discovery for the schema.

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[zerafachris]

Thanks @IceS2 — great catch on the includeTables/includeViews inversion. Address pushed:

What changed:

• Restored POSTGRES_GET_TABLE_NAMES and GREENPLUM_GET_TABLE_NAMES to relkind in ('r', 'p', 'f') — materialized views removed from the table path.
• Added query_view_names_and_types() override in both PostgresSource and GreenplumSource: Inspector.get_view_names() → TableType.View, Inspector.get_materialized_view_names() → TableType.MaterializedView, with a try/except guard for Greenplum versions that may not support the latter.
• Replaced the SQL-string assertion tests with behavioral tests:
  • test_query_view_names_and_types_returns_materialized_views — mocks the inspector, verifies both View and MaterializedView are returned.
  • test_query_table_names_excludes_materialized_views — verifies 'm' is absent from POSTGRES_GET_TABLE_NAMES.

Materialized views are now only discovered when includeViews=True, matching the expected configuration semantics.

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[zerafachris]

Applied make py_format (Black) to resolve the py-checkstyle CI failure — no logic changes, formatting only.

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[github-actions]

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[zerafachris]

Hi @IceS2 — thanks for the detailed review. All three points addressed in the rebased commit:

Inverted config behavior fixed: Materialized views are now discovered in query_view_names_and_types() (the view path, controlled by includeViews) via a new PgMatviewMixin in common_pg_mappings.py. The POSTGRES_GET_TABLE_NAMES and GREENPLUM_GET_TABLE_NAMES queries are restored to tables-only filters (relkind ≠ 'm'), so includeTables no longer governs materialized views.

Behavioral regression tests added:

• includeViews=True → materialized view emitted as TableType.MaterializedView
• includeViews=False → view path not called, materialized view absent
• Materialized view absent from the table path regardless of includeTables
  (for both Postgres and Greenplum)

Integration test: test_metadata.py creates a real materialized view and asserts it appears as MaterializedView type in the ingested entity list.

PgMatviewMixin guards against inspectors that lack get_materialized_view_names() (logs a warning and returns an empty list), so Greenplum falls back gracefully if the dialect doesn't expose the method.

Branch rebased cleanly onto current main (squashed to one commit to eliminate the noisy merge-into-branch history).

Prepared with AI assistance (Claude Code, Anthropic), reviewed for correctness before submission.

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[github-actions]

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[gitar-bot]

Code Review ✅ Approved 1 resolved / 1 findings

Adds materialized view discovery to the PostgreSQL and Greenplum connectors by mapping relkind 'm' and updating view-path queries, addressing the matview lookup failure issue. No issues found.

✅ 1 resolved

✅ Edge Case: Matview lookup failure drops all views for the schema

📄 ingestion/src/metadata/ingestion/source/database/greenplum/metadata.py:129-135 📄 ingestion/src/metadata/ingestion/source/database/postgres/metadata.py:175-181
The override builds one combined list where the regular-view comprehension and the get_materialized_view_names comprehension are concatenated and evaluated eagerly. If get_materialized_view_names(schema_name) raises (e.g., on an older Greenplum server that lacks matview support, or on a permission error), the whole expression throws, the upstream try/except in get_tables_name_and_type sets view_iter = [], and regular views for that schema are silently dropped too — a regression versus the base implementation which never called the matview API. Fetch and guard the two lists separately so a matview failure cannot suppress regular views.

Options

Display: compact → Counting what did not apply, without listing it.

Comment with these commands to change the behavior for this request:

Compact

gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source}

[sonarqubecloud]

Quality Gate passed for 'open-metadata-ingestion'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
52.9% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[github-actions]

🚦 Removed from the merge queue — failed_checks (2026-09-09T15:00:56Z)

Blocked the queue: playwright-summary

• Postgresql PR Playwright E2E Tests — playwright-summary, playwright / playwright-ci (ingestion-01)