Fixes #32787: Generate Doris-compatible datetime filters

[XXUCHAN]

Describe your changes:

Fixes #32787

This PR fixes Doris Sample Data and Profiler failures caused by generating an unsupported CAST(... AS TIMESTAMP) expression for time-based partition filters.

Doris does not support TIMESTAMP as a cast target and uses DATETIME instead. This change adds dedicated Doris SQLAlchemy compiler handlers for both DatetimeAddFn and TimestampAddFn, generating:

CAST(CURRENT_TIMESTAMP - interval '1' DAY AS DATETIME)

The existing MySQL compiler behavior remains unchanged.

Type of change:

• Bug fix

High-level design:

The Doris SQLAlchemy dialect previously fell back to the generic datetime compiler, which generates AS TIMESTAMP.

This change:

• Registers dedicated pydoris compiler handlers for DatetimeAddFn and TimestampAddFn.
• Adds a Doris-specific doris_function that generates AS DATETIME.
• Keeps the Doris implementation separate from the MySQL implementation so future dialect-specific changes remain isolated.
• Does not change schemas, configuration, migrations, or other database dialects.

Tests:

Use cases covered

• A Doris DATETIME partition column generates a lookback boundary using AS DATETIME.
• A Doris TIMESTAMP SQLAlchemy column representation also generates AS DATETIME.
• Existing generic date and datetime dispatch behavior remains unchanged.

Unit tests

• I added unit tests for the changed logic.
• File updated:
  • ingestion/tests/unit/source/database/doris/test_connection.py
• The parameterized regression test compiles both DATETIME and TIMESTAMP partition boundaries using the actual DorisDialect.
• Result: 4 relevant tests passed.

Command executed:

PYTHONPATH=ingestion/src pytest -q \
  ingestion/tests/unit/topology/test_sqa_utils.py \
  ingestion/tests/unit/source/database/doris/test_connection.py \
  -k 'test_doris_time_partition_filter_uses_datetime_cast or test_dispatch_to_date_or_datetime'

Backend integration tests

• Not applicable — no backend API changes.

Ingestion integration tests

• I added/updated ingestion integration tests for connector changes.

A live Doris integration test was not added. The regression is covered at the SQLAlchemy compilation boundary using the actual pydoris dialect. No live Doris environment was available for this change.

Playwright (UI) tests

• Not applicable — no UI changes.

Manual testing performed

1. Confirmed that the regression test generated AS TIMESTAMP and failed before the fix.
2. Confirmed that both Doris DATETIME and TIMESTAMP paths generate AS DATETIME after the fix.
3. Verified DATE, DATETIME, and TIMESTAMP partition expressions for the supported YEAR, MONTH, DAY, and HOUR interval units.
4. Ran the relevant unit tests successfully.
5. Ran Ruff lint and formatting checks successfully.
6. Ran all configured pre-commit hooks successfully.

UI screen recording / screenshots:

Not applicable.

Checklist:

• I have read the CONTRIBUTING document.
• My PR title is Fixes #32787: Generate Doris-compatible datetime filters.
• My PR is linked to a GitHub issue via Fixes #32787 above.
• I have commented on my code, particularly in hard-to-understand areas.
• For JSON Schema changes: not applicable — no schema or migration changes.
• For UI changes: not applicable — no UI changes.
• I have added tests and listed them above.

Bug fix checklist

• I have added a test that covers the exact scenario we are fixing.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[github-actions]

❌ PR checklist incomplete

This PR cannot be merged until the following are addressed on its linked issue:

• Linked issue Doris sampler generates unsupported CAST(... AS TIMESTAMP) for datetime partition filters #32787 is not added to the Shipping project.

The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically.

Maintainers can bypass this check by adding the skip-pr-checks label.

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[gitar-bot]

Code Review ✅ Approved

Fixes Doris Sample Data and Profiler failures by adding dedicated Doris SQLAlchemy compiler handlers for DatetimeAddFn and TimestampAddFn that generate AS DATETIME instead of the unsupported AS TIMESTAMP. No issues found.

Options

Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Compact

gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source}