Fixes 32505: anchor Airflow 3 DAG start_date on pipeline updatedAt, not wall-clock now

[belbed1to9]

Describe your changes:

Fixes #32505

build_dag_configs() runs on every DAG-processor reparse, not just on first DAG creation. On Airflow 3, start_date was set to timezone.utcnow(), so every reparse recomputed it to a new "now". A cron schedule only fires once its interval — measured from start_date — has elapsed; with start_date perpetually reset, the interval never elapses and scheduled ingestion pipelines never fire on their own, even though they deploy and show up in the Airflow UI just fine (see #32505 for the full repro/root-cause writeup).

This was introduced by #24577 ("MINOR: Fix airflow 3 scheduling"). That PR's intent was correct — avoid Airflow 3 immediately firing a catch-up run for a freshly (re)created pipeline, since Airflow 3 fires even with catchup=False when start_date is in the past — but the fix used a start_date that wasn't stable across reparses, which is what actually broke scheduling.

Fix: anchor start_date on the pipeline's own updatedAt timestamp instead of wall-clock now. createOrUpdate is a no-op (bumps neither version nor updatedAt) when a redeploy doesn't actually change the stored pipeline config, so:

• start_date stays stable across routine reparses/redeploys of an unchanged pipeline → the interval elapses → the DAG fires on schedule.
• start_date still resets close to "now" whenever the pipeline is genuinely (re)configured → preserves the original PR's intent of not backfilling stale runs.

Falls back to timezone.utcnow() if updatedAt isn't set (defensive; always present for a persisted pipeline entity).

Verified the anchoring logic directly against Airflow 3.0.3 (airflow.utils.timezone.from_timestamp) — two calls with an unchanged updatedAt return byte-identical start_dates, confirming the reparse-stability property this fix relies on.

Type of change:

• Bug fix

Checklist:

• I have read the CONTRIBUTING document.
• My PR title is Fixes 32505: <short explanation>
• I have commented on my code, particularly in hard-to-understand areas.
• I have added a test that covers the exact scenario we are fixing (test_build_dag_configs_start_date.py): start_date stability across repeated build_dag_configs() calls on an unchanged pipeline, start_date tracking updatedAt, and the no-updatedAt fallback.

🤖 Generated with Claude Code

[github-actions]

❌ PR checklist incomplete

This PR cannot be merged until the following are addressed on its linked issue:

• Linked issue Airflow 3: scheduled ingestion pipelines never fire (start_date recomputed to utcnow() on every DAG reparse) #32505 is not added to the Shipping project.

The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically.

Maintainers can bypass this check by adding the skip-pr-checks label.

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[gitar-bot]

Code Review ✅ Approved

Fixes Airflow 3 scheduled ingestion pipelines never firing by anchoring DAG start_date on the pipeline's updatedAt timestamp instead of wall-clock now. This ensures start_date remains stable across routine reparses—allowing cron intervals to elapse and schedules to fire—while still resetting near

Options

Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Compact

gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source}

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[sonarqubecloud]

Quality Gate failed for 'open-metadata-airflow-apis'

Failed conditions
0.0% Coverage on New Code (required ≥ 20%)

See analysis details on SonarQube Cloud