ci: drop static AWS keys from athena profile (CORE-687)

[haritamar]

Summary

Pairs with the elementary-cli + elementary-internal OIDC migration. Drops the static aws_access_key_id / aws_secret_access_key rendered into the dbt-athena profile (sourced from CI_WAREHOUSE_SECRETS) and pins the profile to the dedicated oss_tests Athena workgroup. boto3 picks up role credentials from the env vars exported by aws-actions/configure-aws-credentials in whichever workflow uses this profile.

Linked

• Linear: https://linear.app/elementary/issue/CORE-687
• Trust policy + workgroup: https://github.com/elementary-data/elementary-internal/pull/5973
• Consumer workflows: ci: switch test-warehouse AWS auth to OIDC role (CORE-687) elementary#2215

Caveat

This integration_tests profile template is also referenced by dbt-data-reliability's own CI workflows (e.g. test-warehouse.yml, cleanup-stale-schemas.yml). Those workflows still authenticate via the static keys. Once this lands, they'll break for the athena target until they're updated to assume the OIDC role too. Either:

1. Land this PR last (after the consumer-side changes), or
2. Follow up with workflow changes in this repo that mirror what elementary-cli's test-warehouse.yml and cleanup-stale-schemas.yml do (add permissions: id-token: write + aws-actions/configure-aws-credentials@v4), and expand the job_workflow_ref allow-list in iac_global/github_actions_oidc.tf to include this repo's workflows + the sub claim to allow repo:elementary-data/dbt-data-reliability:*.

Test plan

• Land the elementary-internal IAM PR first.
• Coordinate with the dbt-data-reliability CI follow-up before merging this.

Summary by CodeRabbit

Release Notes

• Chores
  • Enhanced CI/CD security by implementing OIDC token minting across build and test workflows, enabling automatic and secure AWS credential provisioning without explicit key storage
  • Refactored integration test suite to leverage OIDC-based authentication for AWS interactions, improving security posture and reducing credential exposure

[linear]

CORE-687 Switch elementary-cli AWS auth from access keys to OIDC + IAM role

[coderabbitai]

📝 Walkthrough

Walkthrough

GitHub workflows are configured to enable OIDC token minting and AWS credential assumption via IAM roles in eu-west-1 for Athena-specific matrix entries. The Athena profile authentication switches from explicit AWS credentials to work group-based configuration.

Changes

Cohort / File(s)	Summary
GitHub Workflows - OIDC Configuration .github/workflows/cleanup-stale-schemas.yml, .github/workflows/test-all-warehouses.yml, .github/workflows/test-warehouse.yml	Added permissions.id-token: write to enable OIDC token minting. Athena matrix entries now conditionally configure AWS credentials via aws-actions/configure-aws-credentials@v4, assuming secrets.AWS_OIDC_ROLE_ARN in eu-west-1.
Integration Test Profile integration_tests/profiles/profiles.yml.j2	Athena profile authentication replaced explicit AWS access/secret keys with fixed work_group: oss_tests configuration.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 Token minting hops with grace,
OIDC takes credentials' place,
AWS roles in regions far,
No more keys beneath the bar,
Security springs forth so bright! ✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely summarizes the main change: dropping static AWS keys from the Athena profile in CI configuration, with the issue reference for context.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch core-687-aws-oidc-auth

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

👋 @haritamar
Thank you for raising your pull request.
Please make sure to add tests and document all user-facing changes.
You can do this by editing the docs files in the elementary repository.

[coderabbitai]

🧹 Nitpick comments (1)

integration_tests/profiles/profiles.yml.j2 (1)

180-180: Keep Athena work_group and region_name tied to one source of truth.

Line 180 is pinned to oss_tests, while region remains separately sourced (Line 177). Since workflows currently pin AWS region to eu-west-1 (for example, .github/workflows/test-warehouse.yml Line 95 and .github/workflows/cleanup-stale-schemas.yml Line 54), this can drift later and cause Athena “workgroup not found in region” failures.

Suggested refactor

-      region_name: {{ athena_region | toyaml }}
+      region_name: eu-west-1
       database: awsdatacatalog
       schema: {{ schema }}
       work_group: oss_tests

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@integration_tests/profiles/profiles.yml.j2` at line 180, The work_group value
is hardcoded to "oss_tests" while region_name is sourced elsewhere, which can
drift and cause "workgroup not found in region" errors; change work_group to use
the same templated variable used for region_name (e.g., replace the literal
oss_tests with the shared Athena/workgroup variable like {{ athena_work_group }}
or reference the same {{ region_name }}-tied source), and ensure the workflows
(.github/workflows/test-warehouse.yml and cleanup-stale-schemas.yml) set that
same variable so both region_name and work_group come from a single source of
truth.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@integration_tests/profiles/profiles.yml.j2`:
- Line 180: The work_group value is hardcoded to "oss_tests" while region_name
is sourced elsewhere, which can drift and cause "workgroup not found in region"
errors; change work_group to use the same templated variable used for
region_name (e.g., replace the literal oss_tests with the shared
Athena/workgroup variable like {{ athena_work_group }} or reference the same {{
region_name }}-tied source), and ensure the workflows
(.github/workflows/test-warehouse.yml and cleanup-stale-schemas.yml) set that
same variable so both region_name and work_group come from a single source of
truth.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 0bf23461-ea16-41aa-808f-8a337416ac47

📥 Commits

Reviewing files that changed from the base of the PR and between 9ad6411 and 85c1467.

📒 Files selected for processing (4)

• .github/workflows/cleanup-stale-schemas.yml
• .github/workflows/test-all-warehouses.yml
• .github/workflows/test-warehouse.yml
• integration_tests/profiles/profiles.yml.j2