Skip to content

ELI-545 | refresh tableau data programmatically#618

Merged
Karthikeyannhs merged 9 commits intomainfrom
elid-545/programatically_updating_tableau_daskboard
Mar 27, 2026
Merged

ELI-545 | refresh tableau data programmatically#618
Karthikeyannhs merged 9 commits intomainfrom
elid-545/programatically_updating_tableau_daskboard

Conversation

@Karthikeyannhs
Copy link
Copy Markdown
Contributor

Description

Context

Type of changes

  • Refactoring (non-breaking change)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would change existing functionality)
  • Bug fix (non-breaking change which fixes an issue)

Checklist

  • I am familiar with the contributing guidelines
  • I have followed the code style of the project
  • I have added tests to cover my changes
  • I have updated the documentation accordingly
  • This PR is a result of pair or mob programming

Sensitive Information Declaration

To ensure the utmost confidentiality and protect your and others privacy, we kindly ask you to NOT including PII (Personal Identifiable Information) / PID (Personal Identifiable Data) or any other sensitive data in this PR (Pull Request) and the codebase changes. We will remove any PR that do contain any sensitive information. We really appreciate your cooperation in this matter.

  • I confirm that neither PII/PID nor sensitive data are included in this PR and the codebase changes.

@Karthikeyannhs Karthikeyannhs force-pushed the elid-545/programatically_updating_tableau_daskboard branch from a8f42a1 to c2f0a19 Compare March 27, 2026 00:18
@Karthikeyannhs Karthikeyannhs force-pushed the elid-545/programatically_updating_tableau_daskboard branch from 14fc337 to b4eec65 Compare March 27, 2026 09:54
@Karthikeyannhs Karthikeyannhs force-pushed the elid-545/programatically_updating_tableau_daskboard branch from b4eec65 to 1b36555 Compare March 27, 2026 09:59
@Karthikeyannhs Karthikeyannhs force-pushed the elid-545/programatically_updating_tableau_daskboard branch from 2925a67 to 47371bb Compare March 27, 2026 11:39
@Karthikeyannhs Karthikeyannhs marked this pull request as ready for review March 27, 2026 11:43
@Karthikeyannhs Karthikeyannhs marked this pull request as draft March 27, 2026 12:12
@Karthikeyannhs Karthikeyannhs marked this pull request as ready for review March 27, 2026 12:12
TOEL2
TOEL2 previously approved these changes Mar 27, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an automated path for refreshing Tableau DQ metrics by generating a .hyper extract from S3 JSON partitions and publishing/refreshing a Tableau datasource on a schedule.

Changes:

  • Add a GitHub Actions scheduled workflow to run the end-to-end S3 → Hyper → Tableau publish/refresh process.
  • Add a script to read recent partitioned JSON from S3 and generate a Tableau Hyper extract.
  • Add a script to publish/overwrite a Tableau datasource and trigger a cache refresh via a view ping.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 9 comments.

File Description
scripts/tableau/tableau_refresh.py Publishes an updated .hyper to Tableau Server and attempts to trigger a view refresh.
scripts/tableau/generate_tableau_data.py Pulls recent JSON from S3 and builds converted.hyper via tableauhyperapi.
.github/workflows/dev-tableau-update.yml Schedules and wires the scripts into a daily DEV automation workflow.

Comment thread .github/workflows/dev-tableau-update.yml Outdated
Comment thread scripts/tableau/tableau_refresh.py Outdated
Comment thread scripts/tableau/generate_tableau_data.py Outdated
Comment thread .github/workflows/dev-tableau-update.yml Outdated
Comment thread scripts/tableau/tableau_refresh.py Outdated
Comment thread scripts/tableau/tableau_refresh.py Outdated
Comment thread scripts/tableau/generate_tableau_data.py Outdated
Comment thread scripts/tableau/generate_tableau_data.py Outdated
Comment thread scripts/tableau/tableau_refresh.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Karthikeyannhs and others added 4 commits March 27, 2026 12:33
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@Karthikeyannhs Karthikeyannhs merged commit 41cdc7f into main Mar 27, 2026
26 checks passed
@Karthikeyannhs Karthikeyannhs deleted the elid-545/programatically_updating_tableau_daskboard branch March 27, 2026 13:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants