Skip to content

Commit 3a2b23e

Browse files
authored
fix(develop): Update Python CI setup doc (#16361)
Outdated, updating. ## IS YOUR CHANGE URGENT? Help us prioritize incoming PRs by letting us know when the change needs to go live. - [ ] Urgent deadline (GA date, etc.): <!-- ENTER DATE HERE --> - [ ] Other deadline: <!-- ENTER DATE HERE --> - [x] None: Not urgent, can wait up to 1 week+ ## SLA - Teamwork makes the dream work, so please add a reviewer to your PRs. - Please give the docs team up to 1 week to review your PR unless you've added an urgent due date to it. Thanks in advance for your help! ## PRE-MERGE CHECKLIST *Make sure you've checked the following before merging your changes:* - [ ] Checked Vercel preview for correctness, including links - [ ] PR was reviewed and approved by any necessary SMEs (subject matter experts) - [ ] PR was reviewed and approved by a member of the [Sentry docs team](https://github.com/orgs/getsentry/teams/docs) ## LEGAL BOILERPLATE <!-- Sentry employees and contractors can delete or ignore this section. --> Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. and is gonna need some rights from me in order to utilize my contributions in this here PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms. ## EXTRA RESOURCES - [Sentry Docs contributor guide](https://docs.sentry.io/contributing/)
1 parent 69ec2ae commit 3a2b23e

1 file changed

Lines changed: 11 additions & 12 deletions

File tree

  • develop-docs/sdk/platform-specifics/python-sdk

develop-docs/sdk/platform-specifics/python-sdk/ci.mdx

Lines changed: 11 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -4,20 +4,19 @@ description: Our test setup and how our CI files are generated.
44
sidebar_order: 20
55
---
66

7-
The GitHub workflow files for testing specific integration groups (like AI, Web Frameworks, Databases, etc.) are generated by [this script](https://github.com/getsentry/sentry-python/blob/master/scripts/split-tox-gh-actions/split-tox-gh-actions.py). Essentially, the script scans our `tox.ini` and generates the group test CI YAML files in `.github/workflows`.
7+
Our test suite consists of a common part that tests basic SDK functionality, and an integrations part that tests individual integrations.
88

9-
There are multiple components to this:
10-
* `tox.ini` is where the configuration (what to test and what versions) is read from
11-
* `scripts/split-tox-gh-actions/split-tox-gh-actions.py` is the script that defines the [test groups](https://github.com/getsentry/sentry-python/blob/0f3e5db0c8aabcad0baf0e8b2d3e31e27e839b3e/scripts/split-tox-gh-actions/split-tox-gh-actions.py#L57) and generates the resulting YAML files
12-
* `scripts/split-tox-gh-actions/templates` contains the jinja2 template snippets used to generate the YAML files
13-
* `.github/workflows/test-integrations-*.yml` are the resulting workflow configuration YAMLs
14-
15-
If you update any of the components without keeping the rest in sync, for instance by making a change in one of the `test-integrations-*.yml` files that is not reflected in the templates in `scripts/split-tox-gh-actions/templates`, CI will error out as it actually checks if everything is in sync, meaning it runs the `split-tox-gh-actions.py` script and compares the result with the committed YAMLs.
9+
Since the number of integrations to test is so large, the tests in CI are split into multiple groups (roughly by integration type) so that they can run in parallel. So there is, for example, [a GitHub workflow](https://github.com/getsentry/sentry-python/blob/master/.github/workflows/test-integrations-flags.yml) that runs all our feature flag integrations, and [another one](https://github.com/getsentry/sentry-python/blob/master/.github/workflows/test-integrations-graphql.yml) that runs the test suites of all GraphQL integrations, and so on. These workflow YAMLs are auto-generated by [this script](https://github.com/getsentry/sentry-python/blob/master/scripts/split-tox-gh-actions/split-tox-gh-actions.py).
1610

17-
## AWS Lambda Test Suite
11+
The ultimate source of truth of what should be tested and on which versions (both Python versions as well as package versions) is stored in [`tox.ini`](https://github.com/getsentry/sentry-python/blob/master/tox.ini). This file is also auto-generated each week by a GitHub cron job that scans PyPI for new releases and updates the test matrix with them. The script that does this lives in [`scripts/populate_tox`](https://github.com/getsentry/sentry-python/tree/master/scripts/populate_tox) and is often referred to as "toxgen".
1812

19-
The AWS Lambda test suite will fail by default on new PRs. If you are an external contributor, this is not something you have to worry about, unless you've made changes actually affecting the AWS Lambda integration. It's a security precaution on our part.
13+
## The Big Picture
2014

21-
Sensitive test suites (currently only AWS Lambda) require maintainer review to ensure that tests do not compromise our secrets. This review must be repeated after any code revisions.
15+
There are multiple components to the test setup that runs in CI:
16+
* on a weekly basis, [`scripts/populate_tox`](https://github.com/getsentry/sentry-python/tree/master/scripts/populate_tox) creates a PR that updates `tox.ini` with new releases
17+
* `tox.ini` is where the configuration (what to test and what versions) is stored and read from
18+
* `scripts/split-tox-gh-actions/split-tox-gh-actions.py` is the script that defines the [test groups](https://github.com/getsentry/sentry-python/blob/0f3e5db0c8aabcad0baf0e8b2d3e31e27e839b3e/scripts/split-tox-gh-actions/split-tox-gh-actions.py#L57) and generates the resulting YAML files
19+
* `scripts/split-tox-gh-actions/templates` contains the jinja2 template snippets used to generate the YAML files
20+
* `.github/workflows/test-integrations-*.yml` are the resulting workflow configuration YAMLs
2221

23-
Before running sensitive test suites, maintainers need to carefully check the PR. Then they will apply the `Trigger: tests using secrets` label. The label will be removed after any code changes to enforce our policy requiring maintainers to review all code revisions before running sensitive tests.
22+
If you update any of the components without keeping the rest in sync, for instance by making a change in one of the `test-integrations-*.yml` files that is not reflected in the templates in `scripts/split-tox-gh-actions/templates`, they will be overwritten next time they're auto-generated.

0 commit comments

Comments
 (0)