Skip to content

Commit 042a38c

Browse files
TineoCTineoC
committed
feat(gcs): support Workload Identity for GCS backup storage
Add support for GKE Workload Identity Federation when authenticating with Google Cloud Storage for backups. This eliminates the requirement for exported service account JSON keys (credentialsSecret). Changes: - Add GCSCredentials struct with WorkloadIdentity field to BackupStorageGCSSpec - Make credentialsSecret optional (omitempty) when workloadIdentity is enabled - Update GetPBMStorageGCSConfig to skip credential secret loading when WI is enabled - Patch PBM's newGoogleClient to fall back to Application Default Credentials (ADC) when no explicit credentials are provided, enabling transparent GKE WI auth When credentials.workloadIdentity is true in the CR: spec.backup.storages.gcs.credentials.workloadIdentity: true The operator skips reading credentialsSecret and PBM authenticates via the pod's Kubernetes Service Account, which is federated to a GCP SA through GKE Workload Identity. This aligns with PBM 2.13.0's native WIF support and Google's security best practice of avoiding exported service account keys, which is required for IL4/FedRAMP environments. Closes: #2314
1 parent 5856026 commit 042a38c

280 files changed

Lines changed: 59945 additions & 8 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎go.mod‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -223,3 +223,5 @@ replace (
223223
github.com/dgrijalva/jwt-go v3.2.0+incompatible => github.com/golang-jwt/jwt/v4 v4.2.0
224224
gopkg.in/natefinch/lumberjack.v2 v2.2.1 => ./cmd/mongodb-healthcheck/logger/lumberjack/ // https://github.com/natefinch/lumberjack/pull/211
225225
)
226+
227+
replace github.com/percona/percona-backup-mongodb => ./pbm-patched

‎go.sum‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -418,8 +418,6 @@ github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8
418418
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
419419
github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040=
420420
github.com/opencontainers/image-spec v1.1.1/go.mod h1:qpqAh3Dmcf36wStyyWU+kCeDgrGnAve2nCC8+7h8Q0M=
421-
github.com/percona/percona-backup-mongodb v1.8.1-0.20251104101930-05ab6d7e1004 h1:Q/HeUxMI63ekZYY0TlnfF0Vp3WZ0mmx7gI9iBzehjeY=
422-
github.com/percona/percona-backup-mongodb v1.8.1-0.20251104101930-05ab6d7e1004/go.mod h1:NYc9wcoGLNXmOHwfsaed+Ej3nxv3dUViHOcCfQneJ84=
423421
github.com/philhofer/fwd v1.2.0 h1:e6DnBTl7vGY+Gz322/ASL4Gyp1FspeMvx1RNDoToZuM=
424422
github.com/philhofer/fwd v1.2.0/go.mod h1:RqIHx9QI14HlwKwm98g9Re5prTQ6LdeRQn+gXJFxsJM=
425423
github.com/pierrec/lz4 v2.6.1+incompatible h1:9UY3+iC23yxF0UfGaYrGplQ+79Rg+h/q9FV9ix19jjM=

‎pbm-patched/.dockerignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
/bin/
2+
/dev/
3+
/.dev
4+
e2e-tests/docker/backups/pbm/

‎pbm-patched/.github/pr-badge.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
label: "JIRA"
2+
url: "https://jira.percona.com/browse/$issuePrefix"
3+
message: "$issuePrefix"
4+
color: "green"
5+
when: "$issuePrefix"

‎pbm-patched/.github/workflows/ci.yml‎

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
name: CI
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
pbm_branch:
7+
description: "PBM branch"
8+
required: false
9+
tests_ver:
10+
description: "Tests version"
11+
required: false
12+
go_ver:
13+
description: "Golang version"
14+
required: false
15+
pr_ver:
16+
description: "PR version"
17+
required: false
18+
19+
pull_request:
20+
branches:
21+
- main
22+
- dev
23+
paths-ignore:
24+
- "e2e-tests/**"
25+
- "packaging/**"
26+
- "version/**"
27+
28+
push:
29+
branches:
30+
- main
31+
- dev
32+
paths-ignore:
33+
- "e2e-tests/**"
34+
- "packaging/**"
35+
- "version/**"
36+
37+
jobs:
38+
test:
39+
runs-on: ubuntu-latest
40+
timeout-minutes: 180
41+
strategy:
42+
fail-fast: false
43+
matrix:
44+
psmdb: ["7.0", "8.0"]
45+
shard: [0, 1, 2, 3, 4, 5, 6]
46+
env:
47+
PBM_BRANCH: ${{ github.event.inputs.pbm_branch || github.ref_name }}
48+
GO_VER: ${{ github.event.inputs.go_ver || '1.25-bookworm' }}
49+
PR_NUMBER: ${{ github.event.number|| github.event.inputs.pr_ver }}
50+
MAKE_TARGET: 'build-cover'
51+
steps:
52+
- name: Checkout testing repo
53+
uses: actions/checkout@v4
54+
with:
55+
repository: Percona-QA/psmdb-testing
56+
ref: ${{ github.event.inputs.tests_ver || 'main'}}
57+
path: psmdb-testing
58+
59+
- name: Setup environment with PSMDB ${{ matrix.psmdb }} for PBM PR/branch ${{ github.event.pull_request.title || env.PR_NUMBER || env.PBM_BRANCH }}
60+
run: |
61+
PSMDB=perconalab/percona-server-mongodb:${{ matrix.psmdb }} docker compose build easyrsa
62+
PSMDB=perconalab/percona-server-mongodb:${{ matrix.psmdb }} docker compose build
63+
docker compose up -d
64+
working-directory: psmdb-testing/pbm-functional/pytest
65+
66+
- name: Run pytest shard number ${{ matrix.shard }} on PSMDB ${{ matrix.psmdb }} for PBM PR/branch ${{ github.event.pull_request.title || env.PR_NUMBER || env.PBM_BRANCH }}
67+
run: |
68+
docker compose run test pytest -s --junitxml=junit.xml --shard-id=${{ matrix.shard }} --num-shards=7 -m 'not jenkins and not skip'
69+
working-directory: psmdb-testing/pbm-functional/pytest
70+
71+
- name: Fetch coverage files
72+
run: |
73+
docker compose run --rm golang_reports cp -r /gocoverdir/reports /test
74+
sudo chmod -R 777 reports
75+
working-directory: psmdb-testing/pbm-functional/pytest
76+
if: success() || failure()
77+
78+
- name: Upload coverage reports
79+
uses: actions/upload-artifact@v4
80+
with:
81+
name: reports-${{ matrix.shard }}-${{ matrix.psmdb }}
82+
path: psmdb-testing/pbm-functional/pytest/reports/
83+
if: success() || failure()
84+
85+
- name: Publish Test Report
86+
uses: mikepenz/action-junit-report@v4
87+
if: success() || failure()
88+
with:
89+
report_paths: "**/junit.xml"
90+
91+
coverage:
92+
if: ${{ always() }}
93+
needs: test
94+
runs-on: ubuntu-latest
95+
steps:
96+
- uses: actions/checkout@v4
97+
- name: Set up Go
98+
uses: actions/setup-go@v5
99+
with:
100+
go-version-file: 'go.mod'
101+
check-latest: true
102+
- name: Download all coverage reports
103+
uses: actions/download-artifact@v4
104+
with:
105+
path: reports
106+
pattern: reports-*
107+
merge-multiple: true
108+
- name: Merge coverage reports
109+
run: |
110+
go tool covdata textfmt -i=./reports -o ./coverage.txt
111+
- name: Upload coverage to Codecov
112+
uses: codecov/codecov-action@v3
113+
with:
114+
files: ./coverage.txt
115+
flags: integration
116+
fail_ci_if_error: false
117+
token: ${{ secrets.CODECOV_TOKEN }}

‎pbm-patched/.github/workflows/codecov.yml‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
name: 'codecov'
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
- dev
8+
pull_request:
9+
branches:
10+
- main
11+
- dev
12+
13+
jobs:
14+
go-test:
15+
name: runner / go-test
16+
runs-on: ubuntu-latest
17+
steps:
18+
- uses: actions/checkout@v4
19+
- uses: actions/setup-go@v4
20+
with:
21+
go-version: "1.25"
22+
- name: test
23+
run: go test -v ./... -covermode=atomic -coverprofile=cover.out
24+
25+
- name: upload coverage report
26+
uses: codecov/codecov-action@v4
27+
with:
28+
file: cover.out
29+
flags: unittests
30+
fail_ci_if_error: false
31+
token: ${{ secrets.CODECOV_TOKEN }}

‎pbm-patched/.github/workflows/reviewdog.yml‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
name: reviewdog
2+
on: [pull_request]
3+
jobs:
4+
shellcheck:
5+
name: runner / shellcheck
6+
runs-on: ubuntu-latest
7+
if: ${{ github.event.pull_request.changed_files < 301 }}
8+
steps:
9+
- uses: actions/checkout@v4
10+
- name: spellcheck
11+
uses: reviewdog/action-shellcheck@v1
12+
with:
13+
reporter: github-pr-review
14+
exclude: |
15+
*/.git/*
16+
./.cache/*
17+
./vendor/*
18+
./packaging/scripts/*
19+
20+
misspell:
21+
name: runner / misspell
22+
runs-on: ubuntu-latest
23+
if: ${{ github.event.pull_request.changed_files < 301 }}
24+
steps:
25+
- uses: actions/checkout@v4
26+
- name: misspell
27+
uses: reviewdog/action-misspell@v1
28+
with:
29+
reporter: github-pr-review
30+
locale: "US"
31+
exclude: |
32+
*/.git/*
33+
./.cache/*
34+
./vendor/*
35+
36+
alex:
37+
name: runner / alex
38+
runs-on: ubuntu-latest
39+
if: ${{ github.event.pull_request.changed_files < 301 }}
40+
steps:
41+
- uses: actions/checkout@v4
42+
- name: alex
43+
uses: reviewdog/action-alex@v1
44+
with:
45+
reporter: github-pr-review
46+
47+
golangci-lint:
48+
name: runner / golangci-lint
49+
runs-on: ubuntu-latest
50+
steps:
51+
- uses: actions/checkout@v4
52+
- uses: actions/setup-go@v4
53+
with:
54+
go-version: "1.25"
55+
- name: golangci-lint
56+
uses: reviewdog/action-golangci-lint@v2
57+
with:
58+
reporter: github-pr-review
59+
level: error
60+
61+
gofmt:
62+
name: runner / gofmt
63+
runs-on: ubuntu-latest
64+
steps:
65+
- uses: actions/checkout@v4
66+
- uses: actions/setup-go@v4
67+
with:
68+
go-version: "1.25"
69+
- run: go install golang.org/x/tools/cmd/goimports@latest
70+
- run: go install mvdan.cc/gofumpt@latest
71+
- run: goimports -w -local "github.com/percona" $(find . -not -path "*/vendor/*" -name "*.go")
72+
- run: gofumpt -w -extra $(find . -not -path "*/vendor/*" -name "*.go")
73+
- uses: reviewdog/action-suggester@v1.22.0
74+
with:
75+
tool_name: gofmt
76+
77+
shfmt:
78+
name: runner / shfmt
79+
runs-on: ubuntu-latest
80+
steps:
81+
- uses: actions/checkout@v4
82+
- uses: actions/setup-go@v4
83+
with:
84+
go-version: "1.25"
85+
- run: go install mvdan.cc/sh/v3/cmd/shfmt@latest
86+
- run: shfmt -f . | grep -v 'vendor' | xargs shfmt -w -s
87+
- name: suggester / shfmt
88+
uses: reviewdog/action-suggester@v1.22.0
89+
with:
90+
tool_name: shfmt

‎pbm-patched/.github/workflows/trivy.yml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
name: Scan
2+
on:
3+
push:
4+
branches: ["main", "dev"]
5+
pull_request:
6+
branches: ["main", "dev"]
7+
jobs:
8+
scan:
9+
name: Trivy
10+
runs-on: ubuntu-latest
11+
steps:
12+
- name: Checkout code
13+
uses: actions/checkout@v4
14+
with:
15+
ref: ${{ github.event.pull_request.head.sha }}
16+
17+
- name: Download latest trivy
18+
run: |
19+
mkdir -p ${{ github.workspace }}/trivy
20+
LATEST_TRIVY_VERSION=$(curl --retry 5 --retry-connrefused --retry-delay 5 --fail -s https://api.github.com/repos/aquasecurity/trivy/releases/latest | jq -r .tag_name)
21+
TRIVY_VERSION_STRIPPED=$(echo "$LATEST_TRIVY_VERSION" | sed 's/^v//')
22+
wget --tries=5 --retry-connrefused --waitretry=5 -O ${{ github.workspace }}/trivy/trivy.tar.gz \
23+
https://github.com/aquasecurity/trivy/releases/download/$LATEST_TRIVY_VERSION/trivy_${TRIVY_VERSION_STRIPPED}_Linux-64bit.tar.gz && break || sleep 5
24+
tar -xzf ${{ github.workspace }}/trivy/trivy.tar.gz -C ${{ github.workspace }}/trivy
25+
26+
- name: Generate SBOM
27+
run: ${{ github.workspace }}/trivy/trivy fs --format cyclonedx --output ${{ github.workspace }}/sbom.json ${{ github.workspace }}
28+
29+
- name: Run trivy scan on SBOM
30+
run: ${{ github.workspace }}/trivy/trivy sbom ${{ github.workspace }}/sbom.json --severity HIGH,CRITICAL --ignore-unfixed --exit-code=1

‎pbm-patched/.gitignore‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
.env
2+
.vscode/*
3+
!.vscode/settings.json
4+
!.vscode/extensions.json
5+
.idea/
6+
/bin/
7+
/.dev
8+
e2e-tests/docker/keyFile
9+
e2e-tests/docker/backups/pbm
10+
e2e-tests/docker/pbm.conf.yaml
11+
e2e-tests/docker/conf/aws.yaml
12+
e2e-tests/docker/conf/gcs.yaml
13+
e2e-tests/docker/conf/azure.yaml

‎pbm-patched/.golangci.yml‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
version: "2"
2+
linters:
3+
enable:
4+
- errname
5+
- predeclared
6+
- staticcheck
7+
- misspell
8+
- nilerr
9+
- nilnil
10+
- lll
11+
- gochecknoinits
12+
- nonamedreturns
13+
disable:
14+
- contextcheck
15+
- exhaustive
16+
- gosec
17+
- wrapcheck
18+
settings:
19+
govet:
20+
disable:
21+
- composites
22+
misspell:
23+
locale: US
24+
exclusions:
25+
generated: lax
26+
presets:
27+
- comments
28+
- common-false-positives
29+
- legacy
30+
- std-error-handling
31+
paths:
32+
- third_party$
33+
- builtin$
34+
- examples$
35+
formatters:
36+
exclusions:
37+
generated: lax
38+
paths:
39+
- third_party$
40+
- builtin$
41+
- examples$

0 commit comments

Comments
 (0)