Skip to content

Allow partialFilterExpression in backbeat index add payload - #6274

Open
maeldonn wants to merge 1 commit into
development/9.4from
bugfix/CLDSRV-991/partial-filter-expression
Open

Allow partialFilterExpression in backbeat index add payload#6274
maeldonn wants to merge 1 commit into
development/9.4from
bugfix/CLDSRV-991/partial-filter-expression

Conversation

@maeldonn

@maeldonn maeldonn commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Since BB-773 (backbeat >= 9.4.1) the lifecycle conductor sends its v2 index
specs with a partialFilterExpression, rejected as an unknown field by the
Joi schema of POST /_/backbeat/index?operation=add. The v2 lifecycle
indexes were never created and buckets fell back to v1 listings (BB-866).

Declare the field as an optional object (the rest of the chain already
forwarded it to MongoDB) and add route-level tests.

Issue: CLDSRV-991

Since BB-773 (backbeat 9.4.1) the lifecycle conductor sends index
specs with a partialFilterExpression, rejected as an unknown field
by the Joi schema of POST /_/backbeat/index?operation=add. The v2
lifecycle indexes were never created. Allow the field and add
route-level tests.

Issue: CLDSRV-991
@maeldonn
maeldonn requested review from a team, DarkIsDude and francoisferrand September 7, 2026 15:48
@bert-e

bert-e commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Hello maeldonn,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options
name description privileged authored
/after_pull_request Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval Bypass the pull request author's approval
/bypass_build_status Bypass the build and test status
/bypass_commit_size Bypass the check on the size of the changeset TBA
/bypass_incompatible_branch Bypass the check on the source branch prefix
/bypass_jira_check Bypass the Jira issue check
/bypass_peer_approval Bypass the pull request peers' approval
/bypass_leader_approval Bypass the pull request leaders' approval
/bypass_source_branch_lineage Bypass the cross-branch contamination check
/approve Instruct Bert-E that the author has approved the pull request. ✍️
/create_pull_requests Allow the creation of integration pull requests.
/create_integration_branches Allow the creation of integration branches.
/no_octopus Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity Change review acceptance criteria from one reviewer at least to all reviewers
/wait Instruct Bert-E not to run until further notice.
Available commands
name description privileged
/help Print Bert-E's manual in the pull request.
/status Print Bert-E's current status in the pull request.
/clear Remove all comments from Bert-E from the history TBA
/retry Re-start a fresh build TBA
/build Re-start a fresh build TBA
/force_reset Delete integration branches & pull requests, and restart merge process from the beginning.
/reset Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

@bert-e

bert-e commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Incorrect fix version

The Fix Version/s in issue CLDSRV-991 contains:

  • None

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

  • 9.4.4

  • 9.5.0

Please check the Fix Version/s of CLDSRV-991, or the target
branch of this pull request.

@maeldonn
maeldonn requested a review from delthas September 7, 2026 15:48
@codecov

codecov Bot commented Sep 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.50%. Comparing base (4ffb60a) to head (d80bc44).
✅ All tests successful. No failed tests found.

Additional details and impacted files

Impacted file tree graph

Files with missing lines Coverage Δ
lib/routes/routeBackbeat.js 78.81% <ø> (ø)
@@               Coverage Diff                @@
##           development/9.4    #6274   +/-   ##
================================================
  Coverage            86.50%   86.50%           
================================================
  Files                  212      212           
  Lines                14579    14579           
================================================
  Hits                 12611    12611           
  Misses                1968     1968           
Flag Coverage Δ
checksums-disabled-tests 35.31% <ø> (ø)
file-ft-tests 70.05% <ø> (+0.04%) ⬆️
file-ft-tests-null-compat 70.48% <ø> (ø)
kmip-ft-tests 28.06% <ø> (ø)
mongo-v0-ft-tests 71.13% <ø> (-0.04%) ⬇️
mongo-v1-ft-tests 71.10% <ø> (-0.07%) ⬇️
multiple-backend 36.07% <ø> (ø)
s3c-ft-tests-v0 65.01% <ø> (ø)
s3c-ft-tests-v0-null-compat 65.07% <ø> (ø)
s3c-ft-tests-v1 64.99% <ø> (ø)
sur-tests 36.61% <ø> (-0.88%) ⬇️
sur-tests-inflights 39.44% <ø> (ø)
unit 74.20% <ø> (+0.02%) ⬆️
utapi-v2-tests 35.29% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@bert-e

bert-e commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

@delthas delthas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wow, good catch 😅 Thanks!

}),
)
.required(),
partialFilterExpression: joi.object(),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we validate it ?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean declare the schema ? Can be hard as it's mongo but pretty sure we can find some joi lib to validate mongo ?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is validated by mongo during the call, so is it worth duplicating the validation?

  • we don't actually own the format, we just pass it through: so not sure we add value.
  • it's fine not to own it, since this is an internal API only.

(the alternative would be to define our own "index definition format", and map it to mongo format in cloudserver : in that case we should indeed do validation)

});
});

describe('routeBackbeat index add payload validation', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing a test : should reject if the field is not well formatted ?

}),
)
.required(),
partialFilterExpression: joi.object(),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is validated by mongo during the call, so is it worth duplicating the validation?

  • we don't actually own the format, we just pass it through: so not sure we add value.
  • it's fine not to own it, since this is an internal API only.

(the alternative would be to define our own "index definition format", and map it to mongo format in cloudserver : in that case we should indeed do validation)

assert.strictEqual(err.code, 'BadRequest');
assert.strictEqual(metadata.putBucketIndexes.called, false);
});
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we have "integration" test, where we actually call mongo?
would be nice to have these, and thus test that we reject the call if either keys or partialFilterExpression is not a valid mongo format and that we pass the fields appropriately

afterEach(() => {
sinon.restore();
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing test where the request fails if backend is not mongo? or is it in some integration tests?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants