feat(batch): add option to continue processing other group IDs on failure in SqsFifoPartialProcessor

[arnabrahman]

Summary

When using SQS FIFO, by default, we will stop processing at the first failure and mark unprocessed messages as failed to preserve ordering. However, this behavior may not be optimal for customers who wish to proceed with processing messages from a different group ID.

Here an option has been given to the customer to skip the default behavior.

Changes

• Introduced a skipGroupOnError option that can be passed to the processPartialResponseSync function.
• Made a generic type so that skipGroupOnError is accepted only when the processor parameter is of type SqsFifoPartialProcessor. Example:

         const processor = new SqsFifoPartialProcessor();
         const result = processPartialResponseSync(
            event,
            sqsRecordHandler,
            processor,
            {
              context,
              skipGroupOnError: true,
            }
          );

• If skipGroupOnError is not true, then short-circuit the process on first failure same as before.
• If skipGroupOnError is true, then continue processing on the failure. Only if the current group has a previous failure then skip processing the current record and log SqsFifoMessageGroupShortCircuitError. Otherwise, process the record.
• Write unit tests for skipGroupOnError flag
• Updated the documentation, added examples and added sequence diagrams. I followed the Python powertools documentation for reference.

Issue number: #2561

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

[arnabrahman]

Ok, there are some merge conflicts. Will fix it and then make this ready.

[dreamorosi]

Hi @arnabrahman - thank you for your patience on this.

I've just got back from PTO and will review this in the next 24hrs tops.

[dreamorosi]

Thank you so much for the work on this PR, the change is super clean and I really appreciate that you went the extra mile and also updated the docs!

I have left only a few minor comments, after which I am looking forward to merge the PR and include the feature in this week's release :)

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
29.8% Duplication on New Code

See analysis details on SonarCloud

[dreamorosi]

Thank you for the thorough work on this PR, I really appreciate the attention to details and openness to work on iterations during the review.

Merging this now 🎉

PS: I've labelled two more issues (list) related to this utility as help wanted just in case you're interested. They're both in similar complexity and along the lines of this one.

[arnabrahman]

Nice, i will take a look after finishing 1774 , @dreamorosi