CCM-13304: Generate reports (#203)

Author: simonlabarere

.editorconfig

@@ -14,6 +14,9 @@ indent_size = unset
 [*.py]
 indent_size = 4
 
+[*.ts]
+quote_type = single
+
 [{Dockerfile,Dockerfile.}*]
 indent_size = 4
 

.github/workflows/stage-2-test.yaml

@@ -132,7 +132,7 @@ jobs:
   test-lint:
     name: "Linting"
     runs-on: ubuntu-latest
-    timeout-minutes: 4
+    timeout-minutes: 6
     permissions:
       contents: read
       packages: read

.trivyignore

@@ -2,5 +2,4 @@
 CVE-2024-49761 # https://avd.aquasec.com/nvd/cve-2024-49761 ## latest Jekyll Webpack (0.2.7) installs old version of rexml
 CVE-2024-47220 # https://avd.aquasec.com/nvd/cve-2024-47220 ## latest lint_roller (1.1.0) installs old version of rexml
 CVE-2024-7254 # https://avd.aquasec.com/nvd/cve-2024-7254 ## latest Jekyll Webpack (0.2.7) installs old version of google-protobuf
-
 CVE-2026-1615 # https://avd.aquasec.com/nvd/cve-2026-1615 ## jsonpath: Arbitrary Code Execution via unsafe JSON Path expression evaluation - Not fixed yet.

README.md

@@ -0,0 +1,10 @@
+---
+title: queue-digital-letter-read
+type: uk.nhs.notify.digital.letters.queue.digital.letter.read.v1
+nice_name: DigitalLetterRead
+service: Queue Services
+schema_envelope:  https://notify.nhs.uk/cloudevents/schemas/digital-letters/2025-10-draft/events/uk.nhs.notify.digital.letters.queue.digital.letter.read.v1.schema.json
+schema_data: https://notify.nhs.uk/cloudevents/schemas/digital-letters/2025-10-draft/data/digital-letters-queue-digital-letter-read-data.schema.json
+---
+
+This indicates that a digital letter has been read on the NHS App.

docs/collections/_events/queue-digital-letter-read.md

@@ -7,4 +7,4 @@ schema_envelope:  https://notify.nhs.uk/cloudevents/schemas/digital-letters/2025
 schema_data: https://notify.nhs.uk/cloudevents/schemas/digital-letters/2025-10-draft/data/digital-letters-queue-item-enqueued-data.schema.json
 ---
 
-This event indicates that a TTL record has been queued for a given letter. The TTL record will either be intentionally removed before its expiry if the letter is read digitally (indicated by an ItemRemoved event) or will be dequeued after the fallback wait time (indicated by a ItemDequeued event) to trigger printing.
+This event indicates that a TTL record has been queued for a given letter. The TTL record will either be intentionally removed before its expiry if the letter is read digitally (indicated by a DigitalLetterRead event) or will be dequeued after the fallback wait time (indicated by a ItemDequeued event) to trigger printing.

docs/collections/_events/queue-item-enqueued.md

@@ -13,6 +13,8 @@ No requirements.
 | <a name="input_apim_auth_token_url"></a> [apim\_auth\_token\_url](#input\_apim\_auth\_token\_url) | URL to generate an APIM auth token | `string` | `"https://int.api.service.nhs.uk/oauth2/token"` | no |
 | <a name="input_apim_base_url"></a> [apim\_base\_url](#input\_apim\_base\_url) | The URL used to send requests to PDM | `string` | `"https://int.api.service.nhs.uk"` | no |
 | <a name="input_apim_keygen_schedule"></a> [apim\_keygen\_schedule](#input\_apim\_keygen\_schedule) | Schedule to refresh key pairs if necessary | `string` | `"cron(0 14 * * ? *)"` | no |
+| <a name="input_athena_query_max_polling_attempts"></a> [athena\_query\_max\_polling\_attempts](#input\_athena\_query\_max\_polling\_attempts) | The number of times athena will be polled to check if a query is completed | `number` | `3` | no |
+| <a name="input_athena_query_polling_time_seconds"></a> [athena\_query\_polling\_time\_seconds](#input\_athena\_query\_polling\_time\_seconds) | The amount of time in seconds to wait between each athena poll | `number` | `15` | no |
 | <a name="input_aws_account_id"></a> [aws\_account\_id](#input\_aws\_account\_id) | The AWS Account ID (numeric) | `string` | n/a | yes |
 | <a name="input_aws_account_type"></a> [aws\_account\_type](#input\_aws\_account\_type) | The AWS Account Type | `string` | n/a | yes |
 | <a name="input_component"></a> [component](#input\_component) | The variable encapsulating the name of this component | `string` | `"dl"` | no |
@@ -42,12 +44,15 @@ No requirements.
 | <a name="input_parent_acct_environment"></a> [parent\_acct\_environment](#input\_parent\_acct\_environment) | Name of the environment responsible for the acct resources used, affects things like DNS zone. Useful for named dev environments | `string` | `"main"` | no |
 | <a name="input_pdm_mock_access_token"></a> [pdm\_mock\_access\_token](#input\_pdm\_mock\_access\_token) | Mock access token for PDM API authentication (used in local/dev environments) | `string` | `"mock-pdm-token"` | no |
 | <a name="input_pdm_use_non_mock_token"></a> [pdm\_use\_non\_mock\_token](#input\_pdm\_use\_non\_mock\_token) | Whether to use the shared APIM access token from SSM (/component/environment/apim/access\_token) instead of the mock token | `bool` | `false` | no |
+| <a name="input_pii_data_retention_non_current_days"></a> [pii\_data\_retention\_non\_current\_days](#input\_pii\_data\_retention\_non\_current\_days) | The number of non current days for data retention policy for PII | `number` | `14` | no |
 | <a name="input_pii_data_retention_policy_days"></a> [pii\_data\_retention\_policy\_days](#input\_pii\_data\_retention\_policy\_days) | The number of days for data retention policy for PII | `number` | `534` | no |
 | <a name="input_project"></a> [project](#input\_project) | The name of the tfscaffold project | `string` | n/a | yes |
 | <a name="input_queue_batch_size"></a> [queue\_batch\_size](#input\_queue\_batch\_size) | maximum number of queue items to process | `number` | `10` | no |
 | <a name="input_queue_batch_window_seconds"></a> [queue\_batch\_window\_seconds](#input\_queue\_batch\_window\_seconds) | maximum time in seconds between processing events | `number` | `1` | no |
 | <a name="input_region"></a> [region](#input\_region) | The AWS Region | `string` | n/a | yes |
 | <a name="input_report_scheduler_schedule"></a> [report\_scheduler\_schedule](#input\_report\_scheduler\_schedule) | Schedule to trigger sender reports | `string` | `"cron(30 4 * * ? *)"` | no |
+| <a name="input_reports_data_retention_non_current_days"></a> [reports\_data\_retention\_non\_current\_days](#input\_reports\_data\_retention\_non\_current\_days) | The number of non current days for data retention policy for reports generated by Athena in the reporting bucket | `number` | `14` | no |
+| <a name="input_reports_data_retention_policy_days"></a> [reports\_data\_retention\_policy\_days](#input\_reports\_data\_retention\_policy\_days) | The number of days for data retention policy for reports generated by Athena in the reporting bucket | `number` | `90` | no |
 | <a name="input_shared_infra_account_id"></a> [shared\_infra\_account\_id](#input\_shared\_infra\_account\_id) | The AWS Shared Infra Account ID (numeric) | `string` | n/a | yes |
 | <a name="input_sns_success_logging_sample_percent"></a> [sns\_success\_logging\_sample\_percent](#input\_sns\_success\_logging\_sample\_percent) | Enable SNS Delivery Successful Sample Percentage | `number` | `0` | no |
 | <a name="input_sqs_max_receive_count"></a> [sqs\_max\_receive\_count](#input\_sqs\_max\_receive\_count) | Maximum number of times a message can be received before being sent to the DLQ | `string` | `"3"` | no |
@@ -73,6 +78,7 @@ No requirements.
 | <a name="module_print_sender"></a> [print\_sender](#module\_print\_sender) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/3.0.6/terraform-lambda.zip | n/a |
 | <a name="module_print_status_handler"></a> [print\_status\_handler](#module\_print\_status\_handler) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/3.0.6/terraform-lambda.zip | n/a |
 | <a name="module_report_event_transformer"></a> [report\_event\_transformer](#module\_report\_event\_transformer) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/3.0.6/terraform-lambda.zip | n/a |
+| <a name="module_report_generator"></a> [report\_generator](#module\_report\_generator) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/3.0.6/terraform-lambda.zip | n/a |
 | <a name="module_report_scheduler"></a> [report\_scheduler](#module\_report\_scheduler) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/3.0.6/terraform-lambda.zip | n/a |
 | <a name="module_report_sender"></a> [report\_sender](#module\_report\_sender) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/3.0.6/terraform-lambda.zip | n/a |
 | <a name="module_s3bucket_cf_logs"></a> [s3bucket\_cf\_logs](#module\_s3bucket\_cf\_logs) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/3.0.6/terraform-s3bucket.zip | n/a |
@@ -93,6 +99,7 @@ No requirements.
 | <a name="module_sqs_print_analyser"></a> [sqs\_print\_analyser](#module\_sqs\_print\_analyser) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/3.0.6/terraform-sqs.zip | n/a |
 | <a name="module_sqs_print_sender"></a> [sqs\_print\_sender](#module\_sqs\_print\_sender) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/3.0.6/terraform-sqs.zip | n/a |
 | <a name="module_sqs_print_status_handler"></a> [sqs\_print\_status\_handler](#module\_sqs\_print\_status\_handler) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/3.0.6/terraform-sqs.zip | n/a |
+| <a name="module_sqs_report_generator"></a> [sqs\_report\_generator](#module\_sqs\_report\_generator) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/3.0.6/terraform-sqs.zip | n/a |
 | <a name="module_sqs_report_sender"></a> [sqs\_report\_sender](#module\_sqs\_report\_sender) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/3.0.6/terraform-sqs.zip | n/a |
 | <a name="module_sqs_scanner"></a> [sqs\_scanner](#module\_sqs\_scanner) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/3.0.6/terraform-sqs.zip | n/a |
 | <a name="module_sqs_ttl"></a> [sqs\_ttl](#module\_sqs\_ttl) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/3.0.6/terraform-sqs.zip | n/a |

docs/collections/_events/queue-item-removed.md

@@ -0,0 +1,6 @@
+resource "aws_athena_named_query" "daily_report" {
+  name      = "${local.csi}-daily-report"
+  workgroup = aws_athena_workgroup.reporting.id
+  database  = aws_glue_catalog_database.reporting.name
+  query     = file("${path.module}/scripts/sql/reports/daily_report.sql")
+}