CCM-12614: Add PDM Poller component

infrastructure/terraform/components/dl/README.md

@@ -31,7 +31,7 @@ No requirements.
 | <a name="input_pdm_use_non_mock_token"></a> [pdm\_use\_non\_mock\_token](#input\_pdm\_use\_non\_mock\_token) | Whether to use the shared APIM access token from SSM (/component/environment/apim/access\_token) instead of the mock token | `bool` | `false` | no |
 | <a name="input_project"></a> [project](#input\_project) | The name of the tfscaffold project | `string` | n/a | yes |
 | <a name="input_queue_batch_size"></a> [queue\_batch\_size](#input\_queue\_batch\_size) | maximum number of queue items to process | `number` | `10` | no |
-| <a name="input_queue_batch_window_seconds"></a> [queue\_batch\_window\_seconds](#input\_queue\_batch\_window\_seconds) | maximum time in seconds between processing events | `number` | `10` | no |
+| <a name="input_queue_batch_window_seconds"></a> [queue\_batch\_window\_seconds](#input\_queue\_batch\_window\_seconds) | maximum time in seconds between processing events | `number` | `1` | no |
 | <a name="input_region"></a> [region](#input\_region) | The AWS Region | `string` | n/a | yes |
 | <a name="input_shared_infra_account_id"></a> [shared\_infra\_account\_id](#input\_shared\_infra\_account\_id) | The AWS Shared Infra Account ID (numeric) | `string` | n/a | yes |
 | <a name="input_ttl_poll_schedule"></a> [ttl\_poll\_schedule](#input\_ttl\_poll\_schedule) | Schedule to poll for any overdue TTL records | `string` | `"rate(10 minutes)"` | no |
@@ -44,11 +44,13 @@ No requirements.
 | <a name="module_lambda_lambda_apim_refresh_token"></a> [lambda\_lambda\_apim\_refresh\_token](#module\_lambda\_lambda\_apim\_refresh\_token) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/v2.0.29/terraform-lambda.zip | n/a |
 | <a name="module_mesh_poll"></a> [mesh\_poll](#module\_mesh\_poll) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/v2.0.29/terraform-lambda.zip | n/a |
 | <a name="module_pdm_mock"></a> [pdm\_mock](#module\_pdm\_mock) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/v2.0.29/terraform-lambda.zip | n/a |
+| <a name="module_pdm_poll"></a> [pdm\_poll](#module\_pdm\_poll) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/v2.0.24/terraform-lambda.zip | n/a |
 | <a name="module_pdm_uploader"></a> [pdm\_uploader](#module\_pdm\_uploader) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/v2.0.29/terraform-lambda.zip | n/a |
 | <a name="module_s3bucket_cf_logs"></a> [s3bucket\_cf\_logs](#module\_s3bucket\_cf\_logs) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/v2.0.24/terraform-s3bucket.zip | n/a |
 | <a name="module_s3bucket_letters"></a> [s3bucket\_letters](#module\_s3bucket\_letters) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/v2.0.24/terraform-s3bucket.zip | n/a |
 | <a name="module_s3bucket_static_assets"></a> [s3bucket\_static\_assets](#module\_s3bucket\_static\_assets) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/v2.0.24/terraform-s3bucket.zip | n/a |
 | <a name="module_sqs_event_publisher_errors"></a> [sqs\_event\_publisher\_errors](#module\_sqs\_event\_publisher\_errors) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/v2.0.24/terraform-sqs.zip | n/a |
+| <a name="module_sqs_pdm_poll"></a> [sqs\_pdm\_poll](#module\_sqs\_pdm\_poll) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/v2.0.24/terraform-sqs.zip | n/a |
 | <a name="module_sqs_pdm_uploader"></a> [sqs\_pdm\_uploader](#module\_sqs\_pdm\_uploader) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/v2.0.24/terraform-sqs.zip | n/a |
 | <a name="module_sqs_ttl"></a> [sqs\_ttl](#module\_sqs\_ttl) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/v2.0.24/terraform-sqs.zip | n/a |
 | <a name="module_sqs_ttl_handle_expiry_errors"></a> [sqs\_ttl\_handle\_expiry\_errors](#module\_sqs\_ttl\_handle\_expiry\_errors) | https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/v2.0.24/terraform-sqs.zip | n/a |

infrastructure/terraform/components/dl/aws_api_gateway_deployment_pdm_mock.tf

@@ -16,7 +16,9 @@ resource "aws_api_gateway_deployment" "pdm_mock" {
       aws_api_gateway_resource.document_reference[0].id,
       aws_api_gateway_resource.document_reference_id[0].id,
       aws_api_gateway_method.create_document_reference[0].id,
+      aws_api_gateway_method.create_document_reference[0].authorization,
       aws_api_gateway_method.get_document_reference[0].id,
+      aws_api_gateway_method.get_document_reference[0].authorization,
       aws_api_gateway_integration.create_document_reference[0].id,
       aws_api_gateway_integration.get_document_reference[0].id,
     ]))

infrastructure/terraform/components/dl/aws_api_gateway_method_create_document_reference.tf

@@ -4,5 +4,5 @@ resource "aws_api_gateway_method" "create_document_reference" {
   rest_api_id   = aws_api_gateway_rest_api.pdm_mock[0].id
   resource_id   = aws_api_gateway_resource.document_reference[0].id
   http_method   = "POST"
-  authorization = "AWS_IAM"
+  authorization = "NONE"
 }

infrastructure/terraform/components/dl/aws_api_gateway_method_get_document_reference.tf

@@ -4,5 +4,5 @@ resource "aws_api_gateway_method" "get_document_reference" {
   rest_api_id   = aws_api_gateway_rest_api.pdm_mock[0].id
   resource_id   = aws_api_gateway_resource.document_reference_id[0].id
   http_method   = "GET"
-  authorization = "AWS_IAM"
+  authorization = "NONE"
 }

infrastructure/terraform/components/dl/cloudwatch_event_rule_pdm_resource_submitted.tf

@@ -0,0 +1,19 @@
+resource "aws_cloudwatch_event_rule" "pdm_resource_submitted" {
+  name           = "${local.csi}-pdm-resource-submitted"
+  description    = "PDM resource submitted event rule"
+  event_bus_name = aws_cloudwatch_event_bus.main.name
+
+  event_pattern = jsonencode({
+    "detail" : {
+      "type" : [
+        "uk.nhs.notify.digital.letters.pdm.resource.submitted.v1"
+      ]
+    }
+  })
+}
+
+resource "aws_cloudwatch_event_target" "pdm_resource_submitted_pdm_poll" {
+  rule           = aws_cloudwatch_event_rule.pdm_resource_submitted.name
+  arn            = module.sqs_pdm_poll.sqs_queue_arn
+  event_bus_name = aws_cloudwatch_event_bus.main.name
+}

infrastructure/terraform/components/dl/cloudwatch_event_rule_pdm_resource_unavailable.tf

@@ -0,0 +1,19 @@
+resource "aws_cloudwatch_event_rule" "pdm_resource_unavailable" {
+  name           = "${local.csi}-pdm-resource-unavailable"
+  description    = "PDM resource unavailable event rule"
+  event_bus_name = aws_cloudwatch_event_bus.main.name
+
+  event_pattern = jsonencode({
+    "detail" : {
+      "type" : [
+        "uk.nhs.notify.digital.letters.pdm.resource.unavailable.v1"
+      ]
+    }
+  })
+}
+
+resource "aws_cloudwatch_event_target" "pdm_resource_unavailable_pdm_poll" {
+  rule           = aws_cloudwatch_event_rule.pdm_resource_unavailable.name
+  arn            = module.sqs_pdm_poll.sqs_queue_arn
+  event_bus_name = aws_cloudwatch_event_bus.main.name
+}

infrastructure/terraform/components/dl/lambda_event_source_mapping_pdm_poll_lambda.tf

@@ -0,0 +1,10 @@
+resource "aws_lambda_event_source_mapping" "pdm_poll_lambda" {
+  event_source_arn                   = module.sqs_pdm_poll.sqs_queue_arn
+  function_name                      = module.pdm_poll.function_name
+  batch_size                         = var.queue_batch_size
+  maximum_batching_window_in_seconds = var.queue_batch_window_seconds
+
+  function_response_types = [
+    "ReportBatchItemFailures"
+  ]
+}

infrastructure/terraform/components/dl/module_lambda_pdm_mock.tf

@@ -34,12 +34,6 @@ module "pdm_mock" {
 
   log_destination_arn       = local.log_destination_arn
   log_subscription_role_arn = local.acct.log_subscription_role_arn
-
-  lambda_env_vars = {
-    MOCK_ACCESS_TOKEN     = var.pdm_mock_access_token
-    ACCESS_TOKEN_SSM_PATH = local.apim_access_token_ssm_parameter_name
-    USE_NON_MOCK_TOKEN    = var.pdm_use_non_mock_token
-  }
 }
 
 data "aws_iam_policy_document" "pdm_mock" {

infrastructure/terraform/components/dl/module_lambda_pdm_poll.tf

@@ -0,0 +1,103 @@
+module "pdm_poll" {
+  source = "https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/v2.0.24/terraform-lambda.zip"
+
+  function_name = "pdm-poll"
+  description   = "A function for polling PDM document status"
+
+  aws_account_id = var.aws_account_id
+  component      = local.component
+  environment    = var.environment
+  project        = var.project
+  region         = var.region
+  group          = var.group
+
+  log_retention_in_days = var.log_retention_in_days
+  kms_key_arn           = module.kms.key_arn
+
+  iam_policy_document = {
+    body = data.aws_iam_policy_document.pdm_poll_lambda.json
+  }
+
+  function_s3_bucket      = local.acct.s3_buckets["lambda_function_artefacts"]["id"]
+  function_code_base_path = local.aws_lambda_functions_dir_path
+  function_code_dir       = "pdm-poll-lambda/dist"
+  function_include_common = true
+  handler_function_name   = "handler"
+  runtime                 = "nodejs22.x"
+  memory                  = 128
+  timeout                 = 60
+  log_level               = var.log_level
+
+  force_lambda_code_deploy = var.force_lambda_code_deploy
+  enable_lambda_insights   = false
+
+  send_to_firehose          = true
+  log_destination_arn       = local.log_destination_arn
+  log_subscription_role_arn = local.acct.log_subscription_role_arn
+
+  lambda_env_vars = {
+    "APIM_BASE_URL"                        = local.deploy_pdm_mock ? aws_api_gateway_stage.pdm_mock[0].invoke_url : var.apim_base_url
+    "APIM_ACCESS_TOKEN_SSM_PARAMETER_NAME" = local.apim_access_token_ssm_parameter_name
+    "EVENT_PUBLISHER_EVENT_BUS_ARN"        = aws_cloudwatch_event_bus.main.arn
+    "EVENT_PUBLISHER_DLQ_URL"              = module.sqs_event_publisher_errors.sqs_queue_url
+    "POLL_MAX_RETRIES"                     = 10
+  }
+}
+
+data "aws_iam_policy_document" "pdm_poll_lambda" {
+  statement {
+    sid    = "AllowSSMParam"
+    effect = "Allow"
+
+    actions = [
+      "ssm:GetParameter",
+      "ssm:GetParameters",
+      "ssm:GetParametersByPath"
+    ]
+
+    resources = [
+      "arn:aws:ssm:${var.region}:${var.aws_account_id}:parameter/${var.component}/${var.environment}/apim/*"
+    ]
+  }
+  statement {
+    sid    = "PutEvents"
+    effect = "Allow"
+
+    actions = [
+      "events:PutEvents",
+    ]
+
+    resources = [
+      aws_cloudwatch_event_bus.main.arn,
+    ]
+  }
+
+  statement {
+    sid    = "SQSPermissionsDLQs"
+    effect = "Allow"
+
+    actions = [
+      "sqs:SendMessage",
+      "sqs:SendMessageBatch",
+    ]
+
+    resources = [
+      module.sqs_event_publisher_errors.sqs_queue_arn,
+    ]
+  }
+  statement {
+    sid    = "SQSPermissionsPollPdmQueue"
+    effect = "Allow"
+
+    actions = [
+      "sqs:ReceiveMessage",
+      "sqs:DeleteMessage",
+      "sqs:GetQueueAttributes",
+      "sqs:GetQueueUrl",
+    ]
+
+    resources = [
+      module.sqs_pdm_poll.sqs_queue_arn,
+    ]
+  }
+}

infrastructure/terraform/components/dl/module_sqs_pdm_poll.tf

@@ -0,0 +1,35 @@
+module "sqs_pdm_poll" {
+  source = "https://github.com/NHSDigital/nhs-notify-shared-modules/releases/download/v2.0.24/terraform-sqs.zip"
+
+  aws_account_id             = var.aws_account_id
+  component                  = local.component
+  environment                = var.environment
+  project                    = var.project
+  region                     = var.region
+  name                       = "pdm-poll"
+  sqs_kms_key_arn            = module.kms.key_arn
+  visibility_timeout_seconds = 60
+  delay_seconds              = 5
+  create_dlq                 = true
+  sqs_policy_overload        = data.aws_iam_policy_document.sqs_pdm_poll.json
+}
+
+data "aws_iam_policy_document" "sqs_pdm_poll" {
+  statement {
+    sid    = "AllowEventBridgeToSendMessage"
+    effect = "Allow"
+
+    principals {
+      type        = "Service"
+      identifiers = ["events.amazonaws.com"]
+    }
+
+    actions = [
+      "sqs:SendMessage"
+    ]
+
+    resources = [
+      "arn:aws:sqs:${var.region}:${var.aws_account_id}:${local.csi}-pdm-poll-queue"
+    ]
+  }
+}

infrastructure/terraform/components/dl/module_sqs_pdm_uploader.tf

@@ -32,7 +32,7 @@ data "aws_iam_policy_document" "sqs_pdm_uploader" {
     ]
 
     resources = [
-      "arn:aws:sqs:${var.region}:${var.aws_account_id}:${var.project}-${var.environment}-${local.component}-pdm-uploader-queue"
+      "arn:aws:sqs:${var.region}:${var.aws_account_id}:${local.csi}-pdm-uploader-queue"
     ]
   }
 }

infrastructure/terraform/components/dl/module_sqs_ttl.tf

@@ -32,7 +32,7 @@ data "aws_iam_policy_document" "sqs_ttl" {
     ]
 
     resources = [
-      "arn:aws:sqs:${var.region}:${var.aws_account_id}:${var.project}-${var.environment}-${local.component}-ttl-queue"
+      "arn:aws:sqs:${var.region}:${var.aws_account_id}:${local.csi}-ttl-queue"
     ]
   }
 }

infrastructure/terraform/components/dl/variables.tf

@@ -101,7 +101,7 @@ variable "queue_batch_size" {
 variable "queue_batch_window_seconds" {
   type        = number
   description = "maximum time in seconds between processing events"
-  default     = 10
+  default     = 1
 }
 
 variable "enable_dynamodb_delete_protection" {

lambdas/pdm-mock-lambda/README.md

@@ -27,8 +27,8 @@ curl -X POST https://<api-gateway-url>/patient-data-manager/FHIR/R4/DocumentRefe
 
 **Headers:**
 
-- `Authorization: Bearer <token>` - Required authentication token (default: `mock-pdm-token`)
-- `Content-Type: application/fhir+json` - Required content type
+- `Authorization: Bearer <token>` - Authentication token is not validated and can be any string value.
+- `Content-Type: application/fhir+json` - Required content type.
 - `X-Request-ID: <UUID>` - This uuid will be used as the DocumentReference `id` in the response.
 
 **Response (201 Created):**
@@ -76,8 +76,8 @@ curl https://<api-gateway-url>/patient-data-manager/FHIR/R4/DocumentReference/te
 
 **Headers:**
 
-- `Authorization: Bearer <token>` - Required authentication token (default: `mock-pdm-token`)
-- `Content-Type: application/fhir+json` - Required content type
+- `Authorization: Bearer <token>` - Authentication token is not validated and can be any string value.
+- `Content-Type: application/fhir+json` - Required content type.
 - `X-Request-ID: <uuid>` - Used for request tracking and correlation. This isn't part of the ID or response that gets returned.
 
 **Response (200 OK):**
@@ -136,17 +136,18 @@ Both GET and POST endpoints require the `X-Request-ID` header. If it's missing,
 
 The mock API supports triggering specific error responses for testing in both endpoints. Use these special resource IDs:
 
-| Resource ID              | Status Code | Error Code          | Description                     |
-| ------------------------ | ----------- | ------------------- | ------------------------------- |
-| `error-400-invalid`      | 400         | INVALID_VALUE       | Invalid resource value          |
-| `error-401-unauthorized` | 401         | UNAUTHORISED        | Unauthorized access             |
-| `error-403-forbidden`    | 403         | FORBIDDEN           | Access forbidden                |
-| `error-404-notfound`     | 404         | RESOURCE_NOT_FOUND  | Resource not found              |
-| `error-409-conflict`     | 409         | CONFLICT            | Resource already exists         |
-| `error-429-ratelimit`    | 429         | TOO_MANY_REQUESTS   | Rate limit exceeded             |
-| `error-500-internal`     | 500         | INTERNAL_ERROR      | Internal server error           |
-| `error-503-unavailable`  | 503         | SERVICE_UNAVAILABLE | Service temporarily unavailable |
-| `empty-response`         | 200         | -                   | Empty success response          |
+| Resource ID              | Status Code | Error Code          | Description                              |
+| ------------------------ | ----------- | ------------------- | ---------------------------------------- |
+| `error-400-invalid`      | 400         | INVALID_VALUE       | Invalid resource value                   |
+| `error-401-unauthorized` | 401         | UNAUTHORISED        | Unauthorized access                      |
+| `error-403-forbidden`    | 403         | FORBIDDEN           | Access forbidden                         |
+| `error-404-notfound`     | 404         | RESOURCE_NOT_FOUND  | Resource not found                       |
+| `error-409-conflict`     | 409         | CONFLICT            | Resource already exists                  |
+| `error-429-ratelimit`    | 429         | TOO_MANY_REQUESTS   | Rate limit exceeded                      |
+| `error-500-internal`     | 500         | INTERNAL_ERROR      | Internal server error                    |
+| `error-503-unavailable`  | 503         | SERVICE_UNAVAILABLE | Service temporarily unavailable          |
+| `empty-response`         | 200         | -                   | Empty success response                   |
+| `unavailable-response`   | 200         | -                   | Success response with no attachment.data |
 
 **Example - Trigger 404 Error:**
 
@@ -177,9 +178,6 @@ curl https://<api-gateway-url>/resource/error-404-notfound \
 
 The lambda is configured via environment variables:
 
-| Variable                | Description                              | Default                    |
-| ----------------------- | ---------------------------------------- | -------------------------- |
-| `MOCK_ACCESS_TOKEN`     | Token to use in local/dev environments   | `mock-pdm-token`           |
-| `ACCESS_TOKEN_SSM_PATH` | SSM parameter path for the access token  | `/dl/main/apim/access_token`|
-| `USE_NON_MOCK_TOKEN`    | Use SSM token instead of mock token      | `false`                    |
-| `LOG_LEVEL`             | Logging level (DEBUG, INFO, WARN, ERROR) | `INFO`                     |
+| Variable                | Description                              | Default                  |
+| ----------------------- | ---------------------------------------- | ------------------------ |
+| `LOG_LEVEL`             | Logging level (DEBUG, INFO, WARN, ERROR) | `INFO`                   |