Skip to content

Commit 5e22d5d

Browse files
dlzhry2nhsdlzhry2nhs
authored
VED-000 README documentation updates and removed obsolete dev tools (#1181)
1 parent b1fce9c commit 5e22d5d

20 files changed

Lines changed: 53 additions & 895 deletions

‎.gitignore‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,6 @@ openapi.json
2929
**/.vscode/**/*
3030
!**/.vscode/settings.json.default
3131

32-
devtools/volume/
3332
**/.coverage
3433
**/test-results.xml
3534
allure-results/

‎README.md‎

Lines changed: 39 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -14,39 +14,49 @@ See https://nhsd-confluence.digital.nhs.uk/display/APM/Glossary.
1414

1515
**Note:** Each Lambda has its own `README.md` file for detailed documentation. For non-Lambda-specific folders, refer to `README.specification.md`.
1616

17-
### Lambdas
18-
19-
| Folder | Description |
20-
| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
21-
| `backend` | **Imms API** – Handles CRUD operations for the Immunisation API. |
22-
| `delta_backend` | **Imms Sync** – Lambda function that reacts to events in the Immunisation database. |
23-
| `ack_backend` | **Imms Batch** – Generates the final Business Acknowledgment (BUSACK) file from processed messages and writes it to the designated S3 location. |
24-
| `filenameprocessor` | **Imms Batch** – Processes batch file names. |
25-
| `mesh_processor` | **Imms Batch** – MESH-specific batch processing functionality. |
26-
| `recordprocessor` | **Imms Batch** – Handles batch record processing. |
27-
| `redis_sync` | **Imms Redis** – Handles sync s3 to REDIS. |
28-
| `id_sync` | **Imms Redis** – Handles sync SQS to IEDS. |
29-
| `shared` | **Imms Redis** – Not a lambda but Shared Code for lambdas |
17+
### Lambdas (compute microservices)
18+
19+
| Folder | Description |
20+
| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
21+
| `ack_backend` | **Imms Batch** – Generates the final Business Acknowledgment (BUSACK) file from processed messages and writes it to the designated destination. |
22+
| `backend` | **Imms API** – Handles CRUD operations for the Immunisation API. |
23+
| `batch_processor_filter` | **Imms Batch** – Controller function that consumes from a queue and forwards file event for processing if filter conditions are met. |
24+
| `delta_backend` | **Imms Sync** – Lambda function that reacts to events in the Immunisation Event Data Store (IEDS). |
25+
| `filenameprocessor` | **Imms Batch** – Validates and processes new batch file events. |
26+
| `id_sync` | **Imms Cross-cutting** – Handles [MNS](https://digital.nhs.uk/developer/api-catalogue/multicast-notification-service) NHS Number Change events and applies updates to affected records. |
27+
| `mesh_processor` | **Imms Batch** – Triggered when new files are received via MESH. Moves them into the Imms Batch processing system. |
28+
| `mns_subscription` | **Imms Cross-cutting** – Simple helper Lambda which sets up our required MNS subscription. Used in pipelines in DEV. |
29+
| `recordforwarder` | **Imms Batch** – Consumes from the stream and applies the processed batch file row operations (CUD) to IEDS. |
30+
| `recordprocessor` | **Imms Batch** – ECS Task - **not** a Lambda function - responsible for processing batch file rows and forwarding to the stream. |
31+
| `redis_sync` | **Imms Cross-cutting** – Handles config file updates. E.g. disease mapping or permission files. |
32+
| `shared` | **Imms Cross-cutting** – Shared `common` code that can be shared by other Lambda functions. This is not a standalone Lambda. |
3033

3134
---
3235

3336
### Pipelines
3437

35-
| Folder | Description |
36-
| ------- | ------------------------------------------- |
37-
| `azure` | Pipeline definition and orchestration code. |
38+
Due to the timing of the project's inception, Azure pipelines have been inherited from the API Management team for deploying
39+
the Apigee proxy and sandbox. The new way to manage and deploy said resources is [Proxygen](https://digital.nhs.uk/developer/api-catalogue/proxy-generator).
40+
41+
In future a migration plan will be provided by the API Management team so we can move to the new process and use purely
42+
GitHub Actions for our entire pipeline.
43+
44+
| Folder | Description |
45+
| --------- | ------------------------------------------------------------------------------------------------------------------------------- |
46+
| `azure` | Pipeline definition and orchestration code concerned purely with the management and deployment of the Apigee proxy and sandbox. |
47+
| `.github` | Pipeline definition and orchestration code for deploying and testing our backend project. |
3848

3949
---
4050

4151
### Infrastructure
4252

43-
| Folder | Description |
44-
| ---------------------- | ------------------------------------------------------------------------------------------------------ |
45-
| `account` | Base infrastructure components. |
46-
| `grafana` | Terraform configuration for Grafana, built on top of core infra. |
47-
| `instance` | Core Terraform infrastructure code. This is run in each PR and sets up lambdas associated with the PR. |
48-
| `terraform_aws_backup` | Streamlined backup processing with AWS. |
49-
| `proxies` | Apigee API proxy definitions. |
53+
| Folder | Description |
54+
| ---------------------- | ---------------------------------------------------------------- |
55+
| `account` | Base infrastructure components deployed on a per account basis. |
56+
| `grafana` | Terraform configuration for Grafana, built on top of core infra. |
57+
| `instance` | Core Terraform app infrastructure. |
58+
| `terraform_aws_backup` | Streamlined backup processing with AWS. |
59+
| `proxies` | Apigee API proxy definitions. |
5060

5161
---
5262

@@ -60,13 +70,12 @@ See https://nhsd-confluence.digital.nhs.uk/display/APM/Glossary.
6070

6171
### Utilities
6272

63-
| Folder | Description |
64-
| ---------------- | ------------------------------------------------------------- |
65-
| `devtools` | Helper tools and utilities for local development |
66-
| `quality_checks` | Dependencies for linting and formatting Python code |
67-
| `scripts` | Standalone or reusable scripts for development and automation |
68-
| `specification` | Specification files to document API and related definitions |
69-
| `sandbox` | Simple sandbox API |
73+
| Folder | Description |
74+
| ------------------- | ------------------------------------------------------------- |
75+
| `quality_checks` | Dependencies for linting and formatting Python code |
76+
| `utilities/scripts` | Standalone or reusable scripts for development and automation |
77+
| `specification` | Specification files to document API and related definitions |
78+
| `sandbox` | Simple sandbox API |
7079

7180
---
7281

@@ -268,11 +277,3 @@ run a different set of tests. To do this:
268277
Please note that this project requires that all commits are verified using a GPG key.
269278
To set up a GPG key please follow the instructions specified here:
270279
https://docs.github.com/en/authentication/managing-commit-signature-verification
271-
272-
## AWS configuration: Getting credentials for AWS federated user account
273-
274-
In the 'Access keys' popup menu under AWS Access Portal:
275-
276-
**NOTE** that AWS's 'Recommended' method of getting credentials **(AWS IAM Identity Center credentials)** will break mocking in unit tests; specifically any tests calling `dynamodb_client.create_table()` will fail with `botocore.errorfactory.ResourceInUseException: Table already exists`.
277-
278-
Instead, use **Option 2 (Add a profile to your AWS credentials file)**.

‎lambdas/delta_backend/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,9 +8,9 @@ This project is designed to convert FHIR-compliant JSON data (e.g., Immunization
88

99
| File Name | What It Does |
1010
| --------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
11-
| **`converter.py`** | 🧠 The main brain — applies the schema, runs conversions, handles errors. |
11+
| **`converter.py`** | The main brain — applies the schema, runs conversions, handles errors. |
1212
| **`conversion_layout.py`** | A plain Python list that defines which fields you want, and how they should be formatted (e.g. date format, renaming rules). |
13-
| **`delta.py`** | Holds the function called by AWS Lambda |
13+
| **`delta.py`** | Holds the function called by AWS Lambda. |
1414
| **`extractor.py`** | Tailored functionality to extract target fields from immunization record received by the delta handler. |
1515
| **`exception_messages.py`** | Holds reusable error messages and codes for clean debugging and validation feedback. |
1616
| **`log_firehose.py`** | Firehose logging functionality. |

‎lambdas/id_sync/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,12 @@
22

33
## Overview
44

5-
**Id Sync** is an AWS Lambda function designed to trigger from SQS. It receives a list of NHS Numbers and checks for changes in PDS. If change found, it updates the Events Table..
5+
**Id Sync** is an AWS Lambda function designed to trigger from SQS. It receives a list of NHS Numbers and checks for changes in PDS. If change found, it updates the Events Table.
66

77
## Features
88

99
- **SQS Event Driven:** Automatically triggered by SQS event.
10-
- **DynamoDb Integration:** Reviews contents of DynbamoDb Events table and updates where required..
10+
- **DynamoDb Integration:** Reviews contents of DynamoDb Events table and updates where required.
1111
- **Logging:** Provides detailed logging for monitoring and troubleshooting.
1212

1313
## How It Works

‎lambdas/recordforwarder/README.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,14 @@
11
# About
22

3-
This document describes the environment setup for the recordforwarder Lambda.
3+
This document describes the purpose and environment setup for the recordforwarder Lambda.
4+
5+
## Overview
6+
7+
The Record Forwarder Lambda consumes from an AWS Kinesis Stream, and is responsible for applying updates to the Immunisation
8+
Event Data Store. It will receive up to 100 records per batch. Each record will have been processed and formatted as a
9+
FHIR Immunization by the ECS Record Processor in the prior batch step.
10+
11+
The Record Forwarder Lambda will then execute the requested operation (Create, Update or Delete) on the given record.
412

513
## Setting up the recordforwarder lambda
614

‎sonar-project.properties‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ sonar.projectKey=NHSDigital_immunisation-fhir-api
33
sonar.organization=nhsdigital
44
sonar.host.url=https://sonarcloud.io
55
sonar.python.version=3.11
6-
sonar.exclusions=**/devtools/**,**/proxies/**,**/utilities/scripts/**,**/infrastructure/account/**,**/infrastructure/instance/**,**/infrastructure/grafana/**,**/terraform_aws_backup/**,**/tests/**
6+
sonar.exclusions=**/proxies/**,**/utilities/scripts/**,**/infrastructure/account/**,**/infrastructure/instance/**,**/infrastructure/grafana/**,**/terraform_aws_backup/**,**/tests/**
77
sonar.coverage.exclusions=lambdas/shared/src/common/models/batch_constants.py
88
sonar.python.coverage.reportPaths=backend-coverage.xml,delta-coverage.xml,ack-lambda-coverage.xml,filenameprocessor-coverage.xml,recordforwarder-coverage.xml,recordprocessor-coverage.xml,mesh_processor-coverage.xml,redis_sync-coverage.xml,mns_subscription-coverage.xml,id_sync-coverage.xml,shared-coverage.xml,batchprocessorfilter-coverage.xml
99
sonar.cpd.exclusions=**/Dockerfile

‎utilities/devtools/.gitignore‎

Lines changed: 0 additions & 4 deletions
This file was deleted.

‎utilities/devtools/.terraform.lock.hcl‎

Lines changed: 0 additions & 10 deletions
This file was deleted.

‎utilities/devtools/Makefile‎

Lines changed: 0 additions & 20 deletions
This file was deleted.

0 commit comments

Comments
 (0)