From 091c55f0471770bae09659687c567aaf15eb6454 Mon Sep 17 00:00:00 2001 From: Gareth Allan <157592212+gareth-allan@users.noreply.github.com> Date: Fri, 12 Dec 2025 10:17:05 +0000 Subject: [PATCH 01/20] CCM-12896: Initial version of python type generation --- .gitignore | 12 ++ Makefile | 2 + .../config/vocabularies/words/accept.txt | 1 + src/digital-letters-events-python/.gitignore | 2 + src/digital-letters-events-python/README.md | 66 ++++++++ src/digital-letters-events-python/__init__.py | 13 ++ .../requirements.txt | 1 + src/pydantic-model-generator/Makefile | 70 +++++++++ src/pydantic-model-generator/README.md | 108 +++++++++++++ src/pydantic-model-generator/pytest.ini | 21 +++ .../requirements-dev.txt | 4 + src/pydantic-model-generator/requirements.txt | 3 + .../scripts/__init__.py | 1 + .../scripts/file_utils.py | 82 ++++++++++ .../scripts/generate_models.py | 90 +++++++++++ .../scripts/model_generator.py | 55 +++++++ .../scripts/schema_processor.py | 39 +++++ .../tests/__init__.py | 1 + .../tests/test_file_utils.py | 144 ++++++++++++++++++ .../tests/test_model_generator.py | 45 ++++++ .../tests/test_schema_processor.py | 67 ++++++++ 21 files changed, 827 insertions(+) create mode 100644 src/digital-letters-events-python/.gitignore create mode 100644 src/digital-letters-events-python/README.md create mode 100644 src/digital-letters-events-python/__init__.py create mode 100644 src/digital-letters-events-python/requirements.txt create mode 100644 src/pydantic-model-generator/Makefile create mode 100644 src/pydantic-model-generator/README.md create mode 100644 src/pydantic-model-generator/pytest.ini create mode 100644 src/pydantic-model-generator/requirements-dev.txt create mode 100644 src/pydantic-model-generator/requirements.txt create mode 100644 src/pydantic-model-generator/scripts/__init__.py create mode 100644 src/pydantic-model-generator/scripts/file_utils.py create mode 100644 src/pydantic-model-generator/scripts/generate_models.py create mode 100644 src/pydantic-model-generator/scripts/model_generator.py create mode 100644 src/pydantic-model-generator/scripts/schema_processor.py create mode 100644 src/pydantic-model-generator/tests/__init__.py create mode 100644 src/pydantic-model-generator/tests/test_file_utils.py create mode 100644 src/pydantic-model-generator/tests/test_model_generator.py create mode 100644 src/pydantic-model-generator/tests/test_schema_processor.py diff --git a/.gitignore b/.gitignore index 25e2dcc5a..3f24d00e8 100644 --- a/.gitignore +++ b/.gitignore @@ -26,6 +26,18 @@ output /schemas .env +# Python +__pycache__/ +.Python +.venv/ +venv/ + +# Testing +.pytest_cache/ +.coverage +htmlcov/ +coverage.xml + # Coverage reports htmlcov/ .coverage diff --git a/Makefile b/Makefile index 206c0b9d7..89c5a3981 100644 --- a/Makefile +++ b/Makefile @@ -14,6 +14,7 @@ dependencies: # Install dependencies needed to build and test the project @Pipel generate: # Generate any autogenerated output @Pipeline npm run generate-dependencies + $(MAKE) -C src/pydantic-model-generator generate build: # Build the project artefact @Pipeline $(MAKE) -C docs build @@ -32,6 +33,7 @@ clean:: # Clean-up project resources (main) @Operations $(MAKE) -C src/cloudevents clean $(MAKE) -C src/eventcatalogasyncapiimporter clean $(MAKE) -C src/eventcatalogasyncapiimporter clean-output + $(MAKE) -C src/pydantic-model-generator clean rm -f .version npm run clean diff --git a/scripts/config/vale/styles/config/vocabularies/words/accept.txt b/scripts/config/vale/styles/config/vocabularies/words/accept.txt index cf79926a7..d702582fc 100644 --- a/scripts/config/vale/styles/config/vocabularies/words/accept.txt +++ b/scripts/config/vale/styles/config/vocabularies/words/accept.txt @@ -69,6 +69,7 @@ Podman producedby projectRoot Protobuf +Pydantic pylint Python quotingType diff --git a/src/digital-letters-events-python/.gitignore b/src/digital-letters-events-python/.gitignore new file mode 100644 index 000000000..38417dc64 --- /dev/null +++ b/src/digital-letters-events-python/.gitignore @@ -0,0 +1,2 @@ +# Models directory (auto-generated) +models/ diff --git a/src/digital-letters-events-python/README.md b/src/digital-letters-events-python/README.md new file mode 100644 index 000000000..28dcac606 --- /dev/null +++ b/src/digital-letters-events-python/README.md @@ -0,0 +1,66 @@ + + +# digital-letters-events-python + + + + +This package contains the automatically-generated Pydantic v2 models that the [pydantic-model-generator](../pydantic-model-generator/) tool produces. + + +The source files in this package should not be edited directly. If changes are required, update the schemas in the [schemas/digital-letters/2025-10-draft/events](../../schemas/digital-letters/2025-10-draft/events) directory and use the `pydantic-model-generator` tool to regenerate them. + +## Using this Package + +### Using Event Models + +The Pydantic models can be used by installing the `digital-letters-events-python` package and importing the desired model: + +```python +from digital_letters_events_python import PDMResourceSubmitted + +try: + # Validate and parse an event + event_data = { + "type": "uk.nhs.notify.digital.letters.pdm.resource.submitted.v1", + "source": "/nhs/england/notify/staging/dev-647563337/data-plane/digitalletters/pdm", + "dataschema": "https://notify.nhs.uk/cloudevents/schemas/digital-letters/2025-10-draft/data/digital-letters-pdm-resource-submitted-data.schema.json", + "specversion": "1.0", + "id": "0249e529-f947-4012-819e-b634eb71be79", + "subject": "pdm-resource-123", + "time": "2025-12-11T10:00:00Z", + "data": { + "something": "example value" + } + } + + # Create and validate the event + event = PDMResourceSubmitted(**event_data) + + # Access validated fields + print(event.id) + print(event.type) + print(event.data.something) +except Exception as e: + print(e) +``` + +### Type Safety and Validation + +All generated models include: + +- Type hints for all fields +- Runtime validation via Pydantic +- Automatic conversion of types where appropriate +- Clear error messages for validation failures + +## Development + +This package is automatically generated. Do not edit the files in the [`models`](./models/) directory directly. + +To regenerate the models: + +```bash +cd ../pydantic-model-generator +make generate +``` diff --git a/src/digital-letters-events-python/__init__.py b/src/digital-letters-events-python/__init__.py new file mode 100644 index 000000000..7d097b631 --- /dev/null +++ b/src/digital-letters-events-python/__init__.py @@ -0,0 +1,13 @@ +"""Digital Letters Events package. + +This package contains automatically-generated Pydantic v2 models for NHS Notify +Digital Letters events. These models are generated from JSON schemas by the +pydantic-model-generator tool. + +DO NOT EDIT: Files in this package are auto-generated. Any manual changes will +be overwritten when the models are regenerated. +""" + +from .models import * + +__all__ = ["models"] diff --git a/src/digital-letters-events-python/requirements.txt b/src/digital-letters-events-python/requirements.txt new file mode 100644 index 000000000..84f73d017 --- /dev/null +++ b/src/digital-letters-events-python/requirements.txt @@ -0,0 +1 @@ +pydantic>=2.0.0,<3.0.0 diff --git a/src/pydantic-model-generator/Makefile b/src/pydantic-model-generator/Makefile new file mode 100644 index 000000000..b24c074d7 --- /dev/null +++ b/src/pydantic-model-generator/Makefile @@ -0,0 +1,70 @@ +# Variables +SCHEMA_SRC_DIR := ../../schemas/digital-letters/2025-10-draft/events +OUTPUT_DIR := ../digital-letters-events-python/models +SCRIPTS_DIR := scripts + +# Python settings +VENV_DIR := ../../.venv +VENV_PYTHON := $(VENV_DIR)/bin/python +VENV_PIP := $(VENV_DIR)/bin/pip + +# Default target +.PHONY: all clean generate install install-dev test coverage help + +all: generate + +# Install production dependencies (uses repo-level venv) +install: + @echo "Installing production dependencies..." + @$(VENV_PIP) install -r requirements.txt + @echo "Production dependencies installed!" + +# Install development dependencies (uses repo-level venv) +install-dev: + @echo "Installing development dependencies..." + @$(VENV_PIP) install -r requirements-dev.txt + @echo "Development dependencies installed!" + +# Generate Pydantic models from JSON schemas +generate: install + @echo "Generating Pydantic models from JSON schemas..." + @mkdir -p $(OUTPUT_DIR) + @$(VENV_PYTHON) $(SCRIPTS_DIR)/generate_models.py \ + --input-dir $(SCHEMA_SRC_DIR) \ + --output-dir $(OUTPUT_DIR) + @echo "Pydantic models generated in $(OUTPUT_DIR)/" + +# Run tests +test: install-dev + @echo "Running tests..." + @cd ../.. && pytest src/pydantic-model-generator/tests/ + +# Generate coverage report +coverage: install-dev + @echo "Generating coverage report..." + @cd ../.. && pytest src/pydantic-model-generator/tests/ \ + --cov=src/pydantic-model-generator \ + --cov-config=src/pydantic-model-generator/pytest.ini \ + --cov-report=html:src/pydantic-model-generator/htmlcov \ + --cov-report=term-missing \ + --cov-report=xml:src/pydantic-model-generator/coverage.xml \ + --cov-branch + @echo "Coverage report generated in htmlcov/" + +# Clean output directory and generated files +clean: + @echo "Cleaning generated models..." + @rm -rf $(OUTPUT_DIR) htmlcov .pytest_cache coverage.xml + @echo "Clean complete!" + +# Show help +help: + @echo "Available targets:" + @echo " all - Generate Pydantic models (default)" + @echo " install - Install production dependencies" + @echo " install-dev - Install development dependencies" + @echo " generate - Generate Pydantic models from JSON schemas" + @echo " test - Run tests" + @echo " coverage - Generate coverage report" + @echo " clean - Clean output directories" + @echo " help - Show this help message" diff --git a/src/pydantic-model-generator/README.md b/src/pydantic-model-generator/README.md new file mode 100644 index 000000000..99e363ca0 --- /dev/null +++ b/src/pydantic-model-generator/README.md @@ -0,0 +1,108 @@ + +# pydantic-model-generator + + +This package provides a tool that generates Pydantic v2 models for the events +that are defined in the JSON schemas that are built as part of this repository. +The generated Pydantic models can be used to validate and parse NHS Notify +Digital Letters events. + +## Setup + +Use `make install-dev` to install the necessary dependencies for this application. + +## Generating Models + +In order for this tool to function, you must first build the JSON schemas for +the Digital Letters events. The simplest way to build these schemas is to run +the `npm run generate-dependencies` command from the root of this repository. + +### Using the Generator + +Once the JSON schemas have been built, Pydantic models can be generated by running: + +```bash +make generate +``` + +This will: + +- Read all JSON event schemas from `schemas/digital-letters/2025-10-draft/events/` +- Generate Pydantic v2 models using `datamodel-code-generator` +- Output the models to `../digital-letters-events-python/models/` + +### Configuration + +The generator can be configured through: + +- Command-line arguments +- Environment variables +- Configuration file (if needed) + +### Output Structure + + + +Generated models are placed in the +[digital-letters-events-python](../digital-letters-events-python/) package: + + + +- `../digital-letters-events-python/models/` - Individual Pydantic model files +- `../digital-letters-events-python/models/__init__.py` - Combined index for easy imports + +## Testing + +Run tests using: + +```bash +make test +``` + +For coverage reporting: + +```bash +make coverage +``` + +## Development + +### Prerequisites + +- Python 3.11+ +- pip or poetry for dependency management + +### Installing Development Dependencies + +```bash +make install-dev +``` + +### Running Tests + +```bash +make test +``` + +## Usage + +Once generated, the models can be used in your code as follows: + +```python +from digital_letters_event_python import PrintLetterAvailable, CloudEventEnvelope + +# Validate incoming event +event_data = CloudEventEnvelope(**raw_event) +if event_data.type == "uk.nhs.notify.digital.letters.letter.available.v1": + letter_available = PrintLetterAvailable(**raw_event) + # Process the validated event +``` + +## Architecture + +The generator consists of: + +- `scripts/generate_models.py` - Main CLI script +- `scripts/schema_processor.py` - Schema loading and processing +- `scripts/model_generator.py` - Pydantic model generation logic +- `scripts/file_utils.py` - File system utilities diff --git a/src/pydantic-model-generator/pytest.ini b/src/pydantic-model-generator/pytest.ini new file mode 100644 index 000000000..0d63d6be4 --- /dev/null +++ b/src/pydantic-model-generator/pytest.ini @@ -0,0 +1,21 @@ +[pytest] +testpaths = tests +python_files = test_*.py +python_classes = Test* +python_functions = test_* +addopts = + -v + --tb=short + +[coverage:run] +relative_files = True +omit = + */tests/* + */test_*.py + test_*.py + */venv/* + */.venv/* + */__pycache__/* + +[coverage:xml] +output = coverage.xml diff --git a/src/pydantic-model-generator/requirements-dev.txt b/src/pydantic-model-generator/requirements-dev.txt new file mode 100644 index 000000000..248f1be10 --- /dev/null +++ b/src/pydantic-model-generator/requirements-dev.txt @@ -0,0 +1,4 @@ +-r requirements.txt +pytest>=7.4.0 +pytest-cov>=4.1.0 +pytest-mock>=3.11.0 diff --git a/src/pydantic-model-generator/requirements.txt b/src/pydantic-model-generator/requirements.txt new file mode 100644 index 000000000..35a70ce5f --- /dev/null +++ b/src/pydantic-model-generator/requirements.txt @@ -0,0 +1,3 @@ +datamodel-code-generator>=0.25.0,<1.0.0 +pydantic>=2.0.0,<3.0.0 +PyYAML>=6.0 diff --git a/src/pydantic-model-generator/scripts/__init__.py b/src/pydantic-model-generator/scripts/__init__.py new file mode 100644 index 000000000..f85e0e43a --- /dev/null +++ b/src/pydantic-model-generator/scripts/__init__.py @@ -0,0 +1 @@ +"""Pydantic model generator package.""" diff --git a/src/pydantic-model-generator/scripts/file_utils.py b/src/pydantic-model-generator/scripts/file_utils.py new file mode 100644 index 000000000..d63674b8c --- /dev/null +++ b/src/pydantic-model-generator/scripts/file_utils.py @@ -0,0 +1,82 @@ +"""File utilities for the Pydantic model generator.""" + +import json +from pathlib import Path +from typing import Any +import re + + + +def list_json_schemas(schema_dir: str) -> list[str]: + """List all flattened JSON schema files in the given directory. + + Args: + schema_dir: Directory containing JSON schema files + + Returns: + List of JSON schema filenames + """ + schema_path = Path(schema_dir) + if not schema_path.exists(): + raise FileNotFoundError(f"Schema directory not found: {schema_dir}") + + flattened_schema_files = sorted(schema_path.glob("*.flattened.schema.json")) + return [f.name for f in flattened_schema_files] + + +def load_json_schema(schema_path: str) -> dict[str, Any]: + """Load a JSON schema from file. + + Args: + schema_path: Path to the JSON schema file + + Returns: + Loaded schema as dictionary + """ + with open(schema_path, encoding="utf-8") as f: + return json.load(f) + + +def write_init_file(output_dir: str, model_names: list[str]) -> None: + """Write __init__.py file with exports for all generated models. + + Args: + output_dir: Directory to write __init__.py to + model_names: List of model class names to export + """ + init_content = '"""Generated Pydantic models for NHS Notify Digital Letters events."""\n\n' + + for model_name in sorted(model_names): + module_name = model_name_to_module_name(model_name) + init_content += f"from .{module_name} import {model_name}\n" + + init_content += "\n__all__ = [\n" + for model_name in sorted(model_names): + init_content += f' "{model_name}",\n' + init_content += "]\n" + + output_path = Path(output_dir) / "__init__.py" + with open(output_path, "w", encoding="utf-8") as f: + f.write(init_content) + + +def model_name_to_module_name(model_name: str) -> str: + """Convert a model class name to a module name. + + Args: + model_name: The model class name (e.g., 'PrintLetterAvailable') + + Returns: + Module name (e.g., 'print_letter_available') + """ + if not model_name: + return "" + + # Handle acronym boundaries like "JSONSchema" -> "JSON_Schema" + step1 = re.sub(r"([A-Z]+)([A-Z][a-z])", r"\1_\2", model_name) + + # Insert underscores between lowercase/digit followed by uppercase: "fooBar" -> "foo_Bar" + step2 = re.sub(r"([a-z0-9])([A-Z])", r"\1_\2", step1) + + # Normalise hyphens to underscores and lower-case the result + return step2.replace("-", "_").lower() diff --git a/src/pydantic-model-generator/scripts/generate_models.py b/src/pydantic-model-generator/scripts/generate_models.py new file mode 100644 index 000000000..66bdf9b55 --- /dev/null +++ b/src/pydantic-model-generator/scripts/generate_models.py @@ -0,0 +1,90 @@ +#!/usr/bin/env python3 +"""CLI tool to generate Pydantic v2 models from JSON schemas. + +This tool reads JSON schemas for NHS Notify Digital Letters events and +generates corresponding Pydantic v2 models that can be used by Python +lambda functions for event validation and parsing. +""" + +import argparse +import sys +from pathlib import Path + +from file_utils import ( + list_json_schemas, + load_json_schema, + model_name_to_module_name, + write_init_file, +) +from model_generator import generate_pydantic_model +from schema_processor import ( + extract_model_name, +) + + +def parse_args() -> argparse.Namespace: + """Parse command-line arguments. + + Returns: + Parsed arguments + """ + parser = argparse.ArgumentParser( + ) + parser.add_argument( + "--input-dir", + type=str, + required=True, + help="Directory containing JSON schema files", + ) + parser.add_argument( + "--output-dir", + type=str, + required=True, + help="Directory to write generated Pydantic models (must exist)", + ) + return parser.parse_args() + + +def main() -> int: + """Main entry point for the generator. + + Returns: + Exit code (0 for success, 1 for failure) + """ + args = parse_args() + + try: + print(f"Generating models in: {args.output_dir}") + + schema_filenames = list_json_schemas(args.input_dir) + print(f"Found {len(schema_filenames)} schema files") + + generated_models = [] + for schema_filename in schema_filenames: + schema_path = str(Path(args.input_dir) / schema_filename) + schema = load_json_schema(schema_path) + + model_name = extract_model_name(schema) + output_filename = model_name_to_module_name(model_name) + ".py" + output_file_path = str(Path(args.output_dir) / output_filename) + + generate_pydantic_model( + schema_path, output_file_path, model_name + ) + + generated_models.append(model_name) + print(f" ✓ {output_filename}") + + write_init_file(args.output_dir, generated_models) + print(f" ✓ __init__.py") + + print(f"\nGeneration complete! Created {len(generated_models)} models ") + return 0 + + except Exception as e: + print(f"Error: {e}", file=sys.stderr) + return 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/src/pydantic-model-generator/scripts/model_generator.py b/src/pydantic-model-generator/scripts/model_generator.py new file mode 100644 index 000000000..45c6ba528 --- /dev/null +++ b/src/pydantic-model-generator/scripts/model_generator.py @@ -0,0 +1,55 @@ +"""Model generator using datamodel-code-generator.""" + +import subprocess +import sys +from pathlib import Path + + + +def generate_pydantic_model( + schema_path: str, output_file: str, class_name: str +) -> None: + """Generate a Pydantic model from a JSON schema. + + Args: + schema_path: The path to the JSON schema file + output_file: Path where the model should be written + class_name: Name for the generated Pydantic model class + + Raises: + RuntimeError: If model generation fails + """ + datamodel_cmd = str(Path(sys.executable).parent / "datamodel-codegen") + cmd = [ + datamodel_cmd, + "--input", + schema_path, + "--output", + output_file, + "--class-name", + class_name, + "--input-file-type", + "jsonschema", + "--output-model-type", + "pydantic_v2.BaseModel", + "--use-schema-description", + "--field-constraints", + "--strict-nullable", + "--snake-case-field", + "--target-python-version", + "3.13", + "--custom-file-header", + '''"""Generated Pydantic model for NHS Notify Digital Letters events. + +This file is auto-generated. Do not edit manually. +""" + +''' + ] + result = subprocess.run( + cmd, capture_output=True, text=True, check=False, encoding="utf-8" + ) + + if result.returncode != 0: + error_msg = f"Failed to generate model: {result.stderr}" + raise RuntimeError(error_msg) diff --git a/src/pydantic-model-generator/scripts/schema_processor.py b/src/pydantic-model-generator/scripts/schema_processor.py new file mode 100644 index 000000000..8f784ec89 --- /dev/null +++ b/src/pydantic-model-generator/scripts/schema_processor.py @@ -0,0 +1,39 @@ +"""Schema processor for extracting information from JSON schemas.""" + +from typing import Any +import re + + + +def extract_model_name(schema: dict[str, Any]) -> str: + """Extract the model name from a schema. + + Args: + schema: The JSON schema dictionary + + Returns: + Model name extracted from the schema's 'title' field + """ + title = schema.get("title") + + if not title: + raise ValueError("Schema does not contain a 'title' field") + + # Sanitize model name by removing spaces and special characters + sanitized_name = re.sub(r'[^a-zA-Z0-9_]', '', title) + + return sanitized_name + + +def extract_event_type(schema: dict[str, Any]) -> str | None: + """Extract the event type from a schema if available. + + Args: + schema: The JSON schema dictionary + + Returns: + Event type string or None if not found + """ + properties = schema.get("properties", {}) + type_prop = properties.get("type", {}) + return type_prop.get("const") diff --git a/src/pydantic-model-generator/tests/__init__.py b/src/pydantic-model-generator/tests/__init__.py new file mode 100644 index 000000000..702505532 --- /dev/null +++ b/src/pydantic-model-generator/tests/__init__.py @@ -0,0 +1 @@ +"""Pydantic model generator test package.""" diff --git a/src/pydantic-model-generator/tests/test_file_utils.py b/src/pydantic-model-generator/tests/test_file_utils.py new file mode 100644 index 000000000..04155834b --- /dev/null +++ b/src/pydantic-model-generator/tests/test_file_utils.py @@ -0,0 +1,144 @@ +"""Tests for file_utils module.""" + +import json + +import pytest + +from scripts.file_utils import ( + list_json_schemas, + load_json_schema, + write_init_file, + model_name_to_module_name +) + + +class TestListJsonSchemas: + """Tests for list_json_schemas function.""" + + def test_lists_json_schema_files(self, tmp_path): + """Test that it finds JSON schema files.""" + (tmp_path / "test1.flattened.schema.json").write_text("{}") + (tmp_path / "test2.flattened.schema.json").write_text("{}") + (tmp_path / "not-flattened.schema.json").write_text("{}") + (tmp_path / "not-a-schema.json").write_text("{}") + + result = list_json_schemas(str(tmp_path)) + + assert len(result) == 2 + assert "test1.flattened.schema.json" in result + assert "test2.flattened.schema.json" in result + assert "not-flattened.schema.json" not in result + assert "not-a-schema.json" not in result + + def test_returns_sorted_list(self, tmp_path): + """Test that results are sorted.""" + (tmp_path / "zebra.flattened.schema.json").write_text("{}") + (tmp_path / "alpha.flattened.schema.json").write_text("{}") + + result = list_json_schemas(str(tmp_path)) + + assert result == ["alpha.flattened.schema.json", "zebra.flattened.schema.json"] + + def test_raises_error_if_directory_not_found(self): + """Test that it raises FileNotFoundError for missing directory.""" + with pytest.raises(FileNotFoundError): + list_json_schemas("/nonexistent/path") + + +class TestLoadJsonSchema: + """Tests for load_json_schema function.""" + + def test_loads_valid_json_schema(self, tmp_path): + """Test loading a valid JSON schema.""" + schema_file = tmp_path / "test.schema.json" + schema_content = {"title": "TestSchema", "type": "object"} + schema_file.write_text(json.dumps(schema_content)) + + result = load_json_schema(str(schema_file)) + + assert result == schema_content + + def test_raises_error_for_invalid_json(self, tmp_path): + """Test that it raises error for invalid JSON.""" + schema_file = tmp_path / "invalid.json" + schema_file.write_text("not valid json") + + with pytest.raises(json.JSONDecodeError): + load_json_schema(str(schema_file)) + + +class TestWriteInitFile: + """Tests for write_init_file function.""" + + def test_writes_init_file_with_imports(self, tmp_path): + """Test writing __init__.py with model imports.""" + model_names = ["ModelA", "ModelB", "ModelC"] + + write_init_file(str(tmp_path), model_names) + + init_file = tmp_path / "__init__.py" + assert init_file.exists() + + content = init_file.read_text() + assert "from .model_a import ModelA" in content + assert "from .model_b import ModelB" in content + assert "from .model_c import ModelC" in content + assert '__all__ = [' in content + assert '"ModelA",' in content + assert '"ModelB",' in content + assert '"ModelC",' in content + assert ']' in content + + def test_sorts_model_names(self, tmp_path): + """Test that model names are sorted in __init__.py.""" + model_names = ["ZebraModel", "AlphaModel"] + + write_init_file(str(tmp_path), model_names) + + init_file = tmp_path / "__init__.py" + assert init_file.exists() + + content = init_file.read_text() + alpha_pos = content.index("AlphaModel") + zebra_pos = content.index("ZebraModel") + assert alpha_pos < zebra_pos + + +class TestModelNameToModuleName: + """Tests for model_name_to_module_name function.""" + + def test_converts_pascal_case_to_snake_case(self): + """Test converting model name to snake_case filename.""" + result = model_name_to_module_name("PrintLetterAvailable") + + assert result == "print_letter_available" + + def test_handles_consecutive_capitals(self): + """Test handling consecutive capital letters.""" + result = model_name_to_module_name("PDFDocument") + + assert result == "pdf_document" + + def test_handles_single_word(self): + """Test handling single word titles.""" + result = model_name_to_module_name("Letter") + + assert result == "letter" + + def test_handles_digits_in_name(self): + """Test handling digits in model names.""" + result = model_name_to_module_name("ModelV2Update") + + assert result == "model_v2_update" + + def test_handles_hyphens(self): + """Test handling hyphens.""" + result = model_name_to_module_name("Model-With-Hyphens") + + assert result == "model_with_hyphens" + + def test_handles_empty_string(self): + """Test that an empty string is returned for empty input.""" + result = model_name_to_module_name("") + + assert result == "" diff --git a/src/pydantic-model-generator/tests/test_model_generator.py b/src/pydantic-model-generator/tests/test_model_generator.py new file mode 100644 index 000000000..52b74563d --- /dev/null +++ b/src/pydantic-model-generator/tests/test_model_generator.py @@ -0,0 +1,45 @@ +"""Tests for model_generator module.""" + +from unittest.mock import MagicMock, patch + +import pytest + +from scripts.model_generator import generate_pydantic_model + + +class TestGeneratePydanticModel: + """Tests for generate_pydantic_model function.""" + + @patch("scripts.model_generator.subprocess.run") + def test_calls_datamodel_codegen_with_expected_arguments(self, mock_run): + """Test successful model generation.""" + # Arrange + schema_path = "test_model.json" + output_file = "test_model.py" + mock_run.return_value = MagicMock(returncode=0, stderr="") + + # Act + generate_pydantic_model(schema_path, output_file, "TestModel") + + # Assert + assert mock_run.called + cmd_args = mock_run.call_args[0][0] + assert "datamodel-codegen" in cmd_args[0] # First arg is the executable + assert "--input" in cmd_args[1] + assert schema_path in cmd_args[2] + assert "--output" in cmd_args[3] + assert output_file in cmd_args[4] + assert "--class-name" in cmd_args[5] + assert "TestModel" in cmd_args[6] + + @patch("scripts.model_generator.subprocess.run") + def test_raises_error_on_generation_failure(self, mock_run): + """Test that it raises error when generation fails.""" + schema_path = "test_model.json" + output_file = "test_model.py" + mock_run.return_value = MagicMock( + returncode=1, stderr="Error: Invalid schema" + ) + + with pytest.raises(RuntimeError, match="Failed to generate model"): + generate_pydantic_model(schema_path, output_file, "TestModel") diff --git a/src/pydantic-model-generator/tests/test_schema_processor.py b/src/pydantic-model-generator/tests/test_schema_processor.py new file mode 100644 index 000000000..10dc39544 --- /dev/null +++ b/src/pydantic-model-generator/tests/test_schema_processor.py @@ -0,0 +1,67 @@ +"""Tests for schema_processor module.""" + +import pytest + +from scripts.schema_processor import ( + extract_event_type, + extract_model_name, +) + + +class TestExtractModelName: + """Tests for extract_model_name function.""" + + def test_extracts_title_from_schema(self): + """Test extracting model name from schema title.""" + schema = {"title": "PrintLetterAvailable", "type": "object"} + + result = extract_model_name(schema) + + assert result == "PrintLetterAvailable" + + def test_removes_invalid_characters(self): + """Tes handling spaces and special characters.""" + schema = {"title": "Print-Letter Available!_v2", "type": "object"} + + result = extract_model_name(schema) + + assert result == "PrintLetterAvailable_v2" + + def test_raises_error_if_no_title(self): + """Test that it raises error if schema has no title.""" + schema = {"type": "object"} + + with pytest.raises(ValueError, match="does not contain a 'title' field"): + extract_model_name(schema) + + +class TestExtractEventType: + """Tests for extract_event_type function.""" + + def test_extracts_event_type_from_schema(self): + """Test extracting event type from schema.""" + schema = { + "properties": { + "type": {"const": "uk.nhs.notify.digital.letters.letter.available.v1"} + } + } + + result = extract_event_type(schema) + + assert result == "uk.nhs.notify.digital.letters.letter.available.v1" + + def test_returns_none_if_no_type_property(self): + """Test that it returns None if no type property exists.""" + schema = {"properties": {}} + + result = extract_event_type(schema) + + assert result is None + + def test_returns_none_if_no_const(self): + """Test that it returns None if type has no const field.""" + schema = {"properties": {"type": {"type": "string"}}} + + result = extract_event_type(schema) + + assert result is None From 34d1332d8b5775d30e4a7ca06ac288aa2e43a681 Mon Sep 17 00:00:00 2001 From: Gareth Allan <157592212+gareth-allan@users.noreply.github.com> Date: Fri, 12 Dec 2025 10:30:13 +0000 Subject: [PATCH 02/20] CCM-12896: Remove venv --- src/pydantic-model-generator/Makefile | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) diff --git a/src/pydantic-model-generator/Makefile b/src/pydantic-model-generator/Makefile index b24c074d7..630edeb67 100644 --- a/src/pydantic-model-generator/Makefile +++ b/src/pydantic-model-generator/Makefile @@ -3,33 +3,28 @@ SCHEMA_SRC_DIR := ../../schemas/digital-letters/2025-10-draft/events OUTPUT_DIR := ../digital-letters-events-python/models SCRIPTS_DIR := scripts -# Python settings -VENV_DIR := ../../.venv -VENV_PYTHON := $(VENV_DIR)/bin/python -VENV_PIP := $(VENV_DIR)/bin/pip - # Default target .PHONY: all clean generate install install-dev test coverage help all: generate -# Install production dependencies (uses repo-level venv) +# Install production dependencies install: @echo "Installing production dependencies..." - @$(VENV_PIP) install -r requirements.txt + @pip install -r requirements.txt @echo "Production dependencies installed!" -# Install development dependencies (uses repo-level venv) +# Install development dependencies install-dev: @echo "Installing development dependencies..." - @$(VENV_PIP) install -r requirements-dev.txt + @pip install -r requirements-dev.txt @echo "Development dependencies installed!" # Generate Pydantic models from JSON schemas generate: install @echo "Generating Pydantic models from JSON schemas..." @mkdir -p $(OUTPUT_DIR) - @$(VENV_PYTHON) $(SCRIPTS_DIR)/generate_models.py \ + @python $(SCRIPTS_DIR)/generate_models.py \ --input-dir $(SCHEMA_SRC_DIR) \ --output-dir $(OUTPUT_DIR) @echo "Pydantic models generated in $(OUTPUT_DIR)/" From 35b960c2dec16e83c21cea9da70b4277d9b7ef82 Mon Sep 17 00:00:00 2001 From: Gareth Allan <157592212+gareth-allan@users.noreply.github.com> Date: Fri, 12 Dec 2025 11:00:02 +0000 Subject: [PATCH 03/20] CCM-12896: Run pydantic-model-generator tests as part of build --- scripts/config/sonar-scanner.properties | 2 +- scripts/tests/unit.sh | 5 +++++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/scripts/config/sonar-scanner.properties b/scripts/config/sonar-scanner.properties index 887943f8b..2a7294a02 100644 --- a/scripts/config/sonar-scanner.properties +++ b/scripts/config/sonar-scanner.properties @@ -8,7 +8,7 @@ sonar.tests=tests/, src/asyncapigenerator/tests, src/cloudeventjekylldocs/tests, sonar.test.inclusions=tests/**, src/**/tests/**, src/**/__tests__/**, lambdas/**/src/__tests__/**, utils/utils/src/__tests__/**, utils/sender-management/src/__tests__/** sonar.terraform.provider.aws.version=5.54.1 sonar.cpd.exclusions=**.test.* -sonar.coverage.exclusions=tests/**, src/**/tests/**, src/**/__tests__/**, **/*.dev.*, lambdas/**/src/__tests__/**, **/jest.config.ts, **/jest.config.cjs, scripts/**/*.*, docs/**/*.*, utils/utils/src/__tests__/**, src/asyncapigenerator/example_usage.py, src/asyncapigenerator/test_generator.py, src/eventcatalogasyncapiimporter/examples.py +sonar.coverage.exclusions=tests/**, src/**/tests/**, src/**/__tests__/**, **/*.dev.*, lambdas/**/src/__tests__/**, **/jest.config.ts, **/jest.config.cjs, scripts/**/*.*, docs/**/*.*, utils/utils/src/__tests__/**, src/asyncapigenerator/example_usage.py, src/asyncapigenerator/test_generator.py, src/eventcatalogasyncapiimporter/examples.py, src/digital-letters-events-python/** sonar.python.coverage.reportPaths=src/asyncapigenerator/coverage.xml,src/cloudeventjekylldocs/coverage.xml,src/eventcatalogasyncapiimporter/coverage.xml sonar.javascript.lcov.reportPaths=lcov.info,src/cloudevents/coverage/lcov.info diff --git a/scripts/tests/unit.sh b/scripts/tests/unit.sh index 5e9f83a40..ef8a65f23 100755 --- a/scripts/tests/unit.sh +++ b/scripts/tests/unit.sh @@ -34,6 +34,11 @@ echo "Setting up and running eventcatalogasyncapiimporter tests..." make -C ./src/eventcatalogasyncapiimporter install-dev make -C ./src/eventcatalogasyncapiimporter coverage # Run with coverage to generate coverage.xml for SonarCloud +# Python projects - pydantic-model-generator +echo "Setting up and running pydantic-model-generator tests..." +make -C ./src/pydantic-model-generator install-dev +make -C ./src/pydantic-model-generator coverage # Run with coverage to generate coverage.xml for SonarCloud + # TypeScript/JavaScript projects (npm workspace) # Note: src/cloudevents is included in workspaces, so it will be tested here npm ci From 81318cc6db521f95a38901e0834d1c044b260e37 Mon Sep 17 00:00:00 2001 From: Gareth Allan <157592212+gareth-allan@users.noreply.github.com> Date: Fri, 12 Dec 2025 11:28:44 +0000 Subject: [PATCH 04/20] CCM-12896: Move generated Python types to digital-letters-events package --- scripts/config/sonar-scanner.properties | 2 +- src/digital-letters-events-python/.gitignore | 2 - src/digital-letters-events-python/README.md | 66 ------------------- src/digital-letters-events/.gitignore | 1 + src/digital-letters-events/README.md | 49 ++++++++++++-- .../__init__.py | 0 .../requirements.txt | 0 src/pydantic-model-generator/Makefile | 2 +- src/pydantic-model-generator/README.md | 8 +-- 9 files changed, 52 insertions(+), 78 deletions(-) delete mode 100644 src/digital-letters-events-python/.gitignore delete mode 100644 src/digital-letters-events-python/README.md rename src/{digital-letters-events-python => digital-letters-events}/__init__.py (100%) rename src/{digital-letters-events-python => digital-letters-events}/requirements.txt (100%) diff --git a/scripts/config/sonar-scanner.properties b/scripts/config/sonar-scanner.properties index 2a7294a02..f76bcf8ca 100644 --- a/scripts/config/sonar-scanner.properties +++ b/scripts/config/sonar-scanner.properties @@ -8,7 +8,7 @@ sonar.tests=tests/, src/asyncapigenerator/tests, src/cloudeventjekylldocs/tests, sonar.test.inclusions=tests/**, src/**/tests/**, src/**/__tests__/**, lambdas/**/src/__tests__/**, utils/utils/src/__tests__/**, utils/sender-management/src/__tests__/** sonar.terraform.provider.aws.version=5.54.1 sonar.cpd.exclusions=**.test.* -sonar.coverage.exclusions=tests/**, src/**/tests/**, src/**/__tests__/**, **/*.dev.*, lambdas/**/src/__tests__/**, **/jest.config.ts, **/jest.config.cjs, scripts/**/*.*, docs/**/*.*, utils/utils/src/__tests__/**, src/asyncapigenerator/example_usage.py, src/asyncapigenerator/test_generator.py, src/eventcatalogasyncapiimporter/examples.py, src/digital-letters-events-python/** +sonar.coverage.exclusions=tests/**, src/**/tests/**, src/**/__tests__/**, **/*.dev.*, lambdas/**/src/__tests__/**, **/jest.config.ts, **/jest.config.cjs, scripts/**/*.*, docs/**/*.*, utils/utils/src/__tests__/**, src/asyncapigenerator/example_usage.py, src/asyncapigenerator/test_generator.py, src/eventcatalogasyncapiimporter/examples.py, src/digital-letters-events/** sonar.python.coverage.reportPaths=src/asyncapigenerator/coverage.xml,src/cloudeventjekylldocs/coverage.xml,src/eventcatalogasyncapiimporter/coverage.xml sonar.javascript.lcov.reportPaths=lcov.info,src/cloudevents/coverage/lcov.info diff --git a/src/digital-letters-events-python/.gitignore b/src/digital-letters-events-python/.gitignore deleted file mode 100644 index 38417dc64..000000000 --- a/src/digital-letters-events-python/.gitignore +++ /dev/null @@ -1,2 +0,0 @@ -# Models directory (auto-generated) -models/ diff --git a/src/digital-letters-events-python/README.md b/src/digital-letters-events-python/README.md deleted file mode 100644 index 28dcac606..000000000 --- a/src/digital-letters-events-python/README.md +++ /dev/null @@ -1,66 +0,0 @@ - - -# digital-letters-events-python - - - - -This package contains the automatically-generated Pydantic v2 models that the [pydantic-model-generator](../pydantic-model-generator/) tool produces. - - -The source files in this package should not be edited directly. If changes are required, update the schemas in the [schemas/digital-letters/2025-10-draft/events](../../schemas/digital-letters/2025-10-draft/events) directory and use the `pydantic-model-generator` tool to regenerate them. - -## Using this Package - -### Using Event Models - -The Pydantic models can be used by installing the `digital-letters-events-python` package and importing the desired model: - -```python -from digital_letters_events_python import PDMResourceSubmitted - -try: - # Validate and parse an event - event_data = { - "type": "uk.nhs.notify.digital.letters.pdm.resource.submitted.v1", - "source": "/nhs/england/notify/staging/dev-647563337/data-plane/digitalletters/pdm", - "dataschema": "https://notify.nhs.uk/cloudevents/schemas/digital-letters/2025-10-draft/data/digital-letters-pdm-resource-submitted-data.schema.json", - "specversion": "1.0", - "id": "0249e529-f947-4012-819e-b634eb71be79", - "subject": "pdm-resource-123", - "time": "2025-12-11T10:00:00Z", - "data": { - "something": "example value" - } - } - - # Create and validate the event - event = PDMResourceSubmitted(**event_data) - - # Access validated fields - print(event.id) - print(event.type) - print(event.data.something) -except Exception as e: - print(e) -``` - -### Type Safety and Validation - -All generated models include: - -- Type hints for all fields -- Runtime validation via Pydantic -- Automatic conversion of types where appropriate -- Clear error messages for validation failures - -## Development - -This package is automatically generated. Do not edit the files in the [`models`](./models/) directory directly. - -To regenerate the models: - -```bash -cd ../pydantic-model-generator -make generate -``` diff --git a/src/digital-letters-events/.gitignore b/src/digital-letters-events/.gitignore index abb5d782c..808d18e93 100644 --- a/src/digital-letters-events/.gitignore +++ b/src/digital-letters-events/.gitignore @@ -1,2 +1,3 @@ validators types +models diff --git a/src/digital-letters-events/README.md b/src/digital-letters-events/README.md index 6177d8115..0bbb98f34 100644 --- a/src/digital-letters-events/README.md +++ b/src/digital-letters-events/README.md @@ -1,16 +1,22 @@ # digital-letters-events + This package contains the automatically-generated code that the -[typescript-schema-generator](../typescript-schema-generator/) tool produces. +[typescript-schema-generator](../typescript-schema-generator/) and the +[pydantic-model-generator](../pydantic-model-generator/) tools produce. + The source files in this package should not be edited directly. If changes are required, update the schemas in the [src/cloudevents/domains](../cloudevents/domains) directory and use the -`typescript-schema-generator` tool to regenerate them. +`typescript-schema-generator` and `pydantic-model-generator` tools to regenerate +them. ## Using this Package -### Using Event Types +### TypeScript + +#### Using Event Types The event types can be used by simply installing the `digital-letters-events` package as a dependency and then importing @@ -51,7 +57,7 @@ const pdmResourceSubmittedEvent: PDMResourceSubmitted = { }; ``` -### Using Event Validator Functions +#### Using Event Validator Functions Validator functions for an event can be used by importing the default export from the relevant JS file in @@ -74,3 +80,38 @@ if (isEventValid) { Note: You will need to make sure the [`allowJs`](https://www.typescriptlang.org/tsconfig/#allowJs) option is set in your package's `tsconfig.json` in order to import the JS files. + +### Python + +#### Using Event Models + +The Pydantic models can be used by installing the `digital-letters-events` package and importing the desired model: + +```python +from digital_letters_events import PDMResourceSubmitted + +try: + # Validate and parse an event + event_data = { + "type": "uk.nhs.notify.digital.letters.pdm.resource.submitted.v1", + "source": "/nhs/england/notify/staging/dev-647563337/data-plane/digitalletters/pdm", + "dataschema": "https://notify.nhs.uk/cloudevents/schemas/digital-letters/2025-10-draft/data/digital-letters-pdm-resource-submitted-data.schema.json", + "specversion": "1.0", + "id": "0249e529-f947-4012-819e-b634eb71be79", + "subject": "pdm-resource-123", + "time": "2025-12-11T10:00:00Z", + "data": { + "something": "example value" + } + } + + # Create and validate the event + event = PDMResourceSubmitted(**event_data) + + # Access validated fields + print(event.id) + print(event.type) + print(event.data.something) +except Exception as e: + print(e) +``` diff --git a/src/digital-letters-events-python/__init__.py b/src/digital-letters-events/__init__.py similarity index 100% rename from src/digital-letters-events-python/__init__.py rename to src/digital-letters-events/__init__.py diff --git a/src/digital-letters-events-python/requirements.txt b/src/digital-letters-events/requirements.txt similarity index 100% rename from src/digital-letters-events-python/requirements.txt rename to src/digital-letters-events/requirements.txt diff --git a/src/pydantic-model-generator/Makefile b/src/pydantic-model-generator/Makefile index 630edeb67..1f15dfdcc 100644 --- a/src/pydantic-model-generator/Makefile +++ b/src/pydantic-model-generator/Makefile @@ -1,6 +1,6 @@ # Variables SCHEMA_SRC_DIR := ../../schemas/digital-letters/2025-10-draft/events -OUTPUT_DIR := ../digital-letters-events-python/models +OUTPUT_DIR := ../digital-letters-events/models SCRIPTS_DIR := scripts # Default target diff --git a/src/pydantic-model-generator/README.md b/src/pydantic-model-generator/README.md index 99e363ca0..d90fc291e 100644 --- a/src/pydantic-model-generator/README.md +++ b/src/pydantic-model-generator/README.md @@ -29,7 +29,7 @@ This will: - Read all JSON event schemas from `schemas/digital-letters/2025-10-draft/events/` - Generate Pydantic v2 models using `datamodel-code-generator` -- Output the models to `../digital-letters-events-python/models/` +- Output the models to `../digital-letters-events/models/` ### Configuration @@ -44,12 +44,12 @@ The generator can be configured through: Generated models are placed in the -[digital-letters-events-python](../digital-letters-events-python/) package: +[digital-letters-events](../digital-letters-events/) package: -- `../digital-letters-events-python/models/` - Individual Pydantic model files -- `../digital-letters-events-python/models/__init__.py` - Combined index for easy imports +- `../digital-letters-events/models/` - Individual Pydantic model files +- `../digital-letters-events/models/__init__.py` - Combined index for easy imports ## Testing From 87679be28e2678f8d9917495756ed67c12a9948b Mon Sep 17 00:00:00 2001 From: Gareth Allan <157592212+gareth-allan@users.noreply.github.com> Date: Fri, 12 Dec 2025 11:38:17 +0000 Subject: [PATCH 05/20] CCM-12896: Rename pydantic-model-generator to python-schema-generator --- Makefile | 4 ++-- scripts/tests/unit.sh | 8 ++++---- src/digital-letters-events/README.md | 6 ++++-- src/digital-letters-events/__init__.py | 2 +- .../Makefile | 12 ++++++------ .../README.md | 4 +++- .../pytest.ini | 0 .../requirements-dev.txt | 0 .../requirements.txt | 0 .../scripts/__init__.py | 0 .../scripts/file_utils.py | 0 .../scripts/generate_models.py | 0 .../scripts/model_generator.py | 0 .../scripts/schema_processor.py | 0 .../tests/__init__.py | 0 .../tests/test_file_utils.py | 0 .../tests/test_model_generator.py | 0 .../tests/test_schema_processor.py | 0 18 files changed, 20 insertions(+), 16 deletions(-) rename src/{pydantic-model-generator => python-schema-generator}/Makefile (84%) rename src/{pydantic-model-generator => python-schema-generator}/README.md (96%) rename src/{pydantic-model-generator => python-schema-generator}/pytest.ini (100%) rename src/{pydantic-model-generator => python-schema-generator}/requirements-dev.txt (100%) rename src/{pydantic-model-generator => python-schema-generator}/requirements.txt (100%) rename src/{pydantic-model-generator => python-schema-generator}/scripts/__init__.py (100%) rename src/{pydantic-model-generator => python-schema-generator}/scripts/file_utils.py (100%) rename src/{pydantic-model-generator => python-schema-generator}/scripts/generate_models.py (100%) rename src/{pydantic-model-generator => python-schema-generator}/scripts/model_generator.py (100%) rename src/{pydantic-model-generator => python-schema-generator}/scripts/schema_processor.py (100%) rename src/{pydantic-model-generator => python-schema-generator}/tests/__init__.py (100%) rename src/{pydantic-model-generator => python-schema-generator}/tests/test_file_utils.py (100%) rename src/{pydantic-model-generator => python-schema-generator}/tests/test_model_generator.py (100%) rename src/{pydantic-model-generator => python-schema-generator}/tests/test_schema_processor.py (100%) diff --git a/Makefile b/Makefile index 89c5a3981..9e5fb9c84 100644 --- a/Makefile +++ b/Makefile @@ -14,7 +14,7 @@ dependencies: # Install dependencies needed to build and test the project @Pipel generate: # Generate any autogenerated output @Pipeline npm run generate-dependencies - $(MAKE) -C src/pydantic-model-generator generate + $(MAKE) -C src/python-schema-generator generate build: # Build the project artefact @Pipeline $(MAKE) -C docs build @@ -33,7 +33,7 @@ clean:: # Clean-up project resources (main) @Operations $(MAKE) -C src/cloudevents clean $(MAKE) -C src/eventcatalogasyncapiimporter clean $(MAKE) -C src/eventcatalogasyncapiimporter clean-output - $(MAKE) -C src/pydantic-model-generator clean + $(MAKE) -C src/python-schema-generator clean rm -f .version npm run clean diff --git a/scripts/tests/unit.sh b/scripts/tests/unit.sh index ef8a65f23..afa1476b0 100755 --- a/scripts/tests/unit.sh +++ b/scripts/tests/unit.sh @@ -34,10 +34,10 @@ echo "Setting up and running eventcatalogasyncapiimporter tests..." make -C ./src/eventcatalogasyncapiimporter install-dev make -C ./src/eventcatalogasyncapiimporter coverage # Run with coverage to generate coverage.xml for SonarCloud -# Python projects - pydantic-model-generator -echo "Setting up and running pydantic-model-generator tests..." -make -C ./src/pydantic-model-generator install-dev -make -C ./src/pydantic-model-generator coverage # Run with coverage to generate coverage.xml for SonarCloud +# Python projects - python-schema-generator +echo "Setting up and running python-schema-generator tests..." +make -C ./src/python-schema-generator install-dev +make -C ./src/python-schema-generator coverage # Run with coverage to generate coverage.xml for SonarCloud # TypeScript/JavaScript projects (npm workspace) # Note: src/cloudevents is included in workspaces, so it will be tested here diff --git a/src/digital-letters-events/README.md b/src/digital-letters-events/README.md index 0bbb98f34..c3b7a45fd 100644 --- a/src/digital-letters-events/README.md +++ b/src/digital-letters-events/README.md @@ -1,15 +1,17 @@ # digital-letters-events + This package contains the automatically-generated code that the [typescript-schema-generator](../typescript-schema-generator/) and the -[pydantic-model-generator](../pydantic-model-generator/) tools produce. +[python-schema-generator](../python-schema-generator/) tools produce. + The source files in this package should not be edited directly. If changes are required, update the schemas in the [src/cloudevents/domains](../cloudevents/domains) directory and use the -`typescript-schema-generator` and `pydantic-model-generator` tools to regenerate +`typescript-schema-generator` and `python-schema-generator` tools to regenerate them. ## Using this Package diff --git a/src/digital-letters-events/__init__.py b/src/digital-letters-events/__init__.py index 7d097b631..2ef191906 100644 --- a/src/digital-letters-events/__init__.py +++ b/src/digital-letters-events/__init__.py @@ -2,7 +2,7 @@ This package contains automatically-generated Pydantic v2 models for NHS Notify Digital Letters events. These models are generated from JSON schemas by the -pydantic-model-generator tool. +python-schema-generator tool. DO NOT EDIT: Files in this package are auto-generated. Any manual changes will be overwritten when the models are regenerated. diff --git a/src/pydantic-model-generator/Makefile b/src/python-schema-generator/Makefile similarity index 84% rename from src/pydantic-model-generator/Makefile rename to src/python-schema-generator/Makefile index 1f15dfdcc..cedc50e80 100644 --- a/src/pydantic-model-generator/Makefile +++ b/src/python-schema-generator/Makefile @@ -32,17 +32,17 @@ generate: install # Run tests test: install-dev @echo "Running tests..." - @cd ../.. && pytest src/pydantic-model-generator/tests/ + @pytest tests/ # Generate coverage report coverage: install-dev @echo "Generating coverage report..." - @cd ../.. && pytest src/pydantic-model-generator/tests/ \ - --cov=src/pydantic-model-generator \ - --cov-config=src/pydantic-model-generator/pytest.ini \ - --cov-report=html:src/pydantic-model-generator/htmlcov \ + @pytest tests/ \ + --cov=. \ + --cov-config=pytest.ini \ + --cov-report=html:htmlcov \ --cov-report=term-missing \ - --cov-report=xml:src/pydantic-model-generator/coverage.xml \ + --cov-report=xml:coverage.xml \ --cov-branch @echo "Coverage report generated in htmlcov/" diff --git a/src/pydantic-model-generator/README.md b/src/python-schema-generator/README.md similarity index 96% rename from src/pydantic-model-generator/README.md rename to src/python-schema-generator/README.md index d90fc291e..1bbb001e3 100644 --- a/src/pydantic-model-generator/README.md +++ b/src/python-schema-generator/README.md @@ -1,5 +1,7 @@ + -# pydantic-model-generator +# python-schema-generator + This package provides a tool that generates Pydantic v2 models for the events diff --git a/src/pydantic-model-generator/pytest.ini b/src/python-schema-generator/pytest.ini similarity index 100% rename from src/pydantic-model-generator/pytest.ini rename to src/python-schema-generator/pytest.ini diff --git a/src/pydantic-model-generator/requirements-dev.txt b/src/python-schema-generator/requirements-dev.txt similarity index 100% rename from src/pydantic-model-generator/requirements-dev.txt rename to src/python-schema-generator/requirements-dev.txt diff --git a/src/pydantic-model-generator/requirements.txt b/src/python-schema-generator/requirements.txt similarity index 100% rename from src/pydantic-model-generator/requirements.txt rename to src/python-schema-generator/requirements.txt diff --git a/src/pydantic-model-generator/scripts/__init__.py b/src/python-schema-generator/scripts/__init__.py similarity index 100% rename from src/pydantic-model-generator/scripts/__init__.py rename to src/python-schema-generator/scripts/__init__.py diff --git a/src/pydantic-model-generator/scripts/file_utils.py b/src/python-schema-generator/scripts/file_utils.py similarity index 100% rename from src/pydantic-model-generator/scripts/file_utils.py rename to src/python-schema-generator/scripts/file_utils.py diff --git a/src/pydantic-model-generator/scripts/generate_models.py b/src/python-schema-generator/scripts/generate_models.py similarity index 100% rename from src/pydantic-model-generator/scripts/generate_models.py rename to src/python-schema-generator/scripts/generate_models.py diff --git a/src/pydantic-model-generator/scripts/model_generator.py b/src/python-schema-generator/scripts/model_generator.py similarity index 100% rename from src/pydantic-model-generator/scripts/model_generator.py rename to src/python-schema-generator/scripts/model_generator.py diff --git a/src/pydantic-model-generator/scripts/schema_processor.py b/src/python-schema-generator/scripts/schema_processor.py similarity index 100% rename from src/pydantic-model-generator/scripts/schema_processor.py rename to src/python-schema-generator/scripts/schema_processor.py diff --git a/src/pydantic-model-generator/tests/__init__.py b/src/python-schema-generator/tests/__init__.py similarity index 100% rename from src/pydantic-model-generator/tests/__init__.py rename to src/python-schema-generator/tests/__init__.py diff --git a/src/pydantic-model-generator/tests/test_file_utils.py b/src/python-schema-generator/tests/test_file_utils.py similarity index 100% rename from src/pydantic-model-generator/tests/test_file_utils.py rename to src/python-schema-generator/tests/test_file_utils.py diff --git a/src/pydantic-model-generator/tests/test_model_generator.py b/src/python-schema-generator/tests/test_model_generator.py similarity index 100% rename from src/pydantic-model-generator/tests/test_model_generator.py rename to src/python-schema-generator/tests/test_model_generator.py diff --git a/src/pydantic-model-generator/tests/test_schema_processor.py b/src/python-schema-generator/tests/test_schema_processor.py similarity index 100% rename from src/pydantic-model-generator/tests/test_schema_processor.py rename to src/python-schema-generator/tests/test_schema_processor.py From 94c58674f7d32fca081a80b27f07958f2a374e60 Mon Sep 17 00:00:00 2001 From: Gareth Allan <157592212+gareth-allan@users.noreply.github.com> Date: Mon, 15 Dec 2025 12:14:53 +0000 Subject: [PATCH 06/20] CCM-12896: Small tweaks to digital-letters-events package for Python --- .gitignore | 1 + .../{ => digital_letters_events}/__init__.py | 0 src/digital-letters-events/setup.py | 7 +++++++ src/python-schema-generator/Makefile | 2 +- src/python-schema-generator/scripts/model_generator.py | 5 ----- 5 files changed, 9 insertions(+), 6 deletions(-) rename src/digital-letters-events/{ => digital_letters_events}/__init__.py (100%) create mode 100644 src/digital-letters-events/setup.py diff --git a/.gitignore b/.gitignore index 3f24d00e8..498acdd93 100644 --- a/.gitignore +++ b/.gitignore @@ -31,6 +31,7 @@ __pycache__/ .Python .venv/ venv/ +*.egg-info/ # Testing .pytest_cache/ diff --git a/src/digital-letters-events/__init__.py b/src/digital-letters-events/digital_letters_events/__init__.py similarity index 100% rename from src/digital-letters-events/__init__.py rename to src/digital-letters-events/digital_letters_events/__init__.py diff --git a/src/digital-letters-events/setup.py b/src/digital-letters-events/setup.py new file mode 100644 index 000000000..b9c3b3f57 --- /dev/null +++ b/src/digital-letters-events/setup.py @@ -0,0 +1,7 @@ +from setuptools import setup, find_packages + +setup( + name="digital-letters-events", + version="0.1.0", + packages=find_packages(), +) diff --git a/src/python-schema-generator/Makefile b/src/python-schema-generator/Makefile index cedc50e80..4cc68a084 100644 --- a/src/python-schema-generator/Makefile +++ b/src/python-schema-generator/Makefile @@ -1,6 +1,6 @@ # Variables SCHEMA_SRC_DIR := ../../schemas/digital-letters/2025-10-draft/events -OUTPUT_DIR := ../digital-letters-events/models +OUTPUT_DIR := ../digital-letters-events/digital_letters_events/models SCRIPTS_DIR := scripts # Default target diff --git a/src/python-schema-generator/scripts/model_generator.py b/src/python-schema-generator/scripts/model_generator.py index 45c6ba528..a1b2df3e1 100644 --- a/src/python-schema-generator/scripts/model_generator.py +++ b/src/python-schema-generator/scripts/model_generator.py @@ -33,11 +33,6 @@ def generate_pydantic_model( "--output-model-type", "pydantic_v2.BaseModel", "--use-schema-description", - "--field-constraints", - "--strict-nullable", - "--snake-case-field", - "--target-python-version", - "3.13", "--custom-file-header", '''"""Generated Pydantic model for NHS Notify Digital Letters events. From d8d4a8fcb71504188dcad038f90a3f7fb1dc5133 Mon Sep 17 00:00:00 2001 From: Gareth Allan <157592212+gareth-allan@users.noreply.github.com> Date: Mon, 15 Dec 2025 12:41:59 +0000 Subject: [PATCH 07/20] CCM-12896: Preprocess schemas to remove allOf --- package-lock.json | 416 ++++++++++++++++-- package.json | 1 + project.code-workspace | 2 +- src/python-schema-generator/.gitignore | 1 + src/python-schema-generator/Makefile | 14 +- src/python-schema-generator/README.md | 8 +- src/python-schema-generator/jest.config.ts | 11 + src/python-schema-generator/package.json | 26 ++ .../{scripts => src}/__init__.py | 0 .../src/__tests__/merge-allof.test.ts | 45 ++ src/python-schema-generator/src/file-utils.ts | 68 +++ .../{scripts => src}/file_utils.py | 0 .../{scripts => src}/generate_models.py | 0 .../src/merge-allof-cli.ts | 3 + .../src/merge-allof.ts | 34 ++ .../{scripts => src}/model_generator.py | 0 .../{scripts => src}/schema_processor.py | 0 .../tests/test_file_utils.py | 2 +- .../tests/test_model_generator.py | 6 +- .../tests/test_schema_processor.py | 2 +- src/python-schema-generator/tsconfig.json | 15 + 21 files changed, 593 insertions(+), 61 deletions(-) create mode 100644 src/python-schema-generator/.gitignore create mode 100644 src/python-schema-generator/jest.config.ts create mode 100644 src/python-schema-generator/package.json rename src/python-schema-generator/{scripts => src}/__init__.py (100%) create mode 100644 src/python-schema-generator/src/__tests__/merge-allof.test.ts create mode 100644 src/python-schema-generator/src/file-utils.ts rename src/python-schema-generator/{scripts => src}/file_utils.py (100%) rename src/python-schema-generator/{scripts => src}/generate_models.py (100%) create mode 100644 src/python-schema-generator/src/merge-allof-cli.ts create mode 100644 src/python-schema-generator/src/merge-allof.ts rename src/python-schema-generator/{scripts => src}/model_generator.py (100%) rename src/python-schema-generator/{scripts => src}/schema_processor.py (100%) create mode 100644 src/python-schema-generator/tsconfig.json diff --git a/package-lock.json b/package-lock.json index d0171bdd1..b0027d5ca 100644 --- a/package-lock.json +++ b/package-lock.json @@ -16,6 +16,7 @@ "utils/sender-management", "src/cloudevents", "src/digital-letters-events", + "src/python-schema-generator", "src/typescript-schema-generator", "tests/playwright" ], @@ -421,7 +422,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/client-sso/-/client-sso-3.928.0.tgz", "integrity": "sha512-Efenb8zV2fJJDXmp2NE4xj8Ymhp4gVJCkQ6ixhdrpfQXgd2PODO7a20C2+BhFM6aGmN3m6XWYJ64ZyhXF4pAyQ==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@aws-crypto/sha256-browser": "5.2.0", "@aws-crypto/sha256-js": "5.2.0", @@ -471,7 +471,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/core/-/core-3.928.0.tgz", "integrity": "sha512-e28J2uKjy2uub4u41dNnmzAu0AN3FGB+LRcLN2Qnwl9Oq3kIcByl5sM8ZD+vWpNG+SFUrUasBCq8cMnHxwXZ4w==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@aws-sdk/types": "3.922.0", "@aws-sdk/xml-builder": "3.921.0", @@ -496,7 +495,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/credential-provider-env/-/credential-provider-env-3.928.0.tgz", "integrity": "sha512-tB8F9Ti0/NFyFVQX8UQtgRik88evtHpyT6WfXOB4bAY6lEnEHA0ubJZmk9y+aUeoE+OsGLx70dC3JUsiiCPJkQ==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@aws-sdk/core": "3.928.0", "@aws-sdk/types": "3.922.0", @@ -513,7 +511,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/credential-provider-http/-/credential-provider-http-3.928.0.tgz", "integrity": "sha512-67ynC/8UW9Y8Gn1ZZtC3OgcQDGWrJelHmkbgpmmxYUrzVhp+NINtz3wiTzrrBFhPH/8Uy6BxvhMfXhn0ptcMEQ==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@aws-sdk/core": "3.928.0", "@aws-sdk/types": "3.922.0", @@ -535,7 +532,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/credential-provider-ini/-/credential-provider-ini-3.928.0.tgz", "integrity": "sha512-WVWYyj+jox6mhKYp11mu8x1B6Xa2sLbXFHAv5K3Jg8CHvXYpePgTcYlCljq3d4XHC4Jl4nCcsdMtBahSpU9bAA==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@aws-sdk/core": "3.928.0", "@aws-sdk/credential-provider-env": "3.928.0", @@ -560,7 +556,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/credential-provider-node/-/credential-provider-node-3.928.0.tgz", "integrity": "sha512-SdXVjxZOIXefIR/NJx+lyXOrn4m0ScTAU2JXpLsFCkW2Cafo6vTqHUghyO8vak/XQ8PpPqpLXVpGbAYFuIPW6Q==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@aws-sdk/credential-provider-env": "3.928.0", "@aws-sdk/credential-provider-http": "3.928.0", @@ -584,7 +579,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/credential-provider-process/-/credential-provider-process-3.928.0.tgz", "integrity": "sha512-XL0juran8yhqwn0mreV+NJeHJOkcRBaExsvVn9fXWW37A4gLh4esSJxM2KbSNh0t+/Bk3ehBI5sL9xad+yRDuw==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@aws-sdk/core": "3.928.0", "@aws-sdk/types": "3.922.0", @@ -602,7 +596,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/credential-provider-sso/-/credential-provider-sso-3.928.0.tgz", "integrity": "sha512-md/y+ePDsO1zqPJrsOyPs4ciKmdpqLL7B0dln1NhqZPnKIS5IBfTqZJ5tJ9eTezqc7Tn4Dbg6HiuemcGvZTeFA==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@aws-sdk/client-sso": "3.928.0", "@aws-sdk/core": "3.928.0", @@ -622,7 +615,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/credential-provider-web-identity/-/credential-provider-web-identity-3.928.0.tgz", "integrity": "sha512-rd97nLY5e/nGOr73ZfsXD+H44iZ9wyGZTKt/2QkiBN3hot/idhgT9+XHsWhRi+o/dThQbpL8RkpAnpF+0ZGthw==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@aws-sdk/core": "3.928.0", "@aws-sdk/nested-clients": "3.928.0", @@ -641,7 +633,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/middleware-endpoint-discovery/-/middleware-endpoint-discovery-3.922.0.tgz", "integrity": "sha512-F7Qhwz/bs/Wkbu4SLwKbAeQKoZ7Bzo+JPpVzSqSJGxEely8KBAfsOItXRF8c0d06OEzyeSyml0S6/3TP8T5KUw==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@aws-sdk/endpoint-cache": "3.893.0", "@aws-sdk/types": "3.922.0", @@ -659,7 +650,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/middleware-host-header/-/middleware-host-header-3.922.0.tgz", "integrity": "sha512-HPquFgBnq/KqKRVkiuCt97PmWbKtxQ5iUNLEc6FIviqOoZTmaYG3EDsIbuFBz9C4RHJU4FKLmHL2bL3FEId6AA==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@aws-sdk/types": "3.922.0", "@smithy/protocol-http": "^5.3.4", @@ -675,7 +665,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/middleware-logger/-/middleware-logger-3.922.0.tgz", "integrity": "sha512-AkvYO6b80FBm5/kk2E636zNNcNgjztNNUxpqVx+huyGn9ZqGTzS4kLqW2hO6CBe5APzVtPCtiQsXL24nzuOlAg==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@aws-sdk/types": "3.922.0", "@smithy/types": "^4.8.1", @@ -690,7 +679,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/middleware-recursion-detection/-/middleware-recursion-detection-3.922.0.tgz", "integrity": "sha512-TtSCEDonV/9R0VhVlCpxZbp/9sxQvTTRKzIf8LxW3uXpby6Wl8IxEciBJlxmSkoqxh542WRcko7NYODlvL/gDA==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@aws-sdk/types": "3.922.0", "@aws/lambda-invoke-store": "^0.1.1", @@ -707,7 +695,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/middleware-user-agent/-/middleware-user-agent-3.928.0.tgz", "integrity": "sha512-ESvcfLx5PtpdUM3ptCwb80toBTd3y5I4w5jaeOPHihiZr7jkRLE/nsaCKzlqscPs6UQ8xI0maav04JUiTskcHw==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@aws-sdk/core": "3.928.0", "@aws-sdk/types": "3.922.0", @@ -726,7 +713,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/nested-clients/-/nested-clients-3.928.0.tgz", "integrity": "sha512-kXzfJkq2cD65KAHDe4hZCsnxcGGEWD5pjHqcZplwG4VFMa/iVn/mWrUY9QdadD2GBpXFNQbgOiKG3U2NkKu+4Q==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@aws-crypto/sha256-browser": "5.2.0", "@aws-crypto/sha256-js": "5.2.0", @@ -776,7 +762,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/region-config-resolver/-/region-config-resolver-3.925.0.tgz", "integrity": "sha512-FOthcdF9oDb1pfQBRCfWPZhJZT5wqpvdAS5aJzB1WDZ+6EuaAhLzLH/fW1slDunIqq1PSQGG3uSnVglVVOvPHQ==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@aws-sdk/types": "3.922.0", "@smithy/config-resolver": "^4.4.2", @@ -793,7 +778,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/token-providers/-/token-providers-3.928.0.tgz", "integrity": "sha512-533NpTdUJNDi98zBwRp4ZpZoqULrAVfc0YgIy+8AZHzk0v7N+v59O0d2Du3YO6zN4VU8HU8766DgKiyEag6Dzg==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@aws-sdk/core": "3.928.0", "@aws-sdk/nested-clients": "3.928.0", @@ -812,7 +796,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/types/-/types-3.922.0.tgz", "integrity": "sha512-eLA6XjVobAUAMivvM7DBL79mnHyrm+32TkXNWZua5mnxF+6kQCfblKKJvxMZLGosO53/Ex46ogim8IY5Nbqv2w==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/types": "^4.8.1", "tslib": "^2.6.2" @@ -841,7 +824,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/util-endpoints/-/util-endpoints-3.922.0.tgz", "integrity": "sha512-4ZdQCSuNMY8HMlR1YN4MRDdXuKd+uQTeKIr5/pIM+g3TjInZoj8imvXudjcrFGA63UF3t92YVTkBq88mg58RXQ==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@aws-sdk/types": "3.922.0", "@smithy/types": "^4.8.1", @@ -858,7 +840,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/util-user-agent-browser/-/util-user-agent-browser-3.922.0.tgz", "integrity": "sha512-qOJAERZ3Plj1st7M4Q5henl5FRpE30uLm6L9edZqZXGR6c7ry9jzexWamWVpQ4H4xVAVmiO9dIEBAfbq4mduOA==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@aws-sdk/types": "3.922.0", "@smithy/types": "^4.8.1", @@ -871,7 +852,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/util-user-agent-node/-/util-user-agent-node-3.928.0.tgz", "integrity": "sha512-s0jP67nQLLWVWfBtqTkZUkSWK5e6OI+rs+wFya2h9VLyWBFir17XSDI891s8HZKIVCEl8eBrup+hhywm4nsIAA==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@aws-sdk/middleware-user-agent": "3.928.0", "@aws-sdk/types": "3.922.0", @@ -896,7 +876,6 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/xml-builder/-/xml-builder-3.921.0.tgz", "integrity": "sha512-LVHg0jgjyicKKvpNIEMXIMr1EBViESxcPkqfOlT+X1FkmUMTNZEEVF18tOJg4m4hV5vxtkWcqtr4IEeWa1C41Q==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/types": "^4.8.1", "fast-xml-parser": "5.2.5", @@ -911,7 +890,6 @@ "resolved": "https://registry.npmjs.org/@aws/lambda-invoke-store/-/lambda-invoke-store-0.1.1.tgz", "integrity": "sha512-RcLam17LdlbSOSp9VxmUu1eI6Mwxp+OwhD2QhiSNmNCzoDb0EeUXTD2n/WbcnrAYMGlmf05th6QYq23VqvJqpA==", "license": "Apache-2.0", - "peer": true, "engines": { "node": ">=18.0.0" } @@ -921,7 +899,6 @@ "resolved": "https://registry.npmjs.org/@smithy/abort-controller/-/abort-controller-4.2.5.tgz", "integrity": "sha512-j7HwVkBw68YW8UmFRcjZOmssE77Rvk0GWAIN1oFBhsaovQmZWYCIcGa9/pwRB0ExI8Sk9MWNALTjftjHZea7VA==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/types": "^4.9.0", "tslib": "^2.6.2" @@ -935,7 +912,6 @@ "resolved": "https://registry.npmjs.org/@smithy/config-resolver/-/config-resolver-4.4.3.tgz", "integrity": "sha512-ezHLe1tKLUxDJo2LHtDuEDyWXolw8WGOR92qb4bQdWq/zKenO5BvctZGrVJBK08zjezSk7bmbKFOXIVyChvDLw==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/node-config-provider": "^4.3.5", "@smithy/types": "^4.9.0", @@ -953,7 +929,6 @@ "resolved": "https://registry.npmjs.org/@smithy/core/-/core-3.18.0.tgz", "integrity": "sha512-vGSDXOJFZgOPTatSI1ly7Gwyy/d/R9zh2TO3y0JZ0uut5qQ88p9IaWaZYIWSSqtdekNM4CGok/JppxbAff4KcQ==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/middleware-serde": "^4.2.5", "@smithy/protocol-http": "^5.3.5", @@ -975,7 +950,6 @@ "resolved": "https://registry.npmjs.org/@smithy/credential-provider-imds/-/credential-provider-imds-4.2.5.tgz", "integrity": "sha512-BZwotjoZWn9+36nimwm/OLIcVe+KYRwzMjfhd4QT7QxPm9WY0HiOV8t/Wlh+HVUif0SBVV7ksq8//hPaBC/okQ==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/node-config-provider": "^4.3.5", "@smithy/property-provider": "^4.2.5", @@ -992,7 +966,6 @@ "resolved": "https://registry.npmjs.org/@smithy/fetch-http-handler/-/fetch-http-handler-5.3.6.tgz", "integrity": "sha512-3+RG3EA6BBJ/ofZUeTFJA7mHfSYrZtQIrDP9dI8Lf7X6Jbos2jptuLrAAteDiFVrmbEmLSuRG/bUKzfAXk7dhg==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/protocol-http": "^5.3.5", "@smithy/querystring-builder": "^4.2.5", @@ -1009,7 +982,6 @@ "resolved": "https://registry.npmjs.org/@smithy/middleware-endpoint/-/middleware-endpoint-4.3.7.tgz", "integrity": "sha512-i8Mi8OuY6Yi82Foe3iu7/yhBj1HBRoOQwBSsUNYglJTNSFaWYTNM2NauBBs/7pq2sqkLRqeUXA3Ogi2utzpUlQ==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/core": "^3.18.0", "@smithy/middleware-serde": "^4.2.5", @@ -1029,7 +1001,6 @@ "resolved": "https://registry.npmjs.org/@smithy/middleware-serde/-/middleware-serde-4.2.5.tgz", "integrity": "sha512-La1ldWTJTZ5NqQyPqnCNeH9B+zjFhrNoQIL1jTh4zuqXRlmXhxYHhMtI1/92OlnoAtp6JoN7kzuwhWoXrBwPqg==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/protocol-http": "^5.3.5", "@smithy/types": "^4.9.0", @@ -1044,7 +1015,6 @@ "resolved": "https://registry.npmjs.org/@smithy/middleware-stack/-/middleware-stack-4.2.5.tgz", "integrity": "sha512-bYrutc+neOyWxtZdbB2USbQttZN0mXaOyYLIsaTbJhFsfpXyGWUxJpEuO1rJ8IIJm2qH4+xJT0mxUSsEDTYwdQ==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/types": "^4.9.0", "tslib": "^2.6.2" @@ -1058,7 +1028,6 @@ "resolved": "https://registry.npmjs.org/@smithy/node-config-provider/-/node-config-provider-4.3.5.tgz", "integrity": "sha512-UTurh1C4qkVCtqggI36DGbLB2Kv8UlcFdMXDcWMbqVY2uRg0XmT9Pb4Vj6oSQ34eizO1fvR0RnFV4Axw4IrrAg==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/property-provider": "^4.2.5", "@smithy/shared-ini-file-loader": "^4.4.0", @@ -1074,7 +1043,6 @@ "resolved": "https://registry.npmjs.org/@smithy/node-http-handler/-/node-http-handler-4.4.5.tgz", "integrity": "sha512-CMnzM9R2WqlqXQGtIlsHMEZfXKJVTIrqCNoSd/QpAyp+Dw0a1Vps13l6ma1fH8g7zSPNsA59B/kWgeylFuA/lw==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/abort-controller": "^4.2.5", "@smithy/protocol-http": "^5.3.5", @@ -1091,7 +1059,6 @@ "resolved": "https://registry.npmjs.org/@smithy/property-provider/-/property-provider-4.2.5.tgz", "integrity": "sha512-8iLN1XSE1rl4MuxvQ+5OSk/Zb5El7NJZ1td6Tn+8dQQHIjp59Lwl6bd0+nzw6SKm2wSSriH2v/I9LPzUic7EOg==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/types": "^4.9.0", "tslib": "^2.6.2" @@ -1105,7 +1072,6 @@ "resolved": "https://registry.npmjs.org/@smithy/protocol-http/-/protocol-http-5.3.5.tgz", "integrity": "sha512-RlaL+sA0LNMp03bf7XPbFmT5gN+w3besXSWMkA8rcmxLSVfiEXElQi4O2IWwPfxzcHkxqrwBFMbngB8yx/RvaQ==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/types": "^4.9.0", "tslib": "^2.6.2" @@ -1119,7 +1085,6 @@ "resolved": "https://registry.npmjs.org/@smithy/querystring-builder/-/querystring-builder-4.2.5.tgz", "integrity": "sha512-y98otMI1saoajeik2kLfGyRp11e5U/iJYH/wLCh3aTV/XutbGT9nziKGkgCaMD1ghK7p6htHMm6b6scl9JRUWg==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/types": "^4.9.0", "@smithy/util-uri-escape": "^4.2.0", @@ -1134,7 +1099,6 @@ "resolved": "https://registry.npmjs.org/@smithy/querystring-parser/-/querystring-parser-4.2.5.tgz", "integrity": "sha512-031WCTdPYgiQRYNPXznHXof2YM0GwL6SeaSyTH/P72M1Vz73TvCNH2Nq8Iu2IEPq9QP2yx0/nrw5YmSeAi/AjQ==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/types": "^4.9.0", "tslib": "^2.6.2" @@ -1148,7 +1112,6 @@ "resolved": "https://registry.npmjs.org/@smithy/shared-ini-file-loader/-/shared-ini-file-loader-4.4.0.tgz", "integrity": "sha512-5WmZ5+kJgJDjwXXIzr1vDTG+RhF9wzSODQBfkrQ2VVkYALKGvZX1lgVSxEkgicSAFnFhPj5rudJV0zoinqS0bA==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/types": "^4.9.0", "tslib": "^2.6.2" @@ -1162,7 +1125,6 @@ "resolved": "https://registry.npmjs.org/@smithy/smithy-client/-/smithy-client-4.9.3.tgz", "integrity": "sha512-8tlueuTgV5n7inQCkhyptrB3jo2AO80uGrps/XTYZivv5MFQKKBj3CIWIGMI2fRY5LEduIiazOhAWdFknY1O9w==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/core": "^3.18.0", "@smithy/middleware-endpoint": "^4.3.7", @@ -1181,7 +1143,6 @@ "resolved": "https://registry.npmjs.org/@smithy/types/-/types-4.9.0.tgz", "integrity": "sha512-MvUbdnXDTwykR8cB1WZvNNwqoWVaTRA0RLlLmf/cIFNMM2cKWz01X4Ly6SMC4Kks30r8tT3Cty0jmeWfiuyHTA==", "license": "Apache-2.0", - "peer": true, "dependencies": { "tslib": "^2.6.2" }, @@ -1194,7 +1155,6 @@ "resolved": "https://registry.npmjs.org/@smithy/url-parser/-/url-parser-4.2.5.tgz", "integrity": "sha512-VaxMGsilqFnK1CeBX+LXnSuaMx4sTL/6znSZh2829txWieazdVxr54HmiyTsIbpOTLcf5nYpq9lpzmwRdxj6rQ==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/querystring-parser": "^4.2.5", "@smithy/types": "^4.9.0", @@ -1209,7 +1169,6 @@ "resolved": "https://registry.npmjs.org/@smithy/util-defaults-mode-node/-/util-defaults-mode-node-4.2.9.tgz", "integrity": "sha512-dgyribrVWN5qE5usYJ0m5M93mVM3L3TyBPZWe1Xl6uZlH2gzfQx3dz+ZCdW93lWqdedJRkOecnvbnoEEXRZ5VQ==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/config-resolver": "^4.4.3", "@smithy/credential-provider-imds": "^4.2.5", @@ -1228,7 +1187,6 @@ "resolved": "https://registry.npmjs.org/@smithy/util-endpoints/-/util-endpoints-3.2.5.tgz", "integrity": "sha512-3O63AAWu2cSNQZp+ayl9I3NapW1p1rR5mlVHcF6hAB1dPZUQFfRPYtplWX/3xrzWthPGj5FqB12taJJCfH6s8A==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/node-config-provider": "^4.3.5", "@smithy/types": "^4.9.0", @@ -1243,7 +1201,6 @@ "resolved": "https://registry.npmjs.org/@smithy/util-middleware/-/util-middleware-4.2.5.tgz", "integrity": "sha512-6Y3+rvBF7+PZOc40ybeZMcGln6xJGVeY60E7jy9Mv5iKpMJpHgRE6dKy9ScsVxvfAYuEX4Q9a65DQX90KaQ3bA==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/types": "^4.9.0", "tslib": "^2.6.2" @@ -1257,7 +1214,6 @@ "resolved": "https://registry.npmjs.org/@smithy/util-stream/-/util-stream-4.5.6.tgz", "integrity": "sha512-qWw/UM59TiaFrPevefOZ8CNBKbYEP6wBAIlLqxn3VAIo9rgnTNc4ASbVrqDmhuwI87usnjhdQrxodzAGFFzbRQ==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/fetch-http-handler": "^5.3.6", "@smithy/node-http-handler": "^4.4.5", @@ -1277,7 +1233,6 @@ "resolved": "https://registry.npmjs.org/@smithy/util-waiter/-/util-waiter-4.2.5.tgz", "integrity": "sha512-Dbun99A3InifQdIrsXZ+QLcC0PGBPAdrl4cj1mTgJvyc9N2zf7QSxg8TBkzsCmGJdE3TLbO9ycwpY0EkWahQ/g==", "license": "Apache-2.0", - "peer": true, "dependencies": { "@smithy/abort-controller": "^4.2.5", "@smithy/types": "^4.9.0", @@ -1304,6 +1259,7 @@ "integrity": "sha512-NIy3oAFp9shda19hy4HK0HRTWKtPJmGdnvywu01nOqNC2vZg+Z+fvJDxpMQA88eb2I9EcafcdjYgsDthnYTvGw==", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "@jest/core": "^29.7.0", "@jest/types": "^29.6.3", @@ -1375,6 +1331,7 @@ "version": "29.7.0", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "@jest/core": "^29.7.0", "@jest/types": "^29.6.3", @@ -2119,6 +2076,7 @@ "node_modules/@aws-sdk/client-dynamodb": { "version": "3.914.0", "license": "Apache-2.0", + "peer": true, "dependencies": { "@aws-crypto/sha256-browser": "5.2.0", "@aws-crypto/sha256-js": "5.2.0", @@ -2761,6 +2719,7 @@ "node_modules/@aws-sdk/client-s3": { "version": "3.914.0", "license": "Apache-2.0", + "peer": true, "dependencies": { "@aws-crypto/sha1-browser": "5.2.0", "@aws-crypto/sha256-browser": "5.2.0", @@ -4101,6 +4060,7 @@ } ], "license": "MIT", + "peer": true, "engines": { "node": ">=18" }, @@ -4122,6 +4082,7 @@ } ], "license": "MIT", + "peer": true, "engines": { "node": ">=18" } @@ -5693,6 +5654,7 @@ "version": "30.2.0", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "@jest/environment": "30.2.0", "@jest/expect": "30.2.0", @@ -6009,6 +5971,7 @@ "version": "15.5.6", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "fast-glob": "3.3.1" } @@ -6968,7 +6931,9 @@ "license": "MIT" }, "node_modules/@tsconfig/node22": { - "version": "22.0.2", + "version": "22.0.5", + "resolved": "https://registry.npmjs.org/@tsconfig/node22/-/node22-22.0.5.tgz", + "integrity": "sha512-hLf2ld+sYN/BtOJjHUWOk568dvjFQkHnLNa6zce25GIH+vxKfvTgm3qpaH6ToF5tu/NN0IH66s+Bb5wElHrLcw==", "dev": true, "license": "MIT" }, @@ -7246,6 +7211,16 @@ "version": "7.0.15", "license": "MIT" }, + "node_modules/@types/json-schema-merge-allof": { + "version": "0.6.5", + "resolved": "https://registry.npmjs.org/@types/json-schema-merge-allof/-/json-schema-merge-allof-0.6.5.tgz", + "integrity": "sha512-5mS11ZUTyFNUVEMpK3uKoPb6BWL/nLgW/ln2VOiI8OOxKEYC4Gl9O3WjS5P49yqVTfkcbCAPKw3T1O4erUah5g==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/json-schema": "*" + } + }, "node_modules/@types/json5": { "version": "0.0.29", "dev": true, @@ -7646,6 +7621,7 @@ "version": "8.15.0", "dev": true, "license": "MIT", + "peer": true, "bin": { "acorn": "bin/acorn" }, @@ -8118,6 +8094,7 @@ "version": "4.1.0", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "@types/sinon": "^17.0.3", "sinon": "^18.0.1", @@ -8386,6 +8363,7 @@ } ], "license": "MIT", + "peer": true, "dependencies": { "baseline-browser-mapping": "^2.8.9", "caniuse-lite": "^1.0.30001746", @@ -8735,6 +8713,27 @@ "node": ">= 12.0.0" } }, + "node_modules/compute-gcd": { + "version": "1.2.1", + "resolved": "https://registry.npmjs.org/compute-gcd/-/compute-gcd-1.2.1.tgz", + "integrity": "sha512-TwMbxBNz0l71+8Sc4czv13h4kEqnchV9igQZBi6QUaz09dnz13juGnnaWWJTRsP3brxOoxeB4SA2WELLw1hCtg==", + "dependencies": { + "validate.io-array": "^1.0.3", + "validate.io-function": "^1.0.2", + "validate.io-integer-array": "^1.0.0" + } + }, + "node_modules/compute-lcm": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/compute-lcm/-/compute-lcm-1.1.2.tgz", + "integrity": "sha512-OFNPdQAXnQhDSKioX8/XYT6sdUlXwpeMjfd6ApxMJfyZ4GxmLR1xvMERctlYhlHwIiz6CSpBc2+qYKjHGZw4TQ==", + "dependencies": { + "compute-gcd": "^1.2.1", + "validate.io-array": "^1.0.3", + "validate.io-function": "^1.0.2", + "validate.io-integer-array": "^1.0.0" + } + }, "node_modules/concat-map": { "version": "0.0.1", "dev": true, @@ -9410,6 +9409,7 @@ "version": "9.37.0", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "@eslint-community/eslint-utils": "^4.8.0", "@eslint-community/regexpp": "^4.12.1", @@ -9603,6 +9603,7 @@ "version": "10.1.8", "dev": true, "license": "MIT", + "peer": true, "bin": { "eslint-config-prettier": "bin/cli.js" }, @@ -9658,6 +9659,7 @@ "version": "4.4.4", "dev": true, "license": "ISC", + "peer": true, "dependencies": { "debug": "^4.4.1", "eslint-import-context": "^0.1.8", @@ -9726,6 +9728,7 @@ "version": "2.32.0", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "@rtsao/scc": "^1.1.0", "array-includes": "^3.1.9", @@ -9758,6 +9761,7 @@ "version": "4.16.1", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "@typescript-eslint/types": "^8.35.0", "comment-parser": "^1.4.1", @@ -9865,6 +9869,7 @@ "version": "6.10.2", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "aria-query": "^5.3.2", "array-includes": "^3.1.8", @@ -9952,6 +9957,7 @@ "version": "7.37.5", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "array-includes": "^3.1.8", "array.prototype.findlast": "^1.2.5", @@ -9983,6 +9989,7 @@ "version": "5.2.0", "dev": true, "license": "MIT", + "peer": true, "engines": { "node": ">=10" }, @@ -11647,6 +11654,7 @@ "version": "30.2.0", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "@jest/core": "30.2.0", "@jest/types": "30.2.0", @@ -13828,6 +13836,7 @@ "version": "26.1.0", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "cssstyle": "^4.2.1", "data-urls": "^5.0.0", @@ -13865,6 +13874,7 @@ "node_modules/jsep": { "version": "1.4.0", "license": "MIT", + "peer": true, "engines": { "node": ">= 10.16.0" } @@ -13890,6 +13900,15 @@ "dev": true, "license": "MIT" }, + "node_modules/json-schema-compare": { + "version": "0.2.2", + "resolved": "https://registry.npmjs.org/json-schema-compare/-/json-schema-compare-0.2.2.tgz", + "integrity": "sha512-c4WYmDKyJXhs7WWvAWm3uIYnfyWFoIp+JEoX34rctVvEkMYCPGhXtvmFFXiffBbxfZsvQ0RNnV5H7GvDF5HCqQ==", + "license": "MIT", + "dependencies": { + "lodash": "^4.17.4" + } + }, "node_modules/json-schema-faker": { "version": "0.5.9", "license": "MIT", @@ -13930,6 +13949,20 @@ "ono": "^4.0.11" } }, + "node_modules/json-schema-merge-allof": { + "version": "0.8.1", + "resolved": "https://registry.npmjs.org/json-schema-merge-allof/-/json-schema-merge-allof-0.8.1.tgz", + "integrity": "sha512-CTUKmIlPJbsWfzRRnOXz+0MjIqvnleIXwFTzz+t9T86HnYX/Rozria6ZVGLktAU9e+NygNljveP+yxqtQp/Q4w==", + "license": "MIT", + "dependencies": { + "compute-lcm": "^1.1.2", + "json-schema-compare": "^0.2.2", + "lodash": "^4.17.20" + }, + "engines": { + "node": ">=12.0.0" + } + }, "node_modules/json-schema-migrate": { "version": "2.0.0", "dev": true, @@ -15372,6 +15405,10 @@ ], "license": "MIT" }, + "node_modules/python-schema-generator": { + "resolved": "src/python-schema-generator", + "link": true + }, "node_modules/qs": { "version": "6.14.0", "resolved": "https://registry.npmjs.org/qs/-/qs-6.14.0.tgz", @@ -16379,6 +16416,7 @@ "node_modules/tinyglobby/node_modules/picomatch": { "version": "4.0.2", "license": "MIT", + "peer": true, "engines": { "node": ">=12" }, @@ -16565,6 +16603,7 @@ "integrity": "sha512-f0FFpIdcHgn8zcPSbf1dRevwt047YMnaiJM3u2w2RewrB+fob/zePZcrOyQoLMMO7aBIddLcQIEK5dYjkLnGrQ==", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "@cspotcode/source-map-support": "^0.8.0", "@tsconfig/node10": "^1.0.7", @@ -16774,6 +16813,7 @@ "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==", "dev": true, "license": "Apache-2.0", + "peer": true, "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" @@ -16786,6 +16826,7 @@ "version": "8.46.1", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "@typescript-eslint/eslint-plugin": "8.46.1", "@typescript-eslint/parser": "8.46.1", @@ -16970,6 +17011,39 @@ "node": ">=10.12.0" } }, + "node_modules/validate.io-array": { + "version": "1.0.6", + "resolved": "https://registry.npmjs.org/validate.io-array/-/validate.io-array-1.0.6.tgz", + "integrity": "sha512-DeOy7CnPEziggrOO5CZhVKJw6S3Yi7e9e65R1Nl/RTN1vTQKnzjfvks0/8kQ40FP/dsjRAOd4hxmJ7uLa6vxkg==", + "license": "MIT" + }, + "node_modules/validate.io-function": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/validate.io-function/-/validate.io-function-1.0.2.tgz", + "integrity": "sha512-LlFybRJEriSuBnUhQyG5bwglhh50EpTL2ul23MPIuR1odjO7XaMLFV8vHGwp7AZciFxtYOeiSCT5st+XSPONiQ==" + }, + "node_modules/validate.io-integer": { + "version": "1.0.5", + "resolved": "https://registry.npmjs.org/validate.io-integer/-/validate.io-integer-1.0.5.tgz", + "integrity": "sha512-22izsYSLojN/P6bppBqhgUDjCkr5RY2jd+N2a3DCAUey8ydvrZ/OkGvFPR7qfOpwR2LC5p4Ngzxz36g5Vgr/hQ==", + "dependencies": { + "validate.io-number": "^1.0.3" + } + }, + "node_modules/validate.io-integer-array": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/validate.io-integer-array/-/validate.io-integer-array-1.0.0.tgz", + "integrity": "sha512-mTrMk/1ytQHtCY0oNO3dztafHYyGU88KL+jRxWuzfOmQb+4qqnWmI+gykvGp8usKZOM0H7keJHEbRaFiYA0VrA==", + "dependencies": { + "validate.io-array": "^1.0.3", + "validate.io-integer": "^1.0.4" + } + }, + "node_modules/validate.io-number": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/validate.io-number/-/validate.io-number-1.0.3.tgz", + "integrity": "sha512-kRAyotcbNaSYoDnXvb4MHg/0a1egJdLwS6oJ38TJY7aw9n93Fl/3blIXdyYvPOp55CNxywooG/3BcrwNrBpcSg==" + }, "node_modules/vscode-json-languageservice": { "version": "4.2.1", "dev": true, @@ -17481,6 +17555,252 @@ "integrity": "sha512-NM8/P9n3XjXhIZn1lLhkFaACTOURQXjWhV4BA/RnOv8xvgqtqpAX9IO4mRQxSx1Rlo4tqzeqb0sOlruaOy3dug==", "license": "MIT" }, + "src/python-schema-generator": { + "version": "1.0.0", + "dependencies": { + "json-schema-merge-allof": "^0.8.1", + "utils": "^0.0.1" + }, + "devDependencies": { + "@tsconfig/node22": "^22.0.5", + "@types/jest": "^29.5.14", + "@types/json-schema-merge-allof": "^0.6.5", + "@types/node": "^25.0.2", + "eslint": "^9.39.2", + "jest": "^29.7.0", + "mock-fs": "^5.5.0" + } + }, + "src/python-schema-generator/node_modules/@eslint/config-array": { + "version": "0.21.1", + "resolved": "https://registry.npmjs.org/@eslint/config-array/-/config-array-0.21.1.tgz", + "integrity": "sha512-aw1gNayWpdI/jSYVgzN5pL0cfzU02GT3NBpeT/DXbx1/1x7ZKxFPd9bwrzygx/qiwIQiJ1sw/zD8qY/kRvlGHA==", + "dev": true, + "license": "Apache-2.0", + "dependencies": { + "@eslint/object-schema": "^2.1.7", + "debug": "^4.3.1", + "minimatch": "^3.1.2" + }, + "engines": { + "node": "^18.18.0 || ^20.9.0 || >=21.1.0" + } + }, + "src/python-schema-generator/node_modules/@eslint/config-helpers": { + "version": "0.4.2", + "resolved": "https://registry.npmjs.org/@eslint/config-helpers/-/config-helpers-0.4.2.tgz", + "integrity": "sha512-gBrxN88gOIf3R7ja5K9slwNayVcZgK6SOUORm2uBzTeIEfeVaIhOpCtTox3P6R7o2jLFwLFTLnC7kU/RGcYEgw==", + "dev": true, + "license": "Apache-2.0", + "dependencies": { + "@eslint/core": "^0.17.0" + }, + "engines": { + "node": "^18.18.0 || ^20.9.0 || >=21.1.0" + } + }, + "src/python-schema-generator/node_modules/@eslint/core": { + "version": "0.17.0", + "resolved": "https://registry.npmjs.org/@eslint/core/-/core-0.17.0.tgz", + "integrity": "sha512-yL/sLrpmtDaFEiUj1osRP4TI2MDz1AddJL+jZ7KSqvBuliN4xqYY54IfdN8qD8Toa6g1iloph1fxQNkjOxrrpQ==", + "dev": true, + "license": "Apache-2.0", + "dependencies": { + "@types/json-schema": "^7.0.15" + }, + "engines": { + "node": "^18.18.0 || ^20.9.0 || >=21.1.0" + } + }, + "src/python-schema-generator/node_modules/@eslint/js": { + "version": "9.39.2", + "resolved": "https://registry.npmjs.org/@eslint/js/-/js-9.39.2.tgz", + "integrity": "sha512-q1mjIoW1VX4IvSocvM/vbTiveKC4k9eLrajNEuSsmjymSDEbpGddtpfOoN7YGAqBK3NG+uqo8ia4PDTt8buCYA==", + "dev": true, + "license": "MIT", + "engines": { + "node": "^18.18.0 || ^20.9.0 || >=21.1.0" + }, + "funding": { + "url": "https://eslint.org/donate" + } + }, + "src/python-schema-generator/node_modules/@eslint/plugin-kit": { + "version": "0.4.1", + "resolved": "https://registry.npmjs.org/@eslint/plugin-kit/-/plugin-kit-0.4.1.tgz", + "integrity": "sha512-43/qtrDUokr7LJqoF2c3+RInu/t4zfrpYdoSDfYyhg52rwLV6TnOvdG4fXm7IkSB3wErkcmJS9iEhjVtOSEjjA==", + "dev": true, + "license": "Apache-2.0", + "dependencies": { + "@eslint/core": "^0.17.0", + "levn": "^0.4.1" + }, + "engines": { + "node": "^18.18.0 || ^20.9.0 || >=21.1.0" + } + }, + "src/python-schema-generator/node_modules/@types/jest": { + "version": "29.5.14", + "resolved": "https://registry.npmjs.org/@types/jest/-/jest-29.5.14.tgz", + "integrity": "sha512-ZN+4sdnLUbo8EVvVc2ao0GFW6oVrQRPn4K2lglySj7APvSrgzxHiNNK99us4WDMi57xxA2yggblIAMNhXOotLQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "expect": "^29.0.0", + "pretty-format": "^29.0.0" + } + }, + "src/python-schema-generator/node_modules/@types/node": { + "version": "25.0.2", + "resolved": "https://registry.npmjs.org/@types/node/-/node-25.0.2.tgz", + "integrity": "sha512-gWEkeiyYE4vqjON/+Obqcoeffmk0NF15WSBwSs7zwVA2bAbTaE0SJ7P0WNGoJn8uE7fiaV5a7dKYIJriEqOrmA==", + "dev": true, + "license": "MIT", + "dependencies": { + "undici-types": "~7.16.0" + } + }, + "src/python-schema-generator/node_modules/brace-expansion": { + "version": "1.1.12", + "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.12.tgz", + "integrity": "sha512-9T9UjW3r0UW5c1Q7GTwllptXwhvYmEzFhzMfZ9H7FQWt+uZePjZPjBP/W1ZEyZ1twGWom5/56TF4lPcqjnDHcg==", + "dev": true, + "license": "MIT", + "dependencies": { + "balanced-match": "^1.0.0", + "concat-map": "0.0.1" + } + }, + "src/python-schema-generator/node_modules/eslint": { + "version": "9.39.2", + "resolved": "https://registry.npmjs.org/eslint/-/eslint-9.39.2.tgz", + "integrity": "sha512-LEyamqS7W5HB3ujJyvi0HQK/dtVINZvd5mAAp9eT5S/ujByGjiZLCzPcHVzuXbpJDJF/cxwHlfceVUDZ2lnSTw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@eslint-community/eslint-utils": "^4.8.0", + "@eslint-community/regexpp": "^4.12.1", + "@eslint/config-array": "^0.21.1", + "@eslint/config-helpers": "^0.4.2", + "@eslint/core": "^0.17.0", + "@eslint/eslintrc": "^3.3.1", + "@eslint/js": "9.39.2", + "@eslint/plugin-kit": "^0.4.1", + "@humanfs/node": "^0.16.6", + "@humanwhocodes/module-importer": "^1.0.1", + "@humanwhocodes/retry": "^0.4.2", + "@types/estree": "^1.0.6", + "ajv": "^6.12.4", + "chalk": "^4.0.0", + "cross-spawn": "^7.0.6", + "debug": "^4.3.2", + "escape-string-regexp": "^4.0.0", + "eslint-scope": "^8.4.0", + "eslint-visitor-keys": "^4.2.1", + "espree": "^10.4.0", + "esquery": "^1.5.0", + "esutils": "^2.0.2", + "fast-deep-equal": "^3.1.3", + "file-entry-cache": "^8.0.0", + "find-up": "^5.0.0", + "glob-parent": "^6.0.2", + "ignore": "^5.2.0", + "imurmurhash": "^0.1.4", + "is-glob": "^4.0.0", + "json-stable-stringify-without-jsonify": "^1.0.1", + "lodash.merge": "^4.6.2", + "minimatch": "^3.1.2", + "natural-compare": "^1.4.0", + "optionator": "^0.9.3" + }, + "bin": { + "eslint": "bin/eslint.js" + }, + "engines": { + "node": "^18.18.0 || ^20.9.0 || >=21.1.0" + }, + "funding": { + "url": "https://eslint.org/donate" + }, + "peerDependencies": { + "jiti": "*" + }, + "peerDependenciesMeta": { + "jiti": { + "optional": true + } + } + }, + "src/python-schema-generator/node_modules/eslint-visitor-keys": { + "version": "4.2.1", + "resolved": "https://registry.npmjs.org/eslint-visitor-keys/-/eslint-visitor-keys-4.2.1.tgz", + "integrity": "sha512-Uhdk5sfqcee/9H/rCOJikYz67o0a2Tw2hGRPOG2Y1R2dg7brRe1uG0yaNQDHu+TO/uQPF/5eCapvYSmHUjt7JQ==", + "dev": true, + "license": "Apache-2.0", + "engines": { + "node": "^18.18.0 || ^20.9.0 || >=21.1.0" + }, + "funding": { + "url": "https://opencollective.com/eslint" + } + }, + "src/python-schema-generator/node_modules/ignore": { + "version": "5.3.2", + "resolved": "https://registry.npmjs.org/ignore/-/ignore-5.3.2.tgz", + "integrity": "sha512-hsBTNUqQTDwkWtcdYI2i06Y/nUBEsNEDJKjWdigLvegy8kDuJAS8uRlpkkcQpyEXL0Z/pjDy5HBmMjRCJ2gq+g==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 4" + } + }, + "src/python-schema-generator/node_modules/jest": { + "version": "29.7.0", + "resolved": "https://registry.npmjs.org/jest/-/jest-29.7.0.tgz", + "integrity": "sha512-NIy3oAFp9shda19hy4HK0HRTWKtPJmGdnvywu01nOqNC2vZg+Z+fvJDxpMQA88eb2I9EcafcdjYgsDthnYTvGw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jest/core": "^29.7.0", + "@jest/types": "^29.6.3", + "import-local": "^3.0.2", + "jest-cli": "^29.7.0" + }, + "bin": { + "jest": "bin/jest.js" + }, + "engines": { + "node": "^14.15.0 || ^16.10.0 || >=18.0.0" + }, + "peerDependencies": { + "node-notifier": "^8.0.1 || ^9.0.0 || ^10.0.0" + }, + "peerDependenciesMeta": { + "node-notifier": { + "optional": true + } + } + }, + "src/python-schema-generator/node_modules/minimatch": { + "version": "3.1.2", + "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.1.2.tgz", + "integrity": "sha512-J7p63hRiAjw1NDEww1W7i37+ByIrOWO5XQQAzZ3VOcL0PNybwpfmV/N05zFAzwQ9USyEcX6t3UO+K5aqBQOIHw==", + "dev": true, + "license": "ISC", + "dependencies": { + "brace-expansion": "^1.1.7" + }, + "engines": { + "node": "*" + } + }, + "src/python-schema-generator/node_modules/undici-types": { + "version": "7.16.0", + "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-7.16.0.tgz", + "integrity": "sha512-Zz+aZWSj8LE6zoxD+xrjh4VfkIG8Ya6LvYkZqtUQGJPZjYl53ypCaUwWqo7eI0x66KBGeRo+mlBEkMSeSZ38Nw==", + "dev": true, + "license": "MIT" + }, "src/typescript-schema-generator": { "version": "0.0.1", "dependencies": { @@ -17783,6 +18103,7 @@ "resolved": "https://registry.npmjs.org/@aws-sdk/client-dynamodb/-/client-dynamodb-3.933.0.tgz", "integrity": "sha512-zRNDq5phdORYZnlof/p9inwm7B3TBwXWI6vPKzmYd+AmTMv/Ue4FQYsAcCX3JrUbTNXLK36CkaCgaH9/ydnnwg==", "license": "Apache-2.0", + "peer": true, "dependencies": { "@aws-crypto/sha256-browser": "5.2.0", "@aws-crypto/sha256-js": "5.2.0", @@ -19651,6 +19972,7 @@ "version": "29.7.0", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "@jest/core": "^29.7.0", "@jest/types": "^29.6.3", diff --git a/package.json b/package.json index df3f71100..dd7e71361 100644 --- a/package.json +++ b/package.json @@ -61,6 +61,7 @@ "utils/sender-management", "src/cloudevents", "src/digital-letters-events", + "src/python-schema-generator", "src/typescript-schema-generator", "tests/playwright" ] diff --git a/project.code-workspace b/project.code-workspace index 6aada36c8..b96948c2b 100644 --- a/project.code-workspace +++ b/project.code-workspace @@ -79,10 +79,10 @@ }, "terminal.integrated.scrollback": 10000, "jest.virtualFolders": [ - { "name": "key-generation", "rootPath": "lambdas/key-generation" }, { "name": "mesh-poll", "rootPath": "lambdas/mesh-poll" }, { "name": "refresh-apim-access-token", "rootPath": "lambdas/refresh-apim-access-token" }, + { "name": "python-schema-generator", "rootPath": "src/python-schema-generator" }, { "name": "ttl-create-lambda", "rootPath": "lambdas/ttl-create-lambda/" }, { "name": "ttl-handle-expiry-lambda", "rootPath": "lambdas/ttl-handle-expiry-lambda" }, { "name": "ttl-poll-lambda", "rootPath": "lambdas/ttl-poll-lambda" }, diff --git a/src/python-schema-generator/.gitignore b/src/python-schema-generator/.gitignore new file mode 100644 index 000000000..36a114ebb --- /dev/null +++ b/src/python-schema-generator/.gitignore @@ -0,0 +1 @@ +schemas diff --git a/src/python-schema-generator/Makefile b/src/python-schema-generator/Makefile index 4cc68a084..9ddf265ac 100644 --- a/src/python-schema-generator/Makefile +++ b/src/python-schema-generator/Makefile @@ -1,7 +1,8 @@ # Variables SCHEMA_SRC_DIR := ../../schemas/digital-letters/2025-10-draft/events OUTPUT_DIR := ../digital-letters-events/digital_letters_events/models -SCRIPTS_DIR := scripts +SRC_DIR := src +SCHEMAS_DIR := $(SRC_DIR)/schemas # Default target .PHONY: all clean generate install install-dev test coverage help @@ -10,9 +11,12 @@ all: generate # Install production dependencies install: - @echo "Installing production dependencies..." + @echo "Installing Python production dependencies..." @pip install -r requirements.txt @echo "Production dependencies installed!" + @echo "Installing Node production dependencies..." + @npm install + @echo "Node production dependencies installed!" # Install development dependencies install-dev: @@ -24,8 +28,9 @@ install-dev: generate: install @echo "Generating Pydantic models from JSON schemas..." @mkdir -p $(OUTPUT_DIR) - @python $(SCRIPTS_DIR)/generate_models.py \ - --input-dir $(SCHEMA_SRC_DIR) \ + @npm run merge + @python $(SRC_DIR)/generate_models.py \ + --input-dir $(SCHEMAS_DIR) \ --output-dir $(OUTPUT_DIR) @echo "Pydantic models generated in $(OUTPUT_DIR)/" @@ -50,6 +55,7 @@ coverage: install-dev clean: @echo "Cleaning generated models..." @rm -rf $(OUTPUT_DIR) htmlcov .pytest_cache coverage.xml + @rm -rf $(SCHEMAS_DIR) @echo "Clean complete!" # Show help diff --git a/src/python-schema-generator/README.md b/src/python-schema-generator/README.md index 1bbb001e3..a5bd3fdd8 100644 --- a/src/python-schema-generator/README.md +++ b/src/python-schema-generator/README.md @@ -104,7 +104,7 @@ if event_data.type == "uk.nhs.notify.digital.letters.letter.available.v1": The generator consists of: -- `scripts/generate_models.py` - Main CLI script -- `scripts/schema_processor.py` - Schema loading and processing -- `scripts/model_generator.py` - Pydantic model generation logic -- `scripts/file_utils.py` - File system utilities +- `src/generate_models.py` - Main CLI script +- `src/schema_processor.py` - Schema loading and processing +- `src/model_generator.py` - Pydantic model generation logic +- `src/file_utils.py` - File system utilities diff --git a/src/python-schema-generator/jest.config.ts b/src/python-schema-generator/jest.config.ts new file mode 100644 index 000000000..6e7e2d889 --- /dev/null +++ b/src/python-schema-generator/jest.config.ts @@ -0,0 +1,11 @@ +import { baseJestConfig } from '../../jest.config.base'; + +const config = { + ...baseJestConfig, + coveragePathIgnorePatterns: [ + ...(baseJestConfig.coveragePathIgnorePatterns ?? []), + 'src/merge-allof-cli.ts', + ], +}; + +export default config; diff --git a/src/python-schema-generator/package.json b/src/python-schema-generator/package.json new file mode 100644 index 000000000..6f3a18402 --- /dev/null +++ b/src/python-schema-generator/package.json @@ -0,0 +1,26 @@ +{ + "dependencies": { + "json-schema-merge-allof": "^0.8.1", + "utils": "^0.0.1" + }, + "description": "A tool to generate Python classes for the application's event schemas.", + "devDependencies": { + "@tsconfig/node22": "^22.0.5", + "@types/jest": "^29.5.14", + "@types/json-schema-merge-allof": "^0.6.5", + "@types/node": "^25.0.2", + "eslint": "^9.39.2", + "jest": "^29.7.0", + "mock-fs": "^5.5.0" + }, + "name": "python-schema-generator", + "private": true, + "scripts": { + "lint": "eslint scripts", + "lint:fix": "eslint scripts --fix", + "merge": "tsx src/merge-allof-cli.ts", + "test:unit": "jest", + "typecheck": "tsc --noEmit" + }, + "version": "1.0.0" +} diff --git a/src/python-schema-generator/scripts/__init__.py b/src/python-schema-generator/src/__init__.py similarity index 100% rename from src/python-schema-generator/scripts/__init__.py rename to src/python-schema-generator/src/__init__.py diff --git a/src/python-schema-generator/src/__tests__/merge-allof.test.ts b/src/python-schema-generator/src/__tests__/merge-allof.test.ts new file mode 100644 index 000000000..e1ba98e68 --- /dev/null +++ b/src/python-schema-generator/src/__tests__/merge-allof.test.ts @@ -0,0 +1,45 @@ +/* eslint-disable security/detect-non-literal-fs-filename */ + +import { eventSchemasDir } from 'file-utils'; +import { mergeAllOfInSchemas } from 'merge-allof'; +import mockFs from 'mock-fs'; +import { readdirSync } from 'node:fs'; +import path from 'node:path'; + +jest.mock('json-schema-to-typescript'); + +describe('merge-allof', () => { + const outputDir = path.resolve(__dirname, '..', 'schemas'); + + beforeEach(() => { + mockFs({ + [eventSchemasDir]: { + 'one.flattened.schema.json': '{"title": "One"}', + 'two.flattened.schema.json': '{"title": "Two"}', + 'three.flattened.schema.json': '{"title": "Three"}', + }, + }); + + jest.spyOn(console, 'log').mockImplementation(() => {}); + jest.spyOn(console, 'group').mockImplementation(() => {}); + }); + + afterEach(() => { + mockFs.restore(); + }); + + it('should generate a schema with merged allOfs for each schema', () => { + mergeAllOfInSchemas(); + + const mergedSchemas = readdirSync(outputDir); + + expect(mergedSchemas.length).toBe(3); + expect(mergedSchemas).toEqual( + expect.arrayContaining([ + 'one.flattened.schema.json', + 'two.flattened.schema.json', + 'three.flattened.schema.json', + ]), + ); + }); +}); diff --git a/src/python-schema-generator/src/file-utils.ts b/src/python-schema-generator/src/file-utils.ts new file mode 100644 index 000000000..7f3c5e8f5 --- /dev/null +++ b/src/python-schema-generator/src/file-utils.ts @@ -0,0 +1,68 @@ +// We don't accept user input, so path traversal attacks should not be a risk. +/* eslint-disable security/detect-non-literal-fs-filename */ + +import { mkdirSync, readFileSync, readdirSync, writeFileSync } from 'node:fs'; +import path from 'node:path'; + +export const eventSchemasDir = path.resolve( + __dirname, + '..', + '..', + '..', + 'schemas', + 'digital-letters', + '2025-10-draft', + 'events', +); + +export const destinationPackageName = 'digital-letters-events'; + +/** + * Lists all event schema filenames in the digital letters schemas directory. + * + * @returns An array of schema filenames. + */ +export function listEventSchemas(): string[] { + const flattenedSchemaFiles = readdirSync(eventSchemasDir).filter((f) => + f.endsWith('.flattened.schema.json'), + ); + + return flattenedSchemaFiles; +} + +/** + * Loads and parses a JSON schema from the specified file path. + * + * @param schemaPath The path to the JSON schema file. + * @returns The parsed JSON schema object. + */ +export function loadSchema(schemaPath: string): any { + return JSON.parse(readFileSync(schemaPath, 'utf8')); +} + +/** + * Creates the specified output directory if it doesn't exist. + * + * @param dirName The name of the directory to create. + * @returns The resolved path to the created directory. + */ +export function createOutputDir(dirName: string): string { + const outputDir = path.resolve(__dirname, dirName); + mkdirSync(outputDir, { recursive: true }); + return outputDir; +} + +/** + * Write a file with the specified content to the given path and filename. + * + * @param outputDir The directory the file will be written to. + * @param fileName The name of the file to write. + * @param content The content to write to the file. + */ +export function writeFile( + outputDir: string, + fileName: string, + content: string, +): void { + writeFileSync(path.join(outputDir, fileName), content); +} diff --git a/src/python-schema-generator/scripts/file_utils.py b/src/python-schema-generator/src/file_utils.py similarity index 100% rename from src/python-schema-generator/scripts/file_utils.py rename to src/python-schema-generator/src/file_utils.py diff --git a/src/python-schema-generator/scripts/generate_models.py b/src/python-schema-generator/src/generate_models.py similarity index 100% rename from src/python-schema-generator/scripts/generate_models.py rename to src/python-schema-generator/src/generate_models.py diff --git a/src/python-schema-generator/src/merge-allof-cli.ts b/src/python-schema-generator/src/merge-allof-cli.ts new file mode 100644 index 000000000..cb9fed7a2 --- /dev/null +++ b/src/python-schema-generator/src/merge-allof-cli.ts @@ -0,0 +1,3 @@ +import { mergeAllOfInSchemas } from 'merge-allof'; + +mergeAllOfInSchemas(); diff --git a/src/python-schema-generator/src/merge-allof.ts b/src/python-schema-generator/src/merge-allof.ts new file mode 100644 index 000000000..6a02a44d8 --- /dev/null +++ b/src/python-schema-generator/src/merge-allof.ts @@ -0,0 +1,34 @@ +/* eslint-disable no-console */ +import { + createOutputDir, + eventSchemasDir, + listEventSchemas, + loadSchema, + writeFile, +} from 'file-utils'; +import mergeAllOf from 'json-schema-merge-allof'; +import path from 'node:path'; + +export function mergeAllOfInSchemas(): void { + const eventSchemaFilenames = listEventSchemas(); + + const outputDir = createOutputDir('schemas'); + + console.log(`Output directory created at ${outputDir}`); + + console.group('Merging allOf entries in schemas:'); + for (const eventSchemaFilename of eventSchemaFilenames) { + const eventSchemaPath = path.join(eventSchemasDir, eventSchemaFilename); + const eventSchema = loadSchema(eventSchemaPath); + + const merged = mergeAllOf(eventSchema, { + resolvers: { + defaultResolver: mergeAllOf.options.resolvers.title, + }, + }); + + writeFile(outputDir, eventSchemaFilename, JSON.stringify(merged, null, 2)); + console.log(eventSchemaFilename); + } + console.groupEnd(); +} diff --git a/src/python-schema-generator/scripts/model_generator.py b/src/python-schema-generator/src/model_generator.py similarity index 100% rename from src/python-schema-generator/scripts/model_generator.py rename to src/python-schema-generator/src/model_generator.py diff --git a/src/python-schema-generator/scripts/schema_processor.py b/src/python-schema-generator/src/schema_processor.py similarity index 100% rename from src/python-schema-generator/scripts/schema_processor.py rename to src/python-schema-generator/src/schema_processor.py diff --git a/src/python-schema-generator/tests/test_file_utils.py b/src/python-schema-generator/tests/test_file_utils.py index 04155834b..98871ec00 100644 --- a/src/python-schema-generator/tests/test_file_utils.py +++ b/src/python-schema-generator/tests/test_file_utils.py @@ -4,7 +4,7 @@ import pytest -from scripts.file_utils import ( +from src.file_utils import ( list_json_schemas, load_json_schema, write_init_file, diff --git a/src/python-schema-generator/tests/test_model_generator.py b/src/python-schema-generator/tests/test_model_generator.py index 52b74563d..feb8f9742 100644 --- a/src/python-schema-generator/tests/test_model_generator.py +++ b/src/python-schema-generator/tests/test_model_generator.py @@ -4,13 +4,13 @@ import pytest -from scripts.model_generator import generate_pydantic_model +from src.model_generator import generate_pydantic_model class TestGeneratePydanticModel: """Tests for generate_pydantic_model function.""" - @patch("scripts.model_generator.subprocess.run") + @patch("src.model_generator.subprocess.run") def test_calls_datamodel_codegen_with_expected_arguments(self, mock_run): """Test successful model generation.""" # Arrange @@ -32,7 +32,7 @@ def test_calls_datamodel_codegen_with_expected_arguments(self, mock_run): assert "--class-name" in cmd_args[5] assert "TestModel" in cmd_args[6] - @patch("scripts.model_generator.subprocess.run") + @patch("src.model_generator.subprocess.run") def test_raises_error_on_generation_failure(self, mock_run): """Test that it raises error when generation fails.""" schema_path = "test_model.json" diff --git a/src/python-schema-generator/tests/test_schema_processor.py b/src/python-schema-generator/tests/test_schema_processor.py index 10dc39544..934595015 100644 --- a/src/python-schema-generator/tests/test_schema_processor.py +++ b/src/python-schema-generator/tests/test_schema_processor.py @@ -2,7 +2,7 @@ import pytest -from scripts.schema_processor import ( +from src.schema_processor import ( extract_event_type, extract_model_name, ) diff --git a/src/python-schema-generator/tsconfig.json b/src/python-schema-generator/tsconfig.json new file mode 100644 index 000000000..718b17e85 --- /dev/null +++ b/src/python-schema-generator/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "baseUrl": "./src/", + "isolatedModules": true, + "outDir": "dist" + }, + "exclude": [ + "node_modules" + ], + "extends": "@tsconfig/node22/tsconfig.json", + "include": [ + "src/**/*", + "./jest.config.ts" + ] +} From 4332daa388cac1090de15204c89b6bf260b1406e Mon Sep 17 00:00:00 2001 From: Gareth Allan <157592212+gareth-allan@users.noreply.github.com> Date: Mon, 15 Dec 2025 13:25:11 +0000 Subject: [PATCH 08/20] CCM-12896: Fix python-schema-generator lint command --- src/python-schema-generator/package.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/python-schema-generator/package.json b/src/python-schema-generator/package.json index 6f3a18402..223672a36 100644 --- a/src/python-schema-generator/package.json +++ b/src/python-schema-generator/package.json @@ -16,8 +16,8 @@ "name": "python-schema-generator", "private": true, "scripts": { - "lint": "eslint scripts", - "lint:fix": "eslint scripts --fix", + "lint": "eslint src", + "lint:fix": "eslint src --fix", "merge": "tsx src/merge-allof-cli.ts", "test:unit": "jest", "typecheck": "tsc --noEmit" From 479f1be48478a31b807f026a6b8bce695d8b5fd3 Mon Sep 17 00:00:00 2001 From: Gareth Allan <157592212+gareth-allan@users.noreply.github.com> Date: Mon, 15 Dec 2025 13:40:20 +0000 Subject: [PATCH 09/20] CCM-12896: Small Sonar fixes --- scripts/config/sonar-scanner.properties | 3 ++- src/python-schema-generator/src/file_utils.py | 3 ++- src/python-schema-generator/src/generate_models.py | 2 +- src/python-schema-generator/src/schema_processor.py | 2 +- 4 files changed, 6 insertions(+), 4 deletions(-) diff --git a/scripts/config/sonar-scanner.properties b/scripts/config/sonar-scanner.properties index f76bcf8ca..ad1341fb4 100644 --- a/scripts/config/sonar-scanner.properties +++ b/scripts/config/sonar-scanner.properties @@ -1,7 +1,8 @@ # Please DO NOT set the following properties `sonar.organization` and `sonar.projectKey` in this file. They must be stored as `SONAR_ORGANISATION_KEY` and `SONAR_PROJECT_KEY` GitHub secrets. sonar.host.url=https://sonarcloud.io -sonar.qualitygate.wait=true +# Temporary change to allow testing of build before fixing Sonar coverage issues +sonar.qualitygate.wait=false sonar.sourceEncoding=UTF-8 sonar.sources=. sonar.tests=tests/, src/asyncapigenerator/tests, src/cloudeventjekylldocs/tests, src/eventcatalogasyncapiimporter/tests, src/cloudevents/tools/builder/__tests__, src/cloudevents/tools/cache/__tests__, src/cloudevents/tools/generator/__tests__, lambdas/mesh-poll/src/__tests__, lambdas/ttl-create-lambda/src/__tests__, lambdas/ttl-poll-lambda/src/__tests__, utils/utils/src/__tests__, utils/sender-management/src/__tests__ diff --git a/src/python-schema-generator/src/file_utils.py b/src/python-schema-generator/src/file_utils.py index d63674b8c..904e4ddd8 100644 --- a/src/python-schema-generator/src/file_utils.py +++ b/src/python-schema-generator/src/file_utils.py @@ -73,7 +73,8 @@ def model_name_to_module_name(model_name: str) -> str: return "" # Handle acronym boundaries like "JSONSchema" -> "JSON_Schema" - step1 = re.sub(r"([A-Z]+)([A-Z][a-z])", r"\1_\2", model_name) + # Limited to acronyms of up to 10 letters to avoid catastrophic backtracking issues. + step1 = re.sub(r"([A-Z]{1,10})([A-Z][a-z])", r"\1_\2", model_name) # Insert underscores between lowercase/digit followed by uppercase: "fooBar" -> "foo_Bar" step2 = re.sub(r"([a-z0-9])([A-Z])", r"\1_\2", step1) diff --git a/src/python-schema-generator/src/generate_models.py b/src/python-schema-generator/src/generate_models.py index 66bdf9b55..73e8edfbe 100644 --- a/src/python-schema-generator/src/generate_models.py +++ b/src/python-schema-generator/src/generate_models.py @@ -76,7 +76,7 @@ def main() -> int: print(f" ✓ {output_filename}") write_init_file(args.output_dir, generated_models) - print(f" ✓ __init__.py") + print(" ✓ __init__.py") print(f"\nGeneration complete! Created {len(generated_models)} models ") return 0 diff --git a/src/python-schema-generator/src/schema_processor.py b/src/python-schema-generator/src/schema_processor.py index 8f784ec89..c55fcfc2b 100644 --- a/src/python-schema-generator/src/schema_processor.py +++ b/src/python-schema-generator/src/schema_processor.py @@ -20,7 +20,7 @@ def extract_model_name(schema: dict[str, Any]) -> str: raise ValueError("Schema does not contain a 'title' field") # Sanitize model name by removing spaces and special characters - sanitized_name = re.sub(r'[^a-zA-Z0-9_]', '', title) + sanitized_name = re.sub(r'\W', '', title) return sanitized_name From c2cf235fcc8d1f3b7794791255f5ef7e3cc97328 Mon Sep 17 00:00:00 2001 From: Gareth Allan <157592212+gareth-allan@users.noreply.github.com> Date: Mon, 15 Dec 2025 16:06:22 +0000 Subject: [PATCH 10/20] CCM-12896: Make python types build in the pipeline --- Makefile | 1 - package.json | 4 +++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index 9e5fb9c84..bf766b93b 100644 --- a/Makefile +++ b/Makefile @@ -14,7 +14,6 @@ dependencies: # Install dependencies needed to build and test the project @Pipel generate: # Generate any autogenerated output @Pipeline npm run generate-dependencies - $(MAKE) -C src/python-schema-generator generate build: # Build the project artefact @Pipeline $(MAKE) -C docs build diff --git a/package.json b/package.json index dd7e71361..df6a61d8b 100644 --- a/package.json +++ b/package.json @@ -42,8 +42,10 @@ "react": "^19.0.0" }, "scripts": { + "build-schemas": "make -C src/cloudevents/domains/digital-letters build-no-bundle publish-bundled-json", "clean": "npm run clean --workspaces --if-present", - "generate-dependencies": "make -C src/cloudevents/domains/digital-letters build-no-bundle publish-bundled-json && npm run generate-dependencies --workspaces --if-present", + "generate-dependencies": "npm run build-schemas && npm run generate-dependencies --workspaces --if-present && npm run generate-python-dependencies", + "generate-python-dependencies": "make -C src/python-schema-generator generate", "lint": "npm run lint --workspaces", "lint:fix": "npm run lint:fix --workspaces", "start": "npm run start --workspace frontend", From cff21940704b9d6cb5840e9f69b25595aeb667db Mon Sep 17 00:00:00 2001 From: Gareth Allan <157592212+gareth-allan@users.noreply.github.com> Date: Mon, 15 Dec 2025 16:14:06 +0000 Subject: [PATCH 11/20] CCM-12896: Extract a schema-utils utility --- package-lock.json | 4 +- .../requirements-dev.txt | 1 - src/python-schema-generator/requirements.txt | 2 - .../src/__tests__/merge-allof.test.ts | 2 +- src/python-schema-generator/src/file-utils.ts | 38 +-------------- .../src/merge-allof.ts | 9 +--- src/typescript-schema-generator/package.json | 3 +- .../src/__tests__/generate-types.test.ts | 2 +- .../src/__tests__/generate-validators.test.ts | 2 +- .../src/file-utils.ts | 39 +-------------- .../src/generate-types.ts | 10 +--- .../src/generate-validators.ts | 6 +-- utils/utils/package.json | 1 + .../schema-utils/schema-utils.test.ts | 47 +++++++++++++++++++ utils/utils/src/index.ts | 1 + utils/utils/src/schema-utils/index.ts | 1 + utils/utils/src/schema-utils/schema-utils.ts | 42 +++++++++++++++++ 17 files changed, 109 insertions(+), 101 deletions(-) create mode 100644 utils/utils/src/__tests__/schema-utils/schema-utils.test.ts create mode 100644 utils/utils/src/schema-utils/index.ts create mode 100644 utils/utils/src/schema-utils/schema-utils.ts diff --git a/package-lock.json b/package-lock.json index b0027d5ca..f4949bef3 100644 --- a/package-lock.json +++ b/package-lock.json @@ -17806,7 +17806,8 @@ "dependencies": { "ajv": "^8.17.1", "ajv-formats": "^3.0.1", - "json-schema-to-typescript": "^15.0.4" + "json-schema-to-typescript": "^15.0.4", + "utils": "^0.0.1" }, "devDependencies": { "@types/jest": "^29.5.14", @@ -19921,6 +19922,7 @@ "aws-sdk-client-mock-jest": "^4.1.0", "jest": "^29.7.0", "jest-mock-extended": "^3.0.7", + "mock-fs": "^5.5.0", "typescript": "^5.8.2" } }, diff --git a/src/python-schema-generator/requirements-dev.txt b/src/python-schema-generator/requirements-dev.txt index 248f1be10..72b8e3154 100644 --- a/src/python-schema-generator/requirements-dev.txt +++ b/src/python-schema-generator/requirements-dev.txt @@ -1,4 +1,3 @@ -r requirements.txt pytest>=7.4.0 pytest-cov>=4.1.0 -pytest-mock>=3.11.0 diff --git a/src/python-schema-generator/requirements.txt b/src/python-schema-generator/requirements.txt index 35a70ce5f..c04c6e0e7 100644 --- a/src/python-schema-generator/requirements.txt +++ b/src/python-schema-generator/requirements.txt @@ -1,3 +1 @@ datamodel-code-generator>=0.25.0,<1.0.0 -pydantic>=2.0.0,<3.0.0 -PyYAML>=6.0 diff --git a/src/python-schema-generator/src/__tests__/merge-allof.test.ts b/src/python-schema-generator/src/__tests__/merge-allof.test.ts index e1ba98e68..7dcc669ea 100644 --- a/src/python-schema-generator/src/__tests__/merge-allof.test.ts +++ b/src/python-schema-generator/src/__tests__/merge-allof.test.ts @@ -1,6 +1,6 @@ /* eslint-disable security/detect-non-literal-fs-filename */ -import { eventSchemasDir } from 'file-utils'; +import { eventSchemasDir } from 'utils'; import { mergeAllOfInSchemas } from 'merge-allof'; import mockFs from 'mock-fs'; import { readdirSync } from 'node:fs'; diff --git a/src/python-schema-generator/src/file-utils.ts b/src/python-schema-generator/src/file-utils.ts index 7f3c5e8f5..5acbc6f96 100644 --- a/src/python-schema-generator/src/file-utils.ts +++ b/src/python-schema-generator/src/file-utils.ts @@ -1,45 +1,9 @@ // We don't accept user input, so path traversal attacks should not be a risk. /* eslint-disable security/detect-non-literal-fs-filename */ -import { mkdirSync, readFileSync, readdirSync, writeFileSync } from 'node:fs'; +import { mkdirSync, writeFileSync } from 'node:fs'; import path from 'node:path'; -export const eventSchemasDir = path.resolve( - __dirname, - '..', - '..', - '..', - 'schemas', - 'digital-letters', - '2025-10-draft', - 'events', -); - -export const destinationPackageName = 'digital-letters-events'; - -/** - * Lists all event schema filenames in the digital letters schemas directory. - * - * @returns An array of schema filenames. - */ -export function listEventSchemas(): string[] { - const flattenedSchemaFiles = readdirSync(eventSchemasDir).filter((f) => - f.endsWith('.flattened.schema.json'), - ); - - return flattenedSchemaFiles; -} - -/** - * Loads and parses a JSON schema from the specified file path. - * - * @param schemaPath The path to the JSON schema file. - * @returns The parsed JSON schema object. - */ -export function loadSchema(schemaPath: string): any { - return JSON.parse(readFileSync(schemaPath, 'utf8')); -} - /** * Creates the specified output directory if it doesn't exist. * diff --git a/src/python-schema-generator/src/merge-allof.ts b/src/python-schema-generator/src/merge-allof.ts index 6a02a44d8..307c0656f 100644 --- a/src/python-schema-generator/src/merge-allof.ts +++ b/src/python-schema-generator/src/merge-allof.ts @@ -1,13 +1,8 @@ /* eslint-disable no-console */ -import { - createOutputDir, - eventSchemasDir, - listEventSchemas, - loadSchema, - writeFile, -} from 'file-utils'; +import { createOutputDir, writeFile } from 'file-utils'; import mergeAllOf from 'json-schema-merge-allof'; import path from 'node:path'; +import { eventSchemasDir, listEventSchemas, loadSchema } from 'utils'; export function mergeAllOfInSchemas(): void { const eventSchemaFilenames = listEventSchemas(); diff --git a/src/typescript-schema-generator/package.json b/src/typescript-schema-generator/package.json index 23b74e9e6..5059875e6 100644 --- a/src/typescript-schema-generator/package.json +++ b/src/typescript-schema-generator/package.json @@ -2,7 +2,8 @@ "dependencies": { "ajv": "^8.17.1", "ajv-formats": "^3.0.1", - "json-schema-to-typescript": "^15.0.4" + "json-schema-to-typescript": "^15.0.4", + "utils": "^0.0.1" }, "description": "A tool to generate Typescript types and standalone validator functions for the application's event schemas.", "devDependencies": { diff --git a/src/typescript-schema-generator/src/__tests__/generate-types.test.ts b/src/typescript-schema-generator/src/__tests__/generate-types.test.ts index 444f0cbd5..478d6363e 100644 --- a/src/typescript-schema-generator/src/__tests__/generate-types.test.ts +++ b/src/typescript-schema-generator/src/__tests__/generate-types.test.ts @@ -1,11 +1,11 @@ /* eslint-disable security/detect-non-literal-fs-filename */ -import { destinationPackageName, eventSchemasDir } from 'file-utils'; import { generateTypes } from 'generate-types'; import { compile } from 'json-schema-to-typescript'; import mockFs from 'mock-fs'; import { readFileSync, readdirSync } from 'node:fs'; import path from 'node:path'; +import { destinationPackageName, eventSchemasDir } from 'utils'; jest.mock('json-schema-to-typescript'); diff --git a/src/typescript-schema-generator/src/__tests__/generate-validators.test.ts b/src/typescript-schema-generator/src/__tests__/generate-validators.test.ts index 00b7467d0..b4612577c 100644 --- a/src/typescript-schema-generator/src/__tests__/generate-validators.test.ts +++ b/src/typescript-schema-generator/src/__tests__/generate-validators.test.ts @@ -1,10 +1,10 @@ /* eslint-disable security/detect-non-literal-fs-filename */ import standaloneCode from 'ajv/dist/standalone'; -import { destinationPackageName, eventSchemasDir } from 'file-utils'; import { generateValidators } from 'generate-validators'; import mockFs from 'mock-fs'; import { readFileSync, readdirSync } from 'node:fs'; import path from 'node:path'; +import { destinationPackageName, eventSchemasDir } from 'utils'; jest.mock('ajv/dist/2020'); jest.mock('ajv/dist/standalone'); diff --git a/src/typescript-schema-generator/src/file-utils.ts b/src/typescript-schema-generator/src/file-utils.ts index f325cfb8c..d6e24f8e9 100644 --- a/src/typescript-schema-generator/src/file-utils.ts +++ b/src/typescript-schema-generator/src/file-utils.ts @@ -1,44 +1,9 @@ // We don't accept user input, so path traversal attacks should not be a risk. /* eslint-disable security/detect-non-literal-fs-filename */ -import { mkdirSync, readFileSync, readdirSync, writeFileSync } from 'node:fs'; +import { mkdirSync, writeFileSync } from 'node:fs'; import path from 'node:path'; - -export const eventSchemasDir = path.resolve( - __dirname, - '..', - '..', - '..', - 'schemas', - 'digital-letters', - '2025-10-draft', - 'events', -); - -export const destinationPackageName = 'digital-letters-events'; - -/** - * Lists all event schema filenames in the digital letters schemas directory. - * - * @returns An array of schema filenames. - */ -export function listEventSchemas(): string[] { - const flattenedSchemaFiles = readdirSync(eventSchemasDir).filter((f) => - f.endsWith('.flattened.schema.json'), - ); - - return flattenedSchemaFiles; -} - -/** - * Loads and parses a JSON schema from the specified file path. - * - * @param schemaPath The path to the JSON schema file. - * @returns The parsed JSON schema object. - */ -export function loadSchema(schemaPath: string): any { - return JSON.parse(readFileSync(schemaPath, 'utf8')); -} +import { destinationPackageName } from 'utils'; /** * Creates the specified output directory if it doesn't exist. diff --git a/src/typescript-schema-generator/src/generate-types.ts b/src/typescript-schema-generator/src/generate-types.ts index e84f0772b..81cc10a2f 100644 --- a/src/typescript-schema-generator/src/generate-types.ts +++ b/src/typescript-schema-generator/src/generate-types.ts @@ -1,14 +1,8 @@ /* eslint-disable no-console */ +import { createOutputDir, writeFile, writeTypesIndex } from 'file-utils'; import { compile } from 'json-schema-to-typescript'; import path from 'node:path'; -import { - createOutputDir, - eventSchemasDir, - listEventSchemas, - loadSchema, - writeFile, - writeTypesIndex, -} from 'file-utils'; +import { eventSchemasDir, listEventSchemas, loadSchema } from 'utils'; export async function generateTypes() { const eventSchemaFilenames = listEventSchemas(); diff --git a/src/typescript-schema-generator/src/generate-validators.ts b/src/typescript-schema-generator/src/generate-validators.ts index 662dd58b8..869ac050b 100644 --- a/src/typescript-schema-generator/src/generate-validators.ts +++ b/src/typescript-schema-generator/src/generate-validators.ts @@ -5,15 +5,13 @@ import path from 'node:path'; import Ajv from 'ajv/dist/2020'; import standaloneCode from 'ajv/dist/standalone'; +import { createOutputDir, writeFile, writeTypesIndex } from 'file-utils'; import { - createOutputDir, destinationPackageName, eventSchemasDir, listEventSchemas, loadSchema, - writeFile, - writeTypesIndex, -} from 'file-utils'; +} from 'utils'; export function generateValidators() { const ajv = new Ajv({ diff --git a/utils/utils/package.json b/utils/utils/package.json index 9b37e42e6..112ae9bb5 100644 --- a/utils/utils/package.json +++ b/utils/utils/package.json @@ -24,6 +24,7 @@ "aws-sdk-client-mock-jest": "^4.1.0", "jest": "^29.7.0", "jest-mock-extended": "^3.0.7", + "mock-fs": "^5.5.0", "typescript": "^5.8.2" }, "exports": { diff --git a/utils/utils/src/__tests__/schema-utils/schema-utils.test.ts b/utils/utils/src/__tests__/schema-utils/schema-utils.test.ts new file mode 100644 index 000000000..a43a4774e --- /dev/null +++ b/utils/utils/src/__tests__/schema-utils/schema-utils.test.ts @@ -0,0 +1,47 @@ +import mockFs from 'mock-fs'; +import path from 'node:path'; +import { eventSchemasDir, listEventSchemas, loadSchema } from 'utils'; + +describe('schema-utils', () => { + beforeEach(() => { + mockFs({ + [eventSchemasDir]: { + 'one.flattened.schema.json': '{"title": "One"}', + 'two.schema.json': '{"title": "Two"}', + 'three.flattened.schema.json': '{"title": "Three"}', + 'four.flattened.schema.txt': '{"title": "Four"}', + }, + }); + }); + + afterEach(() => { + mockFs.restore(); + }); + + describe('listEventSchemas', () => { + it('should list all flattened schemas', () => { + const schemas = listEventSchemas(); + + expect(schemas.length).toBe(2); + expect(schemas).toEqual( + expect.arrayContaining([ + 'one.flattened.schema.json', + 'three.flattened.schema.json', + ]), + ); + }); + }); + + describe('loadSchema', () => { + it('should load and parse a JSON schema', () => { + const schemaPath = path.resolve( + eventSchemasDir, + 'one.flattened.schema.json', + ); + + const schema = loadSchema(schemaPath); + + expect(schema).toEqual({ title: 'One' }); + }); + }); +}); diff --git a/utils/utils/src/index.ts b/utils/utils/src/index.ts index 96bc77808..46ef7231a 100644 --- a/utils/utils/src/index.ts +++ b/utils/utils/src/index.ts @@ -13,3 +13,4 @@ export * from './types'; export * from './event-publisher'; export * from './event-bridge-utils'; export * from './key-generation-utils'; +export * from './schema-utils'; diff --git a/utils/utils/src/schema-utils/index.ts b/utils/utils/src/schema-utils/index.ts new file mode 100644 index 000000000..df11652bc --- /dev/null +++ b/utils/utils/src/schema-utils/index.ts @@ -0,0 +1 @@ +export * from './schema-utils'; diff --git a/utils/utils/src/schema-utils/schema-utils.ts b/utils/utils/src/schema-utils/schema-utils.ts new file mode 100644 index 000000000..e8562cf33 --- /dev/null +++ b/utils/utils/src/schema-utils/schema-utils.ts @@ -0,0 +1,42 @@ +// We don't accept user input, so path traversal attacks should not be a risk. +/* eslint-disable security/detect-non-literal-fs-filename */ + +import { readFileSync, readdirSync } from 'node:fs'; +import path from 'node:path'; + +export const eventSchemasDir = path.resolve( + __dirname, + '..', + '..', + '..', + '..', + 'schemas', + 'digital-letters', + '2025-10-draft', + 'events', +); + +export const destinationPackageName = 'digital-letters-events'; + +/** + * Lists all event schema filenames in the digital letters schemas directory. + * + * @returns An array of schema filenames. + */ +export function listEventSchemas(): string[] { + const flattenedSchemaFiles = readdirSync(eventSchemasDir).filter((f) => + f.endsWith('.flattened.schema.json'), + ); + + return flattenedSchemaFiles; +} + +/** + * Loads and parses a JSON schema from the specified file path. + * + * @param schemaPath The path to the JSON schema file. + * @returns The parsed JSON schema object. + */ +export function loadSchema(schemaPath: string): any { + return JSON.parse(readFileSync(schemaPath, 'utf8')); +} From 607d6a5fe77ab833b90153c232447cf55284be81 Mon Sep 17 00:00:00 2001 From: Gareth Allan <157592212+gareth-allan@users.noreply.github.com> Date: Mon, 15 Dec 2025 16:37:44 +0000 Subject: [PATCH 12/20] CCM-12896: Changes following self-review --- package-lock.json | 494 +++++++++++++++++- src/digital-letters-events/README.md | 26 +- src/python-schema-generator/package.json | 4 +- .../src/merge-allof.ts | 1 - .../src/__tests__/generate-types.test.ts | 3 +- .../src/__tests__/generate-validators.test.ts | 3 +- .../src/file-utils.ts | 3 +- .../src/generate-validators.ts | 10 +- utils/utils/src/schema-utils/schema-utils.ts | 2 - 9 files changed, 528 insertions(+), 18 deletions(-) diff --git a/package-lock.json b/package-lock.json index f4949bef3..04ccd2e26 100644 --- a/package-lock.json +++ b/package-lock.json @@ -17559,6 +17559,7 @@ "version": "1.0.0", "dependencies": { "json-schema-merge-allof": "^0.8.1", + "tsx": "^4.21.0", "utils": "^0.0.1" }, "devDependencies": { @@ -17568,7 +17569,424 @@ "@types/node": "^25.0.2", "eslint": "^9.39.2", "jest": "^29.7.0", - "mock-fs": "^5.5.0" + "mock-fs": "^5.5.0", + "typescript": "^5.9.3" + } + }, + "src/python-schema-generator/node_modules/@esbuild/aix-ppc64": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.27.1.tgz", + "integrity": "sha512-HHB50pdsBX6k47S4u5g/CaLjqS3qwaOVE5ILsq64jyzgMhLuCuZ8rGzM9yhsAjfjkbgUPMzZEPa7DAp7yz6vuA==", + "cpu": [ + "ppc64" + ], + "license": "MIT", + "optional": true, + "os": [ + "aix" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/android-arm": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.27.1.tgz", + "integrity": "sha512-kFqa6/UcaTbGm/NncN9kzVOODjhZW8e+FRdSeypWe6j33gzclHtwlANs26JrupOntlcWmB0u8+8HZo8s7thHvg==", + "cpu": [ + "arm" + ], + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/android-arm64": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.27.1.tgz", + "integrity": "sha512-45fuKmAJpxnQWixOGCrS+ro4Uvb4Re9+UTieUY2f8AEc+t7d4AaZ6eUJ3Hva7dtrxAAWHtlEFsXFMAgNnGU9uQ==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/android-x64": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.27.1.tgz", + "integrity": "sha512-LBEpOz0BsgMEeHgenf5aqmn/lLNTFXVfoWMUox8CtWWYK9X4jmQzWjoGoNb8lmAYml/tQ/Ysvm8q7szu7BoxRQ==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/darwin-arm64": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.27.1.tgz", + "integrity": "sha512-veg7fL8eMSCVKL7IW4pxb54QERtedFDfY/ASrumK/SbFsXnRazxY4YykN/THYqFnFwJ0aVjiUrVG2PwcdAEqQQ==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/darwin-x64": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.27.1.tgz", + "integrity": "sha512-+3ELd+nTzhfWb07Vol7EZ+5PTbJ/u74nC6iv4/lwIU99Ip5uuY6QoIf0Hn4m2HoV0qcnRivN3KSqc+FyCHjoVQ==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/freebsd-arm64": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.27.1.tgz", + "integrity": "sha512-/8Rfgns4XD9XOSXlzUDepG8PX+AVWHliYlUkFI3K3GB6tqbdjYqdhcb4BKRd7C0BhZSoaCxhv8kTcBrcZWP+xg==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/freebsd-x64": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.27.1.tgz", + "integrity": "sha512-GITpD8dK9C+r+5yRT/UKVT36h/DQLOHdwGVwwoHidlnA168oD3uxA878XloXebK4Ul3gDBBIvEdL7go9gCUFzQ==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/linux-arm": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.27.1.tgz", + "integrity": "sha512-ieMID0JRZY/ZeCrsFQ3Y3NlHNCqIhTprJfDgSB3/lv5jJZ8FX3hqPyXWhe+gvS5ARMBJ242PM+VNz/ctNj//eA==", + "cpu": [ + "arm" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/linux-arm64": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.27.1.tgz", + "integrity": "sha512-W9//kCrh/6in9rWIBdKaMtuTTzNj6jSeG/haWBADqLLa9P8O5YSRDzgD5y9QBok4AYlzS6ARHifAb75V6G670Q==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/linux-ia32": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.27.1.tgz", + "integrity": "sha512-VIUV4z8GD8rtSVMfAj1aXFahsi/+tcoXXNYmXgzISL+KB381vbSTNdeZHHHIYqFyXcoEhu9n5cT+05tRv13rlw==", + "cpu": [ + "ia32" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/linux-loong64": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.27.1.tgz", + "integrity": "sha512-l4rfiiJRN7sTNI//ff65zJ9z8U+k6zcCg0LALU5iEWzY+a1mVZ8iWC1k5EsNKThZ7XCQ6YWtsZ8EWYm7r1UEsg==", + "cpu": [ + "loong64" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/linux-mips64el": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.27.1.tgz", + "integrity": "sha512-U0bEuAOLvO/DWFdygTHWY8C067FXz+UbzKgxYhXC0fDieFa0kDIra1FAhsAARRJbvEyso8aAqvPdNxzWuStBnA==", + "cpu": [ + "mips64el" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/linux-ppc64": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.27.1.tgz", + "integrity": "sha512-NzdQ/Xwu6vPSf/GkdmRNsOfIeSGnh7muundsWItmBsVpMoNPVpM61qNzAVY3pZ1glzzAxLR40UyYM23eaDDbYQ==", + "cpu": [ + "ppc64" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/linux-riscv64": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.27.1.tgz", + "integrity": "sha512-7zlw8p3IApcsN7mFw0O1Z1PyEk6PlKMu18roImfl3iQHTnr/yAfYv6s4hXPidbDoI2Q0pW+5xeoM4eTCC0UdrQ==", + "cpu": [ + "riscv64" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/linux-s390x": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.27.1.tgz", + "integrity": "sha512-cGj5wli+G+nkVQdZo3+7FDKC25Uh4ZVwOAK6A06Hsvgr8WqBBuOy/1s+PUEd/6Je+vjfm6stX0kmib5b/O2Ykw==", + "cpu": [ + "s390x" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/linux-x64": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.27.1.tgz", + "integrity": "sha512-z3H/HYI9MM0HTv3hQZ81f+AKb+yEoCRlUby1F80vbQ5XdzEMyY/9iNlAmhqiBKw4MJXwfgsh7ERGEOhrM1niMA==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/netbsd-arm64": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.27.1.tgz", + "integrity": "sha512-wzC24DxAvk8Em01YmVXyjl96Mr+ecTPyOuADAvjGg+fyBpGmxmcr2E5ttf7Im8D0sXZihpxzO1isus8MdjMCXQ==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/netbsd-x64": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.27.1.tgz", + "integrity": "sha512-1YQ8ybGi2yIXswu6eNzJsrYIGFpnlzEWRl6iR5gMgmsrR0FcNoV1m9k9sc3PuP5rUBLshOZylc9nqSgymI+TYg==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/openbsd-arm64": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.27.1.tgz", + "integrity": "sha512-5Z+DzLCrq5wmU7RDaMDe2DVXMRm2tTDvX2KU14JJVBN2CT/qov7XVix85QoJqHltpvAOZUAc3ndU56HSMWrv8g==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/openbsd-x64": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.27.1.tgz", + "integrity": "sha512-Q73ENzIdPF5jap4wqLtsfh8YbYSZ8Q0wnxplOlZUOyZy7B4ZKW8DXGWgTCZmF8VWD7Tciwv5F4NsRf6vYlZtqg==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/openharmony-arm64": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.27.1.tgz", + "integrity": "sha512-ajbHrGM/XiK+sXM0JzEbJAen+0E+JMQZ2l4RR4VFwvV9JEERx+oxtgkpoKv1SevhjavK2z2ReHk32pjzktWbGg==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/sunos-x64": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.27.1.tgz", + "integrity": "sha512-IPUW+y4VIjuDVn+OMzHc5FV4GubIwPnsz6ubkvN8cuhEqH81NovB53IUlrlBkPMEPxvNnf79MGBoz8rZ2iW8HA==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "sunos" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/win32-arm64": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.27.1.tgz", + "integrity": "sha512-RIVRWiljWA6CdVu8zkWcRmGP7iRRIIwvhDKem8UMBjPql2TXM5PkDVvvrzMtj1V+WFPB4K7zkIGM7VzRtFkjdg==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/win32-ia32": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.27.1.tgz", + "integrity": "sha512-2BR5M8CPbptC1AK5JbJT1fWrHLvejwZidKx3UMSF0ecHMa+smhi16drIrCEggkgviBwLYd5nwrFLSl5Kho96RQ==", + "cpu": [ + "ia32" + ], + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "src/python-schema-generator/node_modules/@esbuild/win32-x64": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.27.1.tgz", + "integrity": "sha512-d5X6RMYv6taIymSk8JBP+nxv8DQAMY6A51GPgusqLdK9wBz5wWIXy1KjTck6HnjE9hqJzJRdk+1p/t5soSbCtw==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" } }, "src/python-schema-generator/node_modules/@eslint/config-array": { @@ -17671,6 +18089,47 @@ "concat-map": "0.0.1" } }, + "src/python-schema-generator/node_modules/esbuild": { + "version": "0.27.1", + "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.27.1.tgz", + "integrity": "sha512-yY35KZckJJuVVPXpvjgxiCuVEJT67F6zDeVTv4rizyPrfGBUpZQsvmxnN+C371c2esD/hNMjj4tpBhuueLN7aA==", + "hasInstallScript": true, + "license": "MIT", + "bin": { + "esbuild": "bin/esbuild" + }, + "engines": { + "node": ">=18" + }, + "optionalDependencies": { + "@esbuild/aix-ppc64": "0.27.1", + "@esbuild/android-arm": "0.27.1", + "@esbuild/android-arm64": "0.27.1", + "@esbuild/android-x64": "0.27.1", + "@esbuild/darwin-arm64": "0.27.1", + "@esbuild/darwin-x64": "0.27.1", + "@esbuild/freebsd-arm64": "0.27.1", + "@esbuild/freebsd-x64": "0.27.1", + "@esbuild/linux-arm": "0.27.1", + "@esbuild/linux-arm64": "0.27.1", + "@esbuild/linux-ia32": "0.27.1", + "@esbuild/linux-loong64": "0.27.1", + "@esbuild/linux-mips64el": "0.27.1", + "@esbuild/linux-ppc64": "0.27.1", + "@esbuild/linux-riscv64": "0.27.1", + "@esbuild/linux-s390x": "0.27.1", + "@esbuild/linux-x64": "0.27.1", + "@esbuild/netbsd-arm64": "0.27.1", + "@esbuild/netbsd-x64": "0.27.1", + "@esbuild/openbsd-arm64": "0.27.1", + "@esbuild/openbsd-x64": "0.27.1", + "@esbuild/openharmony-arm64": "0.27.1", + "@esbuild/sunos-x64": "0.27.1", + "@esbuild/win32-arm64": "0.27.1", + "@esbuild/win32-ia32": "0.27.1", + "@esbuild/win32-x64": "0.27.1" + } + }, "src/python-schema-generator/node_modules/eslint": { "version": "9.39.2", "resolved": "https://registry.npmjs.org/eslint/-/eslint-9.39.2.tgz", @@ -17744,6 +18203,20 @@ "url": "https://opencollective.com/eslint" } }, + "src/python-schema-generator/node_modules/fsevents": { + "version": "2.3.3", + "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz", + "integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==", + "hasInstallScript": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": "^8.16.0 || ^10.6.0 || >=11.0.0" + } + }, "src/python-schema-generator/node_modules/ignore": { "version": "5.3.2", "resolved": "https://registry.npmjs.org/ignore/-/ignore-5.3.2.tgz", @@ -17794,6 +18267,25 @@ "node": "*" } }, + "src/python-schema-generator/node_modules/tsx": { + "version": "4.21.0", + "resolved": "https://registry.npmjs.org/tsx/-/tsx-4.21.0.tgz", + "integrity": "sha512-5C1sg4USs1lfG0GFb2RLXsdpXqBSEhAaA/0kPL01wxzpMqLILNxIxIOKiILz+cdg/pLnOUxFYOR5yhHU666wbw==", + "license": "MIT", + "dependencies": { + "esbuild": "~0.27.0", + "get-tsconfig": "^4.7.5" + }, + "bin": { + "tsx": "dist/cli.mjs" + }, + "engines": { + "node": ">=18.0.0" + }, + "optionalDependencies": { + "fsevents": "~2.3.3" + } + }, "src/python-schema-generator/node_modules/undici-types": { "version": "7.16.0", "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-7.16.0.tgz", diff --git a/src/digital-letters-events/README.md b/src/digital-letters-events/README.md index c3b7a45fd..4bec683b4 100644 --- a/src/digital-letters-events/README.md +++ b/src/digital-letters-events/README.md @@ -100,11 +100,26 @@ try: "dataschema": "https://notify.nhs.uk/cloudevents/schemas/digital-letters/2025-10-draft/data/digital-letters-pdm-resource-submitted-data.schema.json", "specversion": "1.0", "id": "0249e529-f947-4012-819e-b634eb71be79", - "subject": "pdm-resource-123", - "time": "2025-12-11T10:00:00Z", + "subject": "customer/7ff8ed41-cd5f-20e4-ef4e-34f96d8cc8ac/75027ace-9b8c-bcfe-866e-6c24242cffc3/q58dnxk5e/4cbek805wwx/yiaw7bl0d/her/1ccb7eb8-c6fe-0a42-279a-2a0e48ff1ca9/zk", + "time": "2025-11-21T16:01:52.268Z", + "datacontenttype": "application/json", + "traceparent": "00-ee4790eb6821064c645406abe918b3da-3a4e6957ce2a15de-01", + "tracestate": "nisi quis", + "partitionkey": "customer-7ff8ed41", + "recordedtime": "2025-11-21T16:01:53.268Z", + "sampledrate": 1, + "sequence": "00000000000350773861", + "severitytext": "INFO", + "severitynumber": 2, + "dataclassification": "restricted", + "dataregulation": "ISO-27001", + "datacategory": "non-sensitive", "data": { - "something": "example value" - } + "messageReference": "incididunt Ut aute laborum", + "senderId": "officia voluptate culpa Ut dolor", + "resourceId": "a2bcbb42-ab7e-42b6-88d6-74f8d3ca4a09", + "retryCount": 97_903_257, + }, } # Create and validate the event @@ -113,7 +128,8 @@ try: # Access validated fields print(event.id) print(event.type) - print(event.data.something) + print(event.data.messageReference) except Exception as e: print(e) + raise ValueError("Error processing event") from e ``` diff --git a/src/python-schema-generator/package.json b/src/python-schema-generator/package.json index 223672a36..b766933eb 100644 --- a/src/python-schema-generator/package.json +++ b/src/python-schema-generator/package.json @@ -1,6 +1,7 @@ { "dependencies": { "json-schema-merge-allof": "^0.8.1", + "tsx": "^4.21.0", "utils": "^0.0.1" }, "description": "A tool to generate Python classes for the application's event schemas.", @@ -11,7 +12,8 @@ "@types/node": "^25.0.2", "eslint": "^9.39.2", "jest": "^29.7.0", - "mock-fs": "^5.5.0" + "mock-fs": "^5.5.0", + "typescript": "^5.9.3" }, "name": "python-schema-generator", "private": true, diff --git a/src/python-schema-generator/src/merge-allof.ts b/src/python-schema-generator/src/merge-allof.ts index 307c0656f..dd5d02fee 100644 --- a/src/python-schema-generator/src/merge-allof.ts +++ b/src/python-schema-generator/src/merge-allof.ts @@ -8,7 +8,6 @@ export function mergeAllOfInSchemas(): void { const eventSchemaFilenames = listEventSchemas(); const outputDir = createOutputDir('schemas'); - console.log(`Output directory created at ${outputDir}`); console.group('Merging allOf entries in schemas:'); diff --git a/src/typescript-schema-generator/src/__tests__/generate-types.test.ts b/src/typescript-schema-generator/src/__tests__/generate-types.test.ts index 478d6363e..153fc005f 100644 --- a/src/typescript-schema-generator/src/__tests__/generate-types.test.ts +++ b/src/typescript-schema-generator/src/__tests__/generate-types.test.ts @@ -1,11 +1,12 @@ /* eslint-disable security/detect-non-literal-fs-filename */ +import { destinationPackageName } from 'file-utils'; import { generateTypes } from 'generate-types'; import { compile } from 'json-schema-to-typescript'; import mockFs from 'mock-fs'; import { readFileSync, readdirSync } from 'node:fs'; import path from 'node:path'; -import { destinationPackageName, eventSchemasDir } from 'utils'; +import { eventSchemasDir } from 'utils'; jest.mock('json-schema-to-typescript'); diff --git a/src/typescript-schema-generator/src/__tests__/generate-validators.test.ts b/src/typescript-schema-generator/src/__tests__/generate-validators.test.ts index b4612577c..639cb8454 100644 --- a/src/typescript-schema-generator/src/__tests__/generate-validators.test.ts +++ b/src/typescript-schema-generator/src/__tests__/generate-validators.test.ts @@ -1,10 +1,11 @@ /* eslint-disable security/detect-non-literal-fs-filename */ import standaloneCode from 'ajv/dist/standalone'; +import { destinationPackageName } from 'file-utils'; import { generateValidators } from 'generate-validators'; import mockFs from 'mock-fs'; import { readFileSync, readdirSync } from 'node:fs'; import path from 'node:path'; -import { destinationPackageName, eventSchemasDir } from 'utils'; +import { eventSchemasDir } from 'utils'; jest.mock('ajv/dist/2020'); jest.mock('ajv/dist/standalone'); diff --git a/src/typescript-schema-generator/src/file-utils.ts b/src/typescript-schema-generator/src/file-utils.ts index d6e24f8e9..4fedca54b 100644 --- a/src/typescript-schema-generator/src/file-utils.ts +++ b/src/typescript-schema-generator/src/file-utils.ts @@ -3,7 +3,8 @@ import { mkdirSync, writeFileSync } from 'node:fs'; import path from 'node:path'; -import { destinationPackageName } from 'utils'; + +export const destinationPackageName = 'digital-letters-events'; /** * Creates the specified output directory if it doesn't exist. diff --git a/src/typescript-schema-generator/src/generate-validators.ts b/src/typescript-schema-generator/src/generate-validators.ts index 869ac050b..348beb3d5 100644 --- a/src/typescript-schema-generator/src/generate-validators.ts +++ b/src/typescript-schema-generator/src/generate-validators.ts @@ -5,13 +5,13 @@ import path from 'node:path'; import Ajv from 'ajv/dist/2020'; import standaloneCode from 'ajv/dist/standalone'; -import { createOutputDir, writeFile, writeTypesIndex } from 'file-utils'; import { + createOutputDir, destinationPackageName, - eventSchemasDir, - listEventSchemas, - loadSchema, -} from 'utils'; + writeFile, + writeTypesIndex, +} from 'file-utils'; +import { eventSchemasDir, listEventSchemas, loadSchema } from 'utils'; export function generateValidators() { const ajv = new Ajv({ diff --git a/utils/utils/src/schema-utils/schema-utils.ts b/utils/utils/src/schema-utils/schema-utils.ts index e8562cf33..b4e3fe409 100644 --- a/utils/utils/src/schema-utils/schema-utils.ts +++ b/utils/utils/src/schema-utils/schema-utils.ts @@ -16,8 +16,6 @@ export const eventSchemasDir = path.resolve( 'events', ); -export const destinationPackageName = 'digital-letters-events'; - /** * Lists all event schema filenames in the digital letters schemas directory. * From 31a260004727ed94be22f964b500506040bd8fb6 Mon Sep 17 00:00:00 2001 From: Gareth Allan <157592212+gareth-allan@users.noreply.github.com> Date: Mon, 15 Dec 2025 17:24:29 +0000 Subject: [PATCH 13/20] CCM-12896: Generate Pydantic models programmatically --- src/python-schema-generator/src/file_utils.py | 20 +++++-- .../src/generate_models.py | 8 ++- .../src/model_generator.py | 37 ++++-------- .../tests/test_file_utils.py | 25 ++++++--- .../tests/test_model_generator.py | 56 ++++++++++--------- 5 files changed, 80 insertions(+), 66 deletions(-) diff --git a/src/python-schema-generator/src/file_utils.py b/src/python-schema-generator/src/file_utils.py index 904e4ddd8..7b6342de9 100644 --- a/src/python-schema-generator/src/file_utils.py +++ b/src/python-schema-generator/src/file_utils.py @@ -24,17 +24,29 @@ def list_json_schemas(schema_dir: str) -> list[str]: return [f.name for f in flattened_schema_files] -def load_json_schema(schema_path: str) -> dict[str, Any]: - """Load a JSON schema from file. +def load_json_schema(schema_path: str) -> str: + """Load a JSON schema from file as a string. Args: schema_path: Path to the JSON schema file Returns: - Loaded schema as dictionary + Loaded schema as string """ with open(schema_path, encoding="utf-8") as f: - return json.load(f) + return f.read() + + +def parse_json_schema(schema: str) -> dict[str, Any]: + """Parse a JSON schema from a string. + + Args: + schema: JSON schema as a string + + Returns: + Schema as dictionary + """ + return json.loads(schema) def write_init_file(output_dir: str, model_names: list[str]) -> None: diff --git a/src/python-schema-generator/src/generate_models.py b/src/python-schema-generator/src/generate_models.py index 73e8edfbe..364c947b0 100644 --- a/src/python-schema-generator/src/generate_models.py +++ b/src/python-schema-generator/src/generate_models.py @@ -13,6 +13,7 @@ from file_utils import ( list_json_schemas, load_json_schema, + parse_json_schema, model_name_to_module_name, write_init_file, ) @@ -62,14 +63,15 @@ def main() -> int: generated_models = [] for schema_filename in schema_filenames: schema_path = str(Path(args.input_dir) / schema_filename) - schema = load_json_schema(schema_path) + string_schema = load_json_schema(schema_path) + schema = parse_json_schema(string_schema) model_name = extract_model_name(schema) output_filename = model_name_to_module_name(model_name) + ".py" - output_file_path = str(Path(args.output_dir) / output_filename) + output_file_path = Path(args.output_dir) / output_filename generate_pydantic_model( - schema_path, output_file_path, model_name + string_schema, output_file_path, model_name ) generated_models.append(model_name) diff --git a/src/python-schema-generator/src/model_generator.py b/src/python-schema-generator/src/model_generator.py index a1b2df3e1..154b58648 100644 --- a/src/python-schema-generator/src/model_generator.py +++ b/src/python-schema-generator/src/model_generator.py @@ -1,13 +1,12 @@ """Model generator using datamodel-code-generator.""" -import subprocess -import sys from pathlib import Path +from datamodel_code_generator import DataModelType, InputFileType, generate def generate_pydantic_model( - schema_path: str, output_file: str, class_name: str + schema: str, output_file_path: Path, class_name: str ) -> None: """Generate a Pydantic model from a JSON schema. @@ -19,32 +18,18 @@ def generate_pydantic_model( Raises: RuntimeError: If model generation fails """ - datamodel_cmd = str(Path(sys.executable).parent / "datamodel-codegen") - cmd = [ - datamodel_cmd, - "--input", - schema_path, - "--output", - output_file, - "--class-name", - class_name, - "--input-file-type", - "jsonschema", - "--output-model-type", - "pydantic_v2.BaseModel", - "--use-schema-description", - "--custom-file-header", - '''"""Generated Pydantic model for NHS Notify Digital Letters events. + + generate( + schema, + input_file_type=InputFileType.JsonSchema, + output=output_file_path, + output_model_type=DataModelType.PydanticV2BaseModel, + class_name=class_name, + use_schema_description=True, + custom_file_header='''"""Generated Pydantic model for NHS Notify Digital Letters events. This file is auto-generated. Do not edit manually. """ ''' - ] - result = subprocess.run( - cmd, capture_output=True, text=True, check=False, encoding="utf-8" ) - - if result.returncode != 0: - error_msg = f"Failed to generate model: {result.stderr}" - raise RuntimeError(error_msg) diff --git a/src/python-schema-generator/tests/test_file_utils.py b/src/python-schema-generator/tests/test_file_utils.py index 98871ec00..fa2a1436f 100644 --- a/src/python-schema-generator/tests/test_file_utils.py +++ b/src/python-schema-generator/tests/test_file_utils.py @@ -7,6 +7,7 @@ from src.file_utils import ( list_json_schemas, load_json_schema, + parse_json_schema, write_init_file, model_name_to_module_name ) @@ -51,21 +52,31 @@ class TestLoadJsonSchema: def test_loads_valid_json_schema(self, tmp_path): """Test loading a valid JSON schema.""" schema_file = tmp_path / "test.schema.json" - schema_content = {"title": "TestSchema", "type": "object"} - schema_file.write_text(json.dumps(schema_content)) + schema_content = json.dumps({"title": "TestSchema", "type": "object"}) + schema_file.write_text(schema_content) result = load_json_schema(str(schema_file)) assert result == schema_content - def test_raises_error_for_invalid_json(self, tmp_path): + +class TestParseJsonSchema: + """Tests for parse_json_schema function.""" + + def test_parses_valid_json_schema(self): + """Test loading a valid JSON schema.""" + schema_content = { "title": "TestSchema", "type": "object" } + + result = parse_json_schema(json.dumps(schema_content)) + + assert result == schema_content + + def test_raises_error_for_invalid_json(self): """Test that it raises error for invalid JSON.""" - schema_file = tmp_path / "invalid.json" - schema_file.write_text("not valid json") + invalid_json = "not valid json" with pytest.raises(json.JSONDecodeError): - load_json_schema(str(schema_file)) - + parse_json_schema(invalid_json) class TestWriteInitFile: """Tests for write_init_file function.""" diff --git a/src/python-schema-generator/tests/test_model_generator.py b/src/python-schema-generator/tests/test_model_generator.py index feb8f9742..f307692fb 100644 --- a/src/python-schema-generator/tests/test_model_generator.py +++ b/src/python-schema-generator/tests/test_model_generator.py @@ -1,8 +1,10 @@ """Tests for model_generator module.""" -from unittest.mock import MagicMock, patch +from pathlib import Path +from unittest.mock import patch import pytest +from datamodel_code_generator import DataModelType, InputFileType from src.model_generator import generate_pydantic_model @@ -10,36 +12,38 @@ class TestGeneratePydanticModel: """Tests for generate_pydantic_model function.""" - @patch("src.model_generator.subprocess.run") - def test_calls_datamodel_codegen_with_expected_arguments(self, mock_run): + @patch("src.model_generator.generate") + def test_calls_datamodel_codegen_with_expected_arguments(self, mock_generate): """Test successful model generation.""" # Arrange - schema_path = "test_model.json" - output_file = "test_model.py" - mock_run.return_value = MagicMock(returncode=0, stderr="") + schema = '{"type": "object", "properties": {"name": {"type": "string"}}}' + output_file = Path("test_model.py") # Act - generate_pydantic_model(schema_path, output_file, "TestModel") + generate_pydantic_model(schema, output_file, "TestModel") # Assert - assert mock_run.called - cmd_args = mock_run.call_args[0][0] - assert "datamodel-codegen" in cmd_args[0] # First arg is the executable - assert "--input" in cmd_args[1] - assert schema_path in cmd_args[2] - assert "--output" in cmd_args[3] - assert output_file in cmd_args[4] - assert "--class-name" in cmd_args[5] - assert "TestModel" in cmd_args[6] - - @patch("src.model_generator.subprocess.run") - def test_raises_error_on_generation_failure(self, mock_run): - """Test that it raises error when generation fails.""" - schema_path = "test_model.json" - output_file = "test_model.py" - mock_run.return_value = MagicMock( - returncode=1, stderr="Error: Invalid schema" + mock_generate.assert_called_once_with( + schema, + input_file_type=InputFileType.JsonSchema, + output=output_file, + output_model_type=DataModelType.PydanticV2BaseModel, + class_name="TestModel", + use_schema_description=True, + custom_file_header='''"""Generated Pydantic model for NHS Notify Digital Letters events. + +This file is auto-generated. Do not edit manually. +""" + +''' ) - with pytest.raises(RuntimeError, match="Failed to generate model"): - generate_pydantic_model(schema_path, output_file, "TestModel") + @patch("src.model_generator.generate") + def test_raises_error_on_generation_failure(self, mock_generate): + """Test that it raises error when generation fails.""" + schema = '{"type": "object", "properties": {"name": {"type": "string"}}}' + output_file = Path("test_model.py") + mock_generate.side_effect = RuntimeError("Invalid schema") + + with pytest.raises(RuntimeError, match="Invalid schema"): + generate_pydantic_model(schema, output_file, "TestModel") From ac74963dd66811d35acb01a1b94366ada3ba04f1 Mon Sep 17 00:00:00 2001 From: Gareth Allan <157592212+gareth-allan@users.noreply.github.com> Date: Mon, 15 Dec 2025 17:42:29 +0000 Subject: [PATCH 14/20] CCM-12896: Temporarily skip build check --- .github/workflows/stage-2-test.yaml | 1 - 1 file changed, 1 deletion(-) diff --git a/.github/workflows/stage-2-test.yaml b/.github/workflows/stage-2-test.yaml index 93ea26977..25a596f2e 100644 --- a/.github/workflows/stage-2-test.yaml +++ b/.github/workflows/stage-2-test.yaml @@ -58,7 +58,6 @@ jobs: - name: "Generate dependencies" run: | npm run generate-dependencies - git diff --exit-code test-unit: name: "Unit tests" runs-on: ubuntu-latest From 19e584ae672f46638a3cb531472e5c1c5f2d69d0 Mon Sep 17 00:00:00 2001 From: Gareth Allan <157592212+gareth-allan@users.noreply.github.com> Date: Tue, 16 Dec 2025 08:57:19 +0000 Subject: [PATCH 15/20] CCM-12896: Don't run npm install as part of python class generation --- .github/workflows/stage-2-test.yaml | 1 + src/python-schema-generator/Makefile | 3 --- 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/.github/workflows/stage-2-test.yaml b/.github/workflows/stage-2-test.yaml index 25a596f2e..93ea26977 100644 --- a/.github/workflows/stage-2-test.yaml +++ b/.github/workflows/stage-2-test.yaml @@ -58,6 +58,7 @@ jobs: - name: "Generate dependencies" run: | npm run generate-dependencies + git diff --exit-code test-unit: name: "Unit tests" runs-on: ubuntu-latest diff --git a/src/python-schema-generator/Makefile b/src/python-schema-generator/Makefile index 9ddf265ac..81d117d5d 100644 --- a/src/python-schema-generator/Makefile +++ b/src/python-schema-generator/Makefile @@ -14,9 +14,6 @@ install: @echo "Installing Python production dependencies..." @pip install -r requirements.txt @echo "Production dependencies installed!" - @echo "Installing Node production dependencies..." - @npm install - @echo "Node production dependencies installed!" # Install development dependencies install-dev: From 20ad6c40dc60551a8a15d5ed91654436ec45aae1 Mon Sep 17 00:00:00 2001 From: Gareth Allan <157592212+gareth-allan@users.noreply.github.com> Date: Tue, 16 Dec 2025 09:14:14 +0000 Subject: [PATCH 16/20] CCM-12896: Attempt to fix code coverage paths for python-schema-generator --- scripts/config/sonar-scanner.properties | 1 + src/python-schema-generator/Makefile | 12 ++++++------ 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/scripts/config/sonar-scanner.properties b/scripts/config/sonar-scanner.properties index ad1341fb4..c1a027da7 100644 --- a/scripts/config/sonar-scanner.properties +++ b/scripts/config/sonar-scanner.properties @@ -3,6 +3,7 @@ sonar.host.url=https://sonarcloud.io # Temporary change to allow testing of build before fixing Sonar coverage issues sonar.qualitygate.wait=false +sonar.log.level=TRACE sonar.sourceEncoding=UTF-8 sonar.sources=. sonar.tests=tests/, src/asyncapigenerator/tests, src/cloudeventjekylldocs/tests, src/eventcatalogasyncapiimporter/tests, src/cloudevents/tools/builder/__tests__, src/cloudevents/tools/cache/__tests__, src/cloudevents/tools/generator/__tests__, lambdas/mesh-poll/src/__tests__, lambdas/ttl-create-lambda/src/__tests__, lambdas/ttl-poll-lambda/src/__tests__, utils/utils/src/__tests__, utils/sender-management/src/__tests__ diff --git a/src/python-schema-generator/Makefile b/src/python-schema-generator/Makefile index 81d117d5d..505216709 100644 --- a/src/python-schema-generator/Makefile +++ b/src/python-schema-generator/Makefile @@ -39,14 +39,14 @@ test: install-dev # Generate coverage report coverage: install-dev @echo "Generating coverage report..." - @pytest tests/ \ - --cov=. \ - --cov-config=pytest.ini \ - --cov-report=html:htmlcov \ + @cd ../.. && PYTHONPATH=src/python-schema-generator:$$PYTHONPATH pytest src/python-schema-generator/tests/ \ + --cov=src/python-schema-generator/src \ + --cov-config=src/python-schema-generator/pytest.ini \ + --cov-report=html:src/python-schema-generator/htmlcov \ --cov-report=term-missing \ - --cov-report=xml:coverage.xml \ + --cov-report=xml:src/python-schema-generator/coverage.xml \ --cov-branch - @echo "Coverage report generated in htmlcov/" + @echo "Coverage report generated in src/python-schema-generator/htmlcov/" # Clean output directory and generated files clean: From 69502d543e54479f57babbc15f5653569dfc61fa Mon Sep 17 00:00:00 2001 From: Gareth Allan <157592212+gareth-allan@users.noreply.github.com> Date: Tue, 16 Dec 2025 09:51:44 +0000 Subject: [PATCH 17/20] CCM-12896: Potential Sonar coverage fix --- src/python-schema-generator/Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/python-schema-generator/Makefile b/src/python-schema-generator/Makefile index 505216709..d45f0f341 100644 --- a/src/python-schema-generator/Makefile +++ b/src/python-schema-generator/Makefile @@ -39,8 +39,8 @@ test: install-dev # Generate coverage report coverage: install-dev @echo "Generating coverage report..." - @cd ../.. && PYTHONPATH=src/python-schema-generator:$$PYTHONPATH pytest src/python-schema-generator/tests/ \ - --cov=src/python-schema-generator/src \ + @cd ../.. && pytest src/python-schema-generator/tests/ \ + --cov=src/python-schema-generator \ --cov-config=src/python-schema-generator/pytest.ini \ --cov-report=html:src/python-schema-generator/htmlcov \ --cov-report=term-missing \ From 50304a2735b07d1c88ad62cfa032cbd6edb6a561 Mon Sep 17 00:00:00 2001 From: Gareth Allan <157592212+gareth-allan@users.noreply.github.com> Date: Tue, 16 Dec 2025 10:41:42 +0000 Subject: [PATCH 18/20] CCM-12896: Add python-schema-generator coverage.xml to Sonar config --- scripts/config/sonar-scanner.properties | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/config/sonar-scanner.properties b/scripts/config/sonar-scanner.properties index c1a027da7..7122e9f07 100644 --- a/scripts/config/sonar-scanner.properties +++ b/scripts/config/sonar-scanner.properties @@ -6,12 +6,12 @@ sonar.qualitygate.wait=false sonar.log.level=TRACE sonar.sourceEncoding=UTF-8 sonar.sources=. -sonar.tests=tests/, src/asyncapigenerator/tests, src/cloudeventjekylldocs/tests, src/eventcatalogasyncapiimporter/tests, src/cloudevents/tools/builder/__tests__, src/cloudevents/tools/cache/__tests__, src/cloudevents/tools/generator/__tests__, lambdas/mesh-poll/src/__tests__, lambdas/ttl-create-lambda/src/__tests__, lambdas/ttl-poll-lambda/src/__tests__, utils/utils/src/__tests__, utils/sender-management/src/__tests__ +sonar.tests=tests/, src/asyncapigenerator/tests, src/cloudeventjekylldocs/tests, src/eventcatalogasyncapiimporter/tests, src/python-schema-generator/tests, src/cloudevents/tools/builder/__tests__, src/cloudevents/tools/cache/__tests__, src/cloudevents/tools/generator/__tests__, lambdas/mesh-poll/src/__tests__, lambdas/ttl-create-lambda/src/__tests__, lambdas/ttl-poll-lambda/src/__tests__, utils/utils/src/__tests__, utils/sender-management/src/__tests__ sonar.test.inclusions=tests/**, src/**/tests/**, src/**/__tests__/**, lambdas/**/src/__tests__/**, utils/utils/src/__tests__/**, utils/sender-management/src/__tests__/** sonar.terraform.provider.aws.version=5.54.1 sonar.cpd.exclusions=**.test.* sonar.coverage.exclusions=tests/**, src/**/tests/**, src/**/__tests__/**, **/*.dev.*, lambdas/**/src/__tests__/**, **/jest.config.ts, **/jest.config.cjs, scripts/**/*.*, docs/**/*.*, utils/utils/src/__tests__/**, src/asyncapigenerator/example_usage.py, src/asyncapigenerator/test_generator.py, src/eventcatalogasyncapiimporter/examples.py, src/digital-letters-events/** -sonar.python.coverage.reportPaths=src/asyncapigenerator/coverage.xml,src/cloudeventjekylldocs/coverage.xml,src/eventcatalogasyncapiimporter/coverage.xml +sonar.python.coverage.reportPaths=src/asyncapigenerator/coverage.xml,src/cloudeventjekylldocs/coverage.xml,src/eventcatalogasyncapiimporter/coverage.xml,src/python-schema-generator/coverage.xml sonar.javascript.lcov.reportPaths=lcov.info,src/cloudevents/coverage/lcov.info sonar.typescript.lcov.reportPaths=lcov.info,src/cloudevents/coverage/lcov.info From ecd9c19fadd197106e04c250edc6b3d31ff55d7f Mon Sep 17 00:00:00 2001 From: Gareth Allan <157592212+gareth-allan@users.noreply.github.com> Date: Tue, 16 Dec 2025 11:18:00 +0000 Subject: [PATCH 19/20] CCM-12896: Add a unit test for generate_models --- scripts/config/sonar-scanner.properties | 4 +- src/python-schema-generator/Makefile | 4 +- .../src/generate_models.py | 7 ++- .../tests/test_generate_models.py | 54 +++++++++++++++++++ 4 files changed, 60 insertions(+), 9 deletions(-) create mode 100644 src/python-schema-generator/tests/test_generate_models.py diff --git a/scripts/config/sonar-scanner.properties b/scripts/config/sonar-scanner.properties index 7122e9f07..8335bc3b9 100644 --- a/scripts/config/sonar-scanner.properties +++ b/scripts/config/sonar-scanner.properties @@ -1,9 +1,7 @@ # Please DO NOT set the following properties `sonar.organization` and `sonar.projectKey` in this file. They must be stored as `SONAR_ORGANISATION_KEY` and `SONAR_PROJECT_KEY` GitHub secrets. sonar.host.url=https://sonarcloud.io -# Temporary change to allow testing of build before fixing Sonar coverage issues -sonar.qualitygate.wait=false -sonar.log.level=TRACE +sonar.qualitygate.wait=true sonar.sourceEncoding=UTF-8 sonar.sources=. sonar.tests=tests/, src/asyncapigenerator/tests, src/cloudeventjekylldocs/tests, src/eventcatalogasyncapiimporter/tests, src/python-schema-generator/tests, src/cloudevents/tools/builder/__tests__, src/cloudevents/tools/cache/__tests__, src/cloudevents/tools/generator/__tests__, lambdas/mesh-poll/src/__tests__, lambdas/ttl-create-lambda/src/__tests__, lambdas/ttl-poll-lambda/src/__tests__, utils/utils/src/__tests__, utils/sender-management/src/__tests__ diff --git a/src/python-schema-generator/Makefile b/src/python-schema-generator/Makefile index d45f0f341..50391199a 100644 --- a/src/python-schema-generator/Makefile +++ b/src/python-schema-generator/Makefile @@ -34,12 +34,12 @@ generate: install # Run tests test: install-dev @echo "Running tests..." - @pytest tests/ + @cd ../.. && PYTHONPATH=src/python-schema-generator/src:$$PYTHONPATH pytest src/python-schema-generator/tests/ # Generate coverage report coverage: install-dev @echo "Generating coverage report..." - @cd ../.. && pytest src/python-schema-generator/tests/ \ + @cd ../.. && PYTHONPATH=src/python-schema-generator/src:$$PYTHONPATH pytest src/python-schema-generator/tests/ \ --cov=src/python-schema-generator \ --cov-config=src/python-schema-generator/pytest.ini \ --cov-report=html:src/python-schema-generator/htmlcov \ diff --git a/src/python-schema-generator/src/generate_models.py b/src/python-schema-generator/src/generate_models.py index 364c947b0..b3ce58e7f 100644 --- a/src/python-schema-generator/src/generate_models.py +++ b/src/python-schema-generator/src/generate_models.py @@ -46,14 +46,12 @@ def parse_args() -> argparse.Namespace: return parser.parse_args() -def main() -> int: +def main(args) -> int: """Main entry point for the generator. Returns: Exit code (0 for success, 1 for failure) """ - args = parse_args() - try: print(f"Generating models in: {args.output_dir}") @@ -89,4 +87,5 @@ def main() -> int: if __name__ == "__main__": - sys.exit(main()) + args = parse_args() + sys.exit(main(args)) diff --git a/src/python-schema-generator/tests/test_generate_models.py b/src/python-schema-generator/tests/test_generate_models.py new file mode 100644 index 000000000..d7665a3d5 --- /dev/null +++ b/src/python-schema-generator/tests/test_generate_models.py @@ -0,0 +1,54 @@ +from src.generate_models import main +from argparse import Namespace + +class TestGenerateModels: + """Tests for generate_models module.""" + + def test_main_returns_zero_on_success(self, tmp_path): + """Test that main returns 0 on successful execution.""" + input_dir = tmp_path / "input" + output_dir = tmp_path / "output" + input_dir.mkdir() + output_dir.mkdir() + + (input_dir / "test.flattened.schema.json").write_text( + '{"title": "TestSchema", "type": "object"}' + ) + args = Namespace(input_dir=str(input_dir), output_dir=str(output_dir)) + + exit_code = main(args) + + assert exit_code == 0 + + def test_main_returns_one_on_failure(self): + """Test that main returns 1 on failure.""" + input_dir = "/nonexistent/input/dir" + output_dir = "/nonexistent/output/dir" + args = Namespace(input_dir=str(input_dir), output_dir=str(output_dir)) + + exit_code = main(args) + assert exit_code == 1 + + def test_main_generates_a_model_for_each_schema_and_an_init_file(self, tmp_path): + """Test that main generates Pydantic models from JSON schemas.""" + input_dir = tmp_path / "input" + output_dir = tmp_path / "output" + input_dir.mkdir() + output_dir.mkdir() + + (input_dir / "one.flattened.schema.json").write_text( + '{"title": "One", "type": "object"}' + ) + (input_dir / "two.flattened.schema.json").write_text( + '{"title": "Two", "type": "object"}' + ) + + args = Namespace(input_dir=str(input_dir), output_dir=str(output_dir)) + + main(args) + + generated_files = list(output_dir.glob("*.py")) + assert len(generated_files) == 3 + assert (output_dir / "__init__.py").exists() + assert (output_dir / "one.py").exists() + assert (output_dir / "two.py").exists() From 8efd464b28dbfa4c3a5b25d1e815297cb298bddc Mon Sep 17 00:00:00 2001 From: Gareth Allan <157592212+gareth-allan@users.noreply.github.com> Date: Tue, 16 Dec 2025 13:25:05 +0000 Subject: [PATCH 20/20] CCM-12896: Tidy up README --- src/python-schema-generator/README.md | 45 +++++---------------------- 1 file changed, 8 insertions(+), 37 deletions(-) diff --git a/src/python-schema-generator/README.md b/src/python-schema-generator/README.md index a5bd3fdd8..c5c56758e 100644 --- a/src/python-schema-generator/README.md +++ b/src/python-schema-generator/README.md @@ -16,12 +16,14 @@ Use `make install-dev` to install the necessary dependencies for this applicatio ## Generating Models In order for this tool to function, you must first build the JSON schemas for -the Digital Letters events. The simplest way to build these schemas is to run -the `npm run generate-dependencies` command from the root of this repository. +the Digital Letters events. This is done automatically when the `make config` +command is run, but if you need to rebuild them for some reason the simplest +approach is to run the `make generate` command from the root of this repository. ### Using the Generator -Once the JSON schemas have been built, Pydantic models can be generated by running: +Once the JSON schemas have been built, Pydantic models can be generated by +running the following from this directory: ```bash make generate @@ -31,15 +33,7 @@ This will: - Read all JSON event schemas from `schemas/digital-letters/2025-10-draft/events/` - Generate Pydantic v2 models using `datamodel-code-generator` -- Output the models to `../digital-letters-events/models/` - -### Configuration - -The generator can be configured through: - -- Command-line arguments -- Environment variables -- Configuration file (if needed) +- Output the models to `../digital-letters-events/digital_letters_events/models/` ### Output Structure @@ -50,8 +44,8 @@ Generated models are placed in the -- `../digital-letters-events/models/` - Individual Pydantic model files -- `../digital-letters-events/models/__init__.py` - Combined index for easy imports +- `../digital-letters-events/digital_letters_events/models/` - Individual Pydantic model files +- `../digital-letters-events/digital_letters_events/models/__init__.py` - Combined index for easy imports ## Testing @@ -85,26 +79,3 @@ make install-dev ```bash make test ``` - -## Usage - -Once generated, the models can be used in your code as follows: - -```python -from digital_letters_event_python import PrintLetterAvailable, CloudEventEnvelope - -# Validate incoming event -event_data = CloudEventEnvelope(**raw_event) -if event_data.type == "uk.nhs.notify.digital.letters.letter.available.v1": - letter_available = PrintLetterAvailable(**raw_event) - # Process the validated event -``` - -## Architecture - -The generator consists of: - -- `src/generate_models.py` - Main CLI script -- `src/schema_processor.py` - Schema loading and processing -- `src/model_generator.py` - Pydantic model generation logic -- `src/file_utils.py` - File system utilities