Skip to content

Commit b0d0791

Browse files
authored
refactor: (class-generator) Migrate schema extraction from OpenAPI v2 to v3 (#2473)
* fix: Extract schemas from CRDs with OpenAPIV3Schema for class generation * fix: Handle Python reserved keywords in generated resource classes - Add keyword module import and sanitization mapping - Create sanitize_python_name() to append underscore to Python keywords - Update prepare_property_dict() to apply keyword sanitization - Modify Jinja template to document renamed parameters - Ensure to_dict() maps sanitized names back to original field names This fixes syntax errors when CRD fields use Python reserved keywords like 'finally' in Tekton Pipeline resources. The solution automatically renames conflicting parameters (e.g., 'finally' → 'finally_') while preserving the original field names in the API representation. Example: Pipeline.finally becomes finally_ parameter but still maps to spec["finally"] in to_dict() method. * fix: Replace mapping.clear() with atomic update in CRD schema extraction - Use enumerate to track mapping index for in-place replacement - Create new mapping dictionary from full_schema.copy() - Add namespaced field to new mapping - Replace old mapping atomically without intermediate empty state This prevents potential data loss if an error occurs between clearing and updating the mapping. The new approach ensures the update is atomic and the original data remains intact if any step fails. * refactor: Extract helper functions from extract_crd_schemas for better maintainability - Extract _should_update_schema() to determine if schema needs updating - Extract _format_crd_schema() to handle schema formatting logic - Extract _update_resource_mapping() to manage resource mapping updates - Reduce main function from 124 to ~50 lines This refactoring improves code maintainability by following the Single Responsibility Principle. Each helper function now has a focused purpose, making the code easier to test, understand, and modify. The main function now clearly shows the high-level flow of CRD schema extraction. * update schema * refactor: Remove individual schema files from git - Keep only __resources-mappings.json and _definitions.json in git - Add other schema files to .gitignore - Schema files remain locally but are removed from repository - This reduces repo size by ~2000 files that are generated during build * refactor: (class-generator) Migrate schema extraction from OpenAPI v2 to v3 * fix(class-generator): migrate to OpenAPI v3, fix duplicate imports and allOf spec handling, improve namespacing detection
1 parent e88d456 commit b0d0791

2,093 files changed

Lines changed: 16791 additions & 855164 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitattributes

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
class_generator/schema/*.json binary
2+
class_generator/schema/*.json -diff

.gitignore

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,15 @@
11
# Temporary files
22
_out
33

4+
# Generated OpenAPI files
5+
class_generator/__k8s-openapi-*.json
6+
7+
# Generated schema files (except the two needed for distribution)
8+
class_generator/schema/*.json
9+
!class_generator/schema/__resources-mappings.json
10+
!class_generator/schema/_definitions.json
11+
class_generator/schema/__not-kind.txt
12+
413
# Created by .ignore support plugin (hsz.mobi)
514
### Python template
615
# Byte-compiled / optimized / DLL files

.pre-commit-config.yaml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
1-
exclude: "class_generator/schema/"
21
minimum_pre_commit_version: 3.3.0
32
default_install_hook_types: [pre-commit, commit-msg]
43

class_generator/README.md

Lines changed: 23 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -138,22 +138,8 @@ class-generator --kind Pod --add-tests
138138
- Dependencies
139139
- Kubernetes/Openshift cluster
140140
- [oc](https://mirror.openshift.com/pub/openshift-v4/x86_64/clients/ocp/stable/) or [kubectl](https://kubernetes.io/docs/tasks/tools/) (latest version)
141-
- [openapi2jsonschema](https://github.com/instrumenta/openapi2jsonschema)
142141
- [uv](https://github.com/astral-sh/uv)
143142

144-
```bash
145-
uv tool install --python python3.9 openapi2jsonschema
146-
```
147-
148-
If install fail ([Issue 1455](https://github.com/astral-sh/uv/issues/1455)) try to install manually:
149-
150-
```bash
151-
git clone https://github.com/instrumenta/openapi2jsonschema
152-
cd openapi2jsonschema
153-
sed -i .bk s/'pyyaml = "^5.1"'/'pyyaml = ">=6.0"'/g pyproject.toml
154-
uv tool install --python python3.9 .
155-
```
156-
157143
- Clone this repository
158144

159145
```bash
@@ -172,3 +158,26 @@ oc login <clster api URL> -u <username> -p <password>
172158
```bash
173159
class-generator --update-schema
174160
```
161+
162+
The schema update process:
163+
- Fetches schemas directly from the OpenAPI v3 endpoint
164+
- Runs in parallel for improved performance
165+
- Stores schemas in `class_generator/schema/__resources-mappings.json` and `class_generator/schema/_definitions.json`
166+
167+
## Version Selection
168+
169+
When multiple API versions exist for the same resource within an API group, the class generator automatically selects the latest stable version according to this precedence:
170+
171+
`v2` > `v1` > `v1beta2` > `v1beta1` > `v1alpha2` > `v1alpha1`
172+
173+
For example:
174+
- If both `v1` and `v1beta1` exist, `v1` will be used
175+
- Resources from different API groups are treated as separate resources
176+
177+
## Python Keyword Handling
178+
179+
Fields that conflict with Python reserved keywords are automatically renamed by appending an underscore. The original field name is preserved in the API calls.
180+
181+
Example:
182+
- CRD field `finally` → Python parameter `finally_`
183+
- The generated `to_dict()` method correctly maps back to `finally` for API operations

0 commit comments

Comments
 (0)