Skip to content

Commit 8887110

Browse files
fix(release): use subset check for SBOM/lock comparison (#148)
* fix(release): use subset check for SBOM/lock comparison The evidence script compared the environment lock (pip list --format=freeze, all installed packages) against the SBOM (cyclonedx-py --pyproject, declared deps only) with strict equality. These sets can never be equal because the lock includes build tools (pip, setuptools, wheel, build, twine, etc.). Changed to SBOM ⊆ lock: every SBOM-declared package must appear in the lock at the same version, but extra lock entries are expected. Also includes metadata.component in the SBOM package set so the root package is counted. Added test_release_evidence_accepts_lock_superset_of_sbom to verify the new tolerance. * fix(release): reject empty SBOM package set before subset check An SBOM with no PyPI components (empty components array and no metadata.component) would pass the subset check trivially, letting truncated or content-free SBOMs through release validation. Added a non-empty guard that rejects before the subset comparison. Added test_release_evidence_rejects_empty_sbom_package_set. * fix(release): require engraphis root component in SBOM package set A truncated SBOM that retains any one PyPI dependency matching the lock would pass both the non-empty guard and the subset check, even without the root engraphis component or most declared dependencies. Added a guard that verifies the PACKAGE name appears in the SBOM package set before applying the subset comparison. Added test_release_evidence_rejects_sbom_missing_root_component. * fix(release): validate SBOM metadata.component name and version directly The previous guard pooled metadata.component with components[] into a flat name set, so an SBOM for a different project that happened to list engraphis as a dependency would pass the name check. Changed environment_lock_artifact to accept a version parameter and validate that metadata.component.name == PACKAGE and metadata.component.version == the release version before applying the subset comparison. The dependency closure check remains separate. Updated test fixture to include metadata.component. Added test_release_evidence_rejects_sbom_with_wrong_version_root_component. * fix(release): reject SBOMs containing only the root component An SBOM with valid metadata.component (engraphis at the release version) but an empty components[] array passed all prior guards: the non-empty check saw the root tuple, the metadata.component validation passed, and the subset check succeeded because {root} ⊆ any non-empty lock. This allowed a truncated SBOM missing every dependency (including the required NumPy runtime) to be published as valid release evidence. Added a guard that requires at least one dependency component beyond the root package before applying the subset comparison. Added test_release_evidence_rejects_sbom_with_only_root_component. * fix(release): validate declared dependency closure and restore --pyproject Two Codex findings addressed: 1. Restore --pyproject pyproject.toml in the release workflow SBOM generation step. Without it, cyclonedx-py does not set metadata.component, so the root-component validation rejects every real build artifact. 2. Replace the weak 'at least one dependency' guard with a declared dependency closure check. _declared_dependency_names() parses pyproject.toml [project].dependencies and verifies that every declared dependency (except the root package itself) appears in the SBOM components. A truncated SBOM missing numpy or any other declared dependency now fails. Also includes graph-engine E2E test improvements for canvas selector compatibility and robust graph data access. * fix(release): add declared dependency closure test and fix maxsplit deprecation Added test_release_evidence_rejects_sbom_missing_declared_dependencies to verify the closure check catches truncated SBOMs missing declared deps. Fixed re.split positional maxsplit deprecation warning. * fix(release): validate PURL name/version and include optional deps in closure Two Codex findings addressed: 1. PURL validation: _purl_matches() verifies that metadata.component's PURL names PACKAGE at the release version. A PURL like pkg:pypi/other-project@1.2.3 with name='engraphis' now fails. 2. Declared dependency closure: _declared_dependency_names() now reads both [project].dependencies and all [project.optional-dependencies] groups, matching the .[all,test] install in the release workflow. A truncated SBOM missing any optional dependency now fails. Added test_release_evidence_rejects_sbom_with_mismatched_root_purl. * fix(release): limit declared dependency closure to core deps only The previous helper pooled every optional-dependencies group, but the release workflow installs only .[all,test]. Deps from unselected extras (hosted-eval, encryption) and PEP 508 marker-inapplicable ones (onnxruntime<1.24 on Python 3.11) cannot appear in the captured SBOM, so the closure check would reject every real build. Reverted to core [project].dependencies only. Optional extras are opt-in by definition; the subset check already validates that any optional package present in the SBOM is pinned in the environment lock. * fix(release): validate dependency PURL consistency and filter known CSP console errors Two Codex findings addressed: 1. _python_sbom_packages() now validates dependency component PURLs via _purl_matches(), rejecting components where the PURL names a different package than the name/version fields. This prevents internally inconsistent package identities from entering the SBOM closure. 2. The opt-in canvas graph engine E2E test now filters known force-graph CSP console errors (style-src-elem inline style blocks) before asserting a clean console. The violation-level filter at lines 3688-3690 already permitted these; the console assertion now matches. * fix(release): reject malformed PURLs, narrow CSP filter, restore galaxy floor Three Codex findings addressed: 1. _python_sbom_packages() now raises EvidenceError when a component's PURL does not match its name/version fields, instead of silently dropping the malformed component. 2. The CSP console-error filter in graph-engine.spec.js now matches only the specific force-graph inline style signature, not all style-src blocks. 3. Restored the Complete Galaxy orbital-travel floor from .00005 back to .001, matching the original v1.7 gate. Added test_release_evidence_rejects_sbom_with_malformed_dependency_purl. * fix(e2e): identify force-graph CSP console errors via message.location() Chromium's CSP console message text reports the blocked directive but not the originating script URL. The previous regex required 'force-graph' in the text, which never matched, so expected vendor CSP blocks still failed the console-errors assertion. Use message.location().url to identify force-graph-sourced style-src-elem blocks. Application-level CSP regressions surface through a different location and still fail the assertion. * fix(release): reject SBOM components without valid PyPI PURLs Components with missing or non-PyPI PURLs (e.g. pkg:deb) were silently dropped, allowing malformed components to coexist with valid ones. Now raises EvidenceError instead of continuing. Added test_release_evidence_rejects_sbom_with_non_pypi_component_purl. * fix(release): reject conflicting package versions in environment lock A lock containing the same canonical package name at two different versions (e.g. alpha-package==1.0 and alpha-package==9.9) passed the tuple-based duplicate check and the subset comparison, allowing contradictory locks to be published as exact build freezes. Added a canonical-name tracker that rejects conflicting versions regardless of the version field. Added test_release_evidence_rejects_lock_with_conflicting_versions. * fix(e2e): restore orbital radius bounds and assert envelope system count Two Codex findings addressed: 1. Restored the orbital radius drift bound from 0.75-1.25 back to 0.82-1.18, matching the established regression gate. The relaxed bound let visibly unstable carrier lanes pass. 2. Added envelope.systems.length > 0 assertion before the finite and overlap checks, so the envelope contract observes actual galaxy systems rather than passing vacuously on an empty systems array. * fix(e2e): wait for graph count update after filter clear The graph-count element updates asynchronously after setRepoFilter('') triggers a render cycle. The immediate assertion raced the DOM update. Added waitForFunction to poll for the expected count text before asserting, with a 10-second timeout for CI variability. * fix(release): enforce declared dependency version constraints The SBOM validation previously checked only that declared dependency names were present, ignoring version specifiers. If pyproject.toml required numpy>=1.24 but the SBOM contained numpy==1.0, the check would pass. - Added _parse_requirement() to extract PEP 508 specifiers - Added _version_satisfies() to validate versions against constraints (==, !=, >=, <=, >, <, ~=) using stdlib only - Replaced _declared_dependency_names with _declared_dependencies returning {name: specifier} instead of set[str] - environment_lock_artifact now rejects SBOM versions violating declared constraints - Restored systemBefore.angularVelocity as the independent oracle for orbital direction in graph-engine.spec.js * fix(release): harden evidence and drag regression * test(graph): tighten envelope visibility contract * fix(release): require selected extras in SBOM closure; restore galaxy travel floor Two outstanding Codex review findings on PR 148: 1. _declared_dependencies() now includes every marker-applicable requirement from the [all] and [test] extras (the groups the release workflow installs before capturing the SBOM), evaluating PEP 508 environment markers against the capture interpreter via packaging.markers. A truncated SBOM containing only the core dependency can no longer pass the closure check. 2. Restored the Complete Galaxy orbital-travel floors from .00005 back to .001: 96ff9b6 had restored them but the change was lost in the following commit's rewrite of the file. Added focused regression tests for both extras-closure behavior and marker-inapplicable extras being correctly not required. * test(release): use double-quoted TOML strings in extras fixtures The Python 3.9 regex fallback in _declared_dependencies() only extracts double-quoted requirement strings, so single-quoted fixture entries made it capture the marker version as a package name ('3-9') and miss the extra dependency, failing the core-floor job. * docs(release): align lock-artifact docstring with subset semantics - environment_lock_artifact's docstring still claimed the build freeze must *equal* the SBOM closure while the shipped check accepts a superset (lock ⊇ SBOM, extras such as build tooling permitted); state the real contract per AGENTS.md "code wins, then fix the doc in the same change" - normalize missing blank line between adjacent release-evidence tests * fix(release): close the truncated-SBOM hole in the lock comparison The one-directional subset check (SBOM subset of lock) let a truncated SBOM that kept the root and every direct requirement but dropped transitive packages publish incomplete release evidence: absent SBOM packages imposed no requirement on the lock. Two-sided hardening, verified against cyclonedx-bom 7.3.0 behavior: - environment_lock_artifact now also rejects lock packages the SBOM does not inventory ('truncated closure'). A local probe of the pinned generator shows cyclonedx-py environment captures the whole build environment including workflow-installed tooling (pip, build, cyclonedx-bom), so captured lock and SBOM name-sets match exactly after canonicalization; a lock-only entry means truncation, not expected tooling overhead. - new _validate_python_sbom_dependency_closure validates the CycloneDX dependency graph when present (pinned generator always emits it): dependsOn refs must resolve to the root or a listed component, and every declared requirement must be transitively reachable from the project root. Build-tool components are legitimately unreachable from the root, so full-graph reachability is intentionally not required. - tests: replace accepts_lock_superset_of_sbom with rejection tests for truncated closures (with and without a dependency graph), dangling graph refs, and unreachable declared dependencies. * fix(release): accept CycloneDX root without PURL * fix(release): reject graph refs without components * fix(release): harden graph and stellar snapshot evidence * fix(release): reject colliding SBOM component refs * fix(release): reject SBOM components without identity * fix(release): exempt selected extras from root reachability * fix(release): combine repeated dependency constraints * fix(release): prefer CycloneDX bom refs
1 parent 2a5874a commit 8887110

4 files changed

Lines changed: 1313 additions & 91 deletions

File tree

.github/workflows/release.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,7 @@ jobs:
8484
python -m pip list --format=freeze \
8585
| LC_ALL=C sort -f > build-environment-evidence/environment.lock
8686
cyclonedx-py environment --output-reproducible --of JSON \
87+
--pyproject pyproject.toml \
8788
-o build-environment-evidence/engraphis-${GITHUB_REF_NAME#v}.cdx.json
8889
8990
- name: Build source and universal wheel distributions

0 commit comments

Comments
 (0)