Commit d9aeb71
authored
fix(release): use subset check for SBOM/lock comparison (#148)
* fix(release): use subset check for SBOM/lock comparison
The evidence script compared the environment lock (pip list --format=freeze,
all installed packages) against the SBOM (cyclonedx-py --pyproject, declared
deps only) with strict equality. These sets can never be equal because the
lock includes build tools (pip, setuptools, wheel, build, twine, etc.).
Changed to SBOM ⊆ lock: every SBOM-declared package must appear in the lock
at the same version, but extra lock entries are expected. Also includes
metadata.component in the SBOM package set so the root package is counted.
Added test_release_evidence_accepts_lock_superset_of_sbom to verify the
new tolerance.
* fix(release): reject empty SBOM package set before subset check
An SBOM with no PyPI components (empty components array and no metadata.component)
would pass the subset check trivially, letting truncated or content-free SBOMs
through release validation. Added a non-empty guard that rejects before the
subset comparison.
Added test_release_evidence_rejects_empty_sbom_package_set.
* fix(release): require engraphis root component in SBOM package set
A truncated SBOM that retains any one PyPI dependency matching the lock
would pass both the non-empty guard and the subset check, even without
the root engraphis component or most declared dependencies.
Added a guard that verifies the PACKAGE name appears in the SBOM package
set before applying the subset comparison.
Added test_release_evidence_rejects_sbom_missing_root_component.
* fix(release): validate SBOM metadata.component name and version directly
The previous guard pooled metadata.component with components[] into a flat
name set, so an SBOM for a different project that happened to list
engraphis as a dependency would pass the name check.
Changed environment_lock_artifact to accept a version parameter and
validate that metadata.component.name == PACKAGE and
metadata.component.version == the release version before applying the
subset comparison. The dependency closure check remains separate.
Updated test fixture to include metadata.component. Added
test_release_evidence_rejects_sbom_with_wrong_version_root_component.
* fix(release): reject SBOMs containing only the root component
An SBOM with valid metadata.component (engraphis at the release version)
but an empty components[] array passed all prior guards: the non-empty
check saw the root tuple, the metadata.component validation passed, and
the subset check succeeded because {root} ⊆ any non-empty lock.
This allowed a truncated SBOM missing every dependency (including the
required NumPy runtime) to be published as valid release evidence.
Added a guard that requires at least one dependency component beyond
the root package before applying the subset comparison.
Added test_release_evidence_rejects_sbom_with_only_root_component.
* fix(release): validate declared dependency closure and restore --pyproject
Two Codex findings addressed:
1. Restore --pyproject pyproject.toml in the release workflow SBOM
generation step. Without it, cyclonedx-py does not set
metadata.component, so the root-component validation rejects every
real build artifact.
2. Replace the weak 'at least one dependency' guard with a declared
dependency closure check. _declared_dependency_names() parses
pyproject.toml [project].dependencies and verifies that every
declared dependency (except the root package itself) appears in the
SBOM components. A truncated SBOM missing numpy or any other declared
dependency now fails.
Also includes graph-engine E2E test improvements for canvas selector
compatibility and robust graph data access.
* fix(release): add declared dependency closure test and fix maxsplit deprecation
Added test_release_evidence_rejects_sbom_missing_declared_dependencies to
verify the closure check catches truncated SBOMs missing declared deps.
Fixed re.split positional maxsplit deprecation warning.
* fix(release): validate PURL name/version and include optional deps in closure
Two Codex findings addressed:
1. PURL validation: _purl_matches() verifies that metadata.component's
PURL names PACKAGE at the release version. A PURL like
pkg:pypi/other-project@1.2.3 with name='engraphis' now fails.
2. Declared dependency closure: _declared_dependency_names() now reads
both [project].dependencies and all [project.optional-dependencies]
groups, matching the .[all,test] install in the release workflow.
A truncated SBOM missing any optional dependency now fails.
Added test_release_evidence_rejects_sbom_with_mismatched_root_purl.
* fix(release): limit declared dependency closure to core deps only
The previous helper pooled every optional-dependencies group, but the
release workflow installs only .[all,test]. Deps from unselected extras
(hosted-eval, encryption) and PEP 508 marker-inapplicable ones
(onnxruntime<1.24 on Python 3.11) cannot appear in the captured SBOM,
so the closure check would reject every real build.
Reverted to core [project].dependencies only. Optional extras are
opt-in by definition; the subset check already validates that any
optional package present in the SBOM is pinned in the environment lock.
* fix(release): validate dependency PURL consistency and filter known CSP console errors
Two Codex findings addressed:
1. _python_sbom_packages() now validates dependency component PURLs via
_purl_matches(), rejecting components where the PURL names a different
package than the name/version fields. This prevents internally
inconsistent package identities from entering the SBOM closure.
2. The opt-in canvas graph engine E2E test now filters known force-graph
CSP console errors (style-src-elem inline style blocks) before
asserting a clean console. The violation-level filter at lines
3688-3690 already permitted these; the console assertion now matches.
* fix(release): reject malformed PURLs, narrow CSP filter, restore galaxy floor
Three Codex findings addressed:
1. _python_sbom_packages() now raises EvidenceError when a component's
PURL does not match its name/version fields, instead of silently
dropping the malformed component.
2. The CSP console-error filter in graph-engine.spec.js now matches
only the specific force-graph inline style signature, not all
style-src blocks.
3. Restored the Complete Galaxy orbital-travel floor from .00005 back
to .001, matching the original v1.7 gate.
Added test_release_evidence_rejects_sbom_with_malformed_dependency_purl.
* fix(e2e): identify force-graph CSP console errors via message.location()
Chromium's CSP console message text reports the blocked directive but
not the originating script URL. The previous regex required 'force-graph'
in the text, which never matched, so expected vendor CSP blocks still
failed the console-errors assertion.
Use message.location().url to identify force-graph-sourced style-src-elem
blocks. Application-level CSP regressions surface through a different
location and still fail the assertion.
* fix(release): reject SBOM components without valid PyPI PURLs
Components with missing or non-PyPI PURLs (e.g. pkg:deb) were silently
dropped, allowing malformed components to coexist with valid ones.
Now raises EvidenceError instead of continuing.
Added test_release_evidence_rejects_sbom_with_non_pypi_component_purl.
* fix(release): reject conflicting package versions in environment lock
A lock containing the same canonical package name at two different
versions (e.g. alpha-package==1.0 and alpha-package==9.9) passed the
tuple-based duplicate check and the subset comparison, allowing
contradictory locks to be published as exact build freezes.
Added a canonical-name tracker that rejects conflicting versions
regardless of the version field.
Added test_release_evidence_rejects_lock_with_conflicting_versions.
* fix(e2e): restore orbital radius bounds and assert envelope system count
Two Codex findings addressed:
1. Restored the orbital radius drift bound from 0.75-1.25 back to
0.82-1.18, matching the established regression gate. The relaxed
bound let visibly unstable carrier lanes pass.
2. Added envelope.systems.length > 0 assertion before the finite and
overlap checks, so the envelope contract observes actual galaxy
systems rather than passing vacuously on an empty systems array.
* fix(e2e): wait for graph count update after filter clear
The graph-count element updates asynchronously after setRepoFilter('')
triggers a render cycle. The immediate assertion raced the DOM update.
Added waitForFunction to poll for the expected count text before
asserting, with a 10-second timeout for CI variability.
* fix(release): enforce declared dependency version constraints
The SBOM validation previously checked only that declared dependency
names were present, ignoring version specifiers. If pyproject.toml
required numpy>=1.24 but the SBOM contained numpy==1.0, the check
would pass.
- Added _parse_requirement() to extract PEP 508 specifiers
- Added _version_satisfies() to validate versions against constraints
(==, !=, >=, <=, >, <, ~=) using stdlib only
- Replaced _declared_dependency_names with _declared_dependencies
returning {name: specifier} instead of set[str]
- environment_lock_artifact now rejects SBOM versions violating
declared constraints
- Restored systemBefore.angularVelocity as the independent oracle
for orbital direction in graph-engine.spec.js
* fix(release): harden evidence and drag regression
* test(graph): tighten envelope visibility contract
* fix(release): require selected extras in SBOM closure; restore galaxy travel floor
Two outstanding Codex review findings on PR 148:
1. _declared_dependencies() now includes every marker-applicable
requirement from the [all] and [test] extras (the groups the release
workflow installs before capturing the SBOM), evaluating PEP 508
environment markers against the capture interpreter via
packaging.markers. A truncated SBOM containing only the core
dependency can no longer pass the closure check.
2. Restored the Complete Galaxy orbital-travel floors from .00005 back
to .001: 96ff9b6 had restored them but the change was lost in the
following commit's rewrite of the file.
Added focused regression tests for both extras-closure behavior and
marker-inapplicable extras being correctly not required.
* test(release): use double-quoted TOML strings in extras fixtures
The Python 3.9 regex fallback in _declared_dependencies() only extracts
double-quoted requirement strings, so single-quoted fixture entries made
it capture the marker version as a package name ('3-9') and miss the
extra dependency, failing the core-floor job.
* docs(release): align lock-artifact docstring with subset semantics
- environment_lock_artifact's docstring still claimed the build freeze must
*equal* the SBOM closure while the shipped check accepts a superset
(lock ⊇ SBOM, extras such as build tooling permitted); state the real
contract per AGENTS.md "code wins, then fix the doc in the same change"
- normalize missing blank line between adjacent release-evidence tests
* fix(release): close the truncated-SBOM hole in the lock comparison
The one-directional subset check (SBOM subset of lock) let a truncated
SBOM that kept the root and every direct requirement but dropped
transitive packages publish incomplete release evidence: absent SBOM
packages imposed no requirement on the lock.
Two-sided hardening, verified against cyclonedx-bom 7.3.0 behavior:
- environment_lock_artifact now also rejects lock packages the SBOM does
not inventory ('truncated closure'). A local probe of the pinned
generator shows cyclonedx-py environment captures the whole build
environment including workflow-installed tooling (pip, build,
cyclonedx-bom), so captured lock and SBOM name-sets match exactly
after canonicalization; a lock-only entry means truncation, not
expected tooling overhead.
- new _validate_python_sbom_dependency_closure validates the CycloneDX
dependency graph when present (pinned generator always emits it):
dependsOn refs must resolve to the root or a listed component, and
every declared requirement must be transitively reachable from the
project root. Build-tool components are legitimately unreachable from
the root, so full-graph reachability is intentionally not required.
- tests: replace accepts_lock_superset_of_sbom with rejection tests for
truncated closures (with and without a dependency graph), dangling
graph refs, and unreachable declared dependencies.
* fix(release): accept CycloneDX root without PURL
* fix(release): reject graph refs without components
* fix(release): harden graph and stellar snapshot evidence
* fix(release): reject colliding SBOM component refs
* fix(release): reject SBOM components without identity
* fix(release): exempt selected extras from root reachability
* fix(release): combine repeated dependency constraints
* fix(release): prefer CycloneDX bom refs1 parent feb83ad commit d9aeb71
4 files changed
Lines changed: 1313 additions & 91 deletions
File tree
- .github/workflows
- scripts
- tests
- e2e
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
84 | 84 | | |
85 | 85 | | |
86 | 86 | | |
| 87 | + | |
87 | 88 | | |
88 | 89 | | |
89 | 90 | | |
| |||
0 commit comments