feat: implement HIP-0025 resource creation sequencing

[caretak3r]

What this PR does / why we need it:

Implements HIP-0025: Better Support for Resource Creation Sequencing, adding native DAG-based ordering for resource groups and subcharts to Helm v4.

Today Helm applies all rendered manifests at once. Chart authors who need ordered creation must lean on hooks (tedious to maintain) or bake ordering into the application itself. HIP-0025 gives authors a first-class way to declare deployment order via annotations and a Chart.yaml field, and gives operators a --wait=ordered flag to opt in. Charts and releases that don't use it are unaffected.

How it works

When --wait=ordered is set, Helm builds two levels of dependency graph and applies resources in topological order (Kahn's algorithm), waiting for each batch to become ready before starting the next:

flowchart TD
    subgraph L1["Level 1 — subchart ordering"]
        A["Chart.yaml <code>depends-on</code><br/>+ helm.sh/depends-on/subcharts"] --> B["Subchart DAG"]
        B --> C["Topological batches"]
    end
    C -->|"for each batch, recursing<br/>into nested subcharts"| D
    subgraph L2["Level 2 — resource-group ordering"]
        D["helm.sh/resource-group<br/>+ helm.sh/depends-on/resource-groups"] --> E["Resource-group DAG"]
        E --> F["Topological batches"]
    end
    F --> G["Apply batch"]
    G --> H["Wait for readiness<br/>kstatus or custom JSONPath"]
    H -->|"next batch"| G

Loading

Readiness defaults to kstatus; authors can override it per-resource with JSONPath expressions via helm.sh/readiness-success / helm.sh/readiness-failure (operators ==, !=, <, <=, >, >=). uninstall and rollback of a sequenced release walk the DAG in reverse.

What's added

Annotations

Annotation	Scope	Purpose
helm.sh/resource-group	Resource	Assigns the resource to a named group
helm.sh/depends-on/resource-groups	Resource	Resource-group ordering dependencies (JSON array)
helm.sh/depends-on/subcharts	Chart.yaml	Parent → subchart ordering dependencies (JSON array)
helm.sh/readiness-success	Resource	Custom JSONPath success conditions (OR semantics)
helm.sh/readiness-failure	Resource	Custom JSONPath failure conditions (OR; takes precedence)

Chart.yaml

• depends-on on dependencies[] entries — subchart ordering by name or alias.

CLI flags

Flag	Commands	Description
--wait=ordered	install, upgrade, rollback, uninstall	Enables sequenced deployment / reverse-sequenced teardown
--readiness-timeout	install, upgrade, rollback	Per-batch readiness timeout (default 1m; must not exceed --timeout)

Also adds a helm dag diagnostic command that prints the computed batches for a chart without touching a cluster.

Schema (all backward-compatible)

• DependsOn []string on chart.Dependency — optional field.
• SequencingInfo{Enabled, Strategy} persisted on release.Release so rollback/uninstall know to use the sequenced path.
• OrderedWaitStrategy constant in pkg/kube.

Design notes

• Generic DAG engine (pkg/chart/v2/util/dag.go) — topological sort decoupled from Helm types and independently tested; cycle detection surfaces at plan-build time with a chart-path-qualified error before anything is applied.
• Two-level execution — subchart ordering decides when a chart is processed; resource-group ordering decides how its resources are batched. chartPath is threaded through recursion so nested subchart manifests are routed correctly at arbitrary depth.
• Backward compatibility — charts without sequencing annotations, and releases created without --wait=ordered, behave exactly as before; the sequenced path is gated on SequencingInfo.Enabled.
• Hook exclusion — hook resources are excluded from sequencing DAGs and continue to use helm.sh/hook-weight.

Limitations / open trade-offs

• Sequential within a batch. Resources in the same batch are applied sequentially today; intra-batch parallelism is deferred (needs error aggregation and log-interleaving handling) and can be added later without an interface change.

• Multi-slash annotation keys. helm.sh/depends-on/resource-groups and helm.sh/depends-on/subcharts are not valid Kubernetes qualified names (multiple /). Helm consumes them at render/sequencing time and strips the rendered-manifest one before server-side apply, so the API server never sees it; helm template --wait=ordered emits ## START/END batch delimiters and is not intended to be piped straight to kubectl apply. Whether these keys should instead be single-slash forms — which would remove the stripping machinery entirely — is an open question I'd welcome maintainer guidance on.

Commits

Staged into 7 reviewable commits:

#	Commit	Scope
1	cf0b6995b	Generic DAG engine + Chart.yaml depends-on schema
2	eabd8147b	Resource-group parsing + release sequencing metadata
3	371ca4b66	kstatus + custom JSONPath readiness engine
4	ba2dd29ad	Sequencing lint rules
5	555553ecd	Sequence install, upgrade, rollback, uninstall
6	50d3fd188	Ordered-wait CLI flags, template output, helm dag command
7	9470c5a6f	Show unresolved subchart batches in helm dag; hoist annotation keys

How to test

# Unit tests
go test ./pkg/chart/v2/util/... ./pkg/release/v1/... ./pkg/kube/... \
        ./pkg/chart/v2/lint/... ./pkg/action/... ./pkg/cmd/... 

# Inspect computed batches without a cluster
helm dag ./pkg/cmd/testdata/testcharts/sequenced-chart

# Against a kind cluster
helm install demo ./pkg/cmd/testdata/testcharts/sequenced-chart --wait=ordered
helm uninstall demo --wait=ordered

make test-unit, make lint, and make vet are clean; DCO signed off on all commits.

---

Special notes for your reviewer:

• 77 files changed, +12,604/−219. The bulk is new packages under pkg/chart/v2/util, pkg/release/v1/sequence, pkg/kube, plus tests and golden fixtures under pkg/cmd/testdata.
• Companion docs PR: helm/helm-www#2068.
• Happy to split this into separate subchart-sequencing and resource-group PRs if that eases review — noted in the thread.

If applicable:

• this PR contains user facing changes (the docs needed label should be applied if so)
• this PR contains unit tests
• this PR has been tested for backwards compatibility

refs https://github.com/helm/community/blob/main/hips/hip-0025.md

Signed-off-by: Rohit Gudi 50377477+caretak3r@users.noreply.github.com

[Copilot]

Pull request overview

Implements HIP-0025 resource creation sequencing for Helm v4 by introducing a pure, DAG-driven sequencing planner (subchart + resource-group DAGs) and wiring it into CLI/template output, lint, and action flows (install/upgrade/rollback/uninstall) with optional per-resource custom readiness.

Changes:

• Added generic DAG utilities plus sequencing plan builder to compute ordered apply/delete batches across subcharts and resource groups.
• Integrated --wait=ordered and --readiness-timeout into commands/actions, including reverse-order rollback/uninstall behavior and release schema gating.
• Added custom JSONPath-based readiness evaluation and comprehensive unit/golden coverage, plus a helm dag diagnostics command and sequencing lint rules.

Reviewed changes

Copilot reviewed 77 out of 77 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
pkg/release/v1/util/resource_group.go	Resource-group annotations parsing + DAG builder
pkg/release/v1/util/resource_group_test.go	Resource-group parsing/DAG tests
pkg/release/v1/util/manifest.go	Strip invalid sequencing annotation keys from template/apply output
pkg/release/v1/util/manifest_test.go	Stripping behavior tests
pkg/release/v1/sequence/plan.go	Sequencing plan types + reverse/display helpers
pkg/release/v1/sequence/plan_test.go	Plan helper tests + import-purity check
pkg/release/v1/sequence/manifest_parse.go	Parse stored/template manifests into typed manifests
pkg/release/v1/sequence/builder.go	Plan builder: subchart batching + resource-group batching
pkg/release/v1/release.go	Release schema: sequenced marker + legacy decoding
pkg/release/v1/release_test.go	Release schema compatibility tests
pkg/kube/statuswait.go	Custom readiness integration into status waiter
pkg/kube/readiness.go	JSONPath readiness expression parsing/validation/eval
pkg/kube/options.go	Wait options: enable custom readiness
pkg/kube/custom_readiness_status_reader.go	StatusReader wrapper implementing custom readiness
pkg/kube/client.go	Ordered wait strategy + status watcher wiring
pkg/kube/client_wait_strategy_test.go	Ordered wait/custom readiness option tests
pkg/cmd/upgrade.go	Propagate readiness-timeout; enable ordered wait flag
pkg/cmd/uninstall.go	Enable ordered wait flag
pkg/cmd/uninstall_test.go	CLI test: uninstall --wait=ordered
pkg/cmd/testdata/testcharts/sequenced-isolated-chart/templates/dd-plain.yaml	Test chart manifest (unsequenced resource)
pkg/cmd/testdata/testcharts/sequenced-isolated-chart/templates/cc-gamma.yaml	Test chart manifest (isolated group)
pkg/cmd/testdata/testcharts/sequenced-isolated-chart/templates/bb-beta.yaml	Test chart manifest (depends-on group)
pkg/cmd/testdata/testcharts/sequenced-isolated-chart/templates/aa-alpha.yaml	Test chart manifest (root group)
pkg/cmd/testdata/testcharts/sequenced-isolated-chart/Chart.yaml	Test chart metadata
pkg/cmd/testdata/testcharts/sequenced-chart/templates/cc-unsequenced-configmap.yaml	Test chart manifest (unsequenced)
pkg/cmd/testdata/testcharts/sequenced-chart/templates/bb-app-configmap.yaml	Test chart manifest (group + depends-on)
pkg/cmd/testdata/testcharts/sequenced-chart/templates/aa-databases-configmap.yaml	Test chart manifest (group)
pkg/cmd/testdata/testcharts/sequenced-chart/charts/worker/templates/aa-worker-configmap.yaml	Subchart test manifest (group)
pkg/cmd/testdata/testcharts/sequenced-chart/charts/worker/Chart.yaml	Subchart metadata
pkg/cmd/testdata/testcharts/sequenced-chart/Chart.yaml	Test chart metadata + dependency
pkg/cmd/testdata/output/template-ordered-isolated.txt	Golden: ordered template output (isolated demotion)
pkg/cmd/testdata/output/template-ordered-delimiters.txt	Golden: ordered template output (delimiters)
pkg/cmd/testdata/output/dag-sequenced-isolated.txt	Golden: helm dag output (isolated)
pkg/cmd/testdata/output/dag-sequenced-chart.txt	Golden: helm dag output (sequenced chart)
pkg/cmd/template.go	Ordered template rendering + stripping + warnings
pkg/cmd/template_test.go	Ordered template tests + compat/strip invariants
pkg/cmd/root.go	Register dag command
pkg/cmd/rollback.go	Enable ordered wait + readiness-timeout
pkg/cmd/install.go	Enable ordered wait + readiness-timeout
pkg/cmd/get_manifest.go	Clarify verbatim stored manifest output behavior
pkg/cmd/get_manifest_test.go	Test: get manifest prints stored annotations verbatim
pkg/cmd/flags.go	--wait=ordered parsing + readiness-timeout flag
pkg/cmd/flags_test.go	Flag parsing tests (wait/readiness-timeout)
pkg/cmd/dag.go	New helm dag command (diagnostics)
pkg/cmd/dag_test.go	helm dag golden + error-path tests
pkg/chart/v2/util/subchart_dag.go	Build subchart DAG from depends-on + annotation
pkg/chart/v2/util/dependencies.go	Rewrite depends-on refs before alias rewrite
pkg/chart/v2/util/dag.go	Generic DAG implementation
pkg/chart/v2/util/dag_test.go	Generic DAG tests
pkg/chart/v2/lint/rules/sequencing.go	Sequencing lint rule: plan build + readiness validation
pkg/chart/v2/lint/lint.go	Register sequencing lint rules
pkg/chart/v2/lint/lint_test.go	Lint integration test for sequencing rules
pkg/chart/v2/dependency.go	Add depends-on to chart dependencies schema
pkg/chart/v2/dependency_test.go	DependsOn sanitization test
pkg/chart/v2/dependency_json_test.go	JSON/YAML schema compatibility tests
pkg/action/warning_system_test.go	Action-layer warning surface tests
pkg/action/validate.go	Fail-fast on nil REST client in conflict checks
pkg/action/validate_test.go	Nil-client regression test
pkg/action/sequencing.go	Sequenced apply/delete engine + stripping + batch waits
pkg/action/rollback.go	Sequenced rollback path + sequencing-gated behavior
pkg/action/rollback_test.go	Rollback sequencing-annotation strip regression test
pkg/action/install.go	Sequenced install path + plan build preflight
pkg/action/hooks.go	Strip sequencing annotations from hook resources
pkg/action/hooks_test.go	Hook strip regression test
pkg/action/backward_compat_test.go	Backward-compat scenarios for ordered wait + rollback
pkg/action/action.go	Render resources returns sorted manifests for sequencing

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[promptless-for-oss]

Promptless prepared a documentation update related to this change.

Triggered by helm/helm#32314 (HIP-0025 resource creation sequencing).

The docs for HIP-0025 have been updated to match this implementation: documenting --wait=ordered and --readiness-timeout across install/upgrade/rollback/uninstall, the helm.sh/resource-group, helm.sh/depends-on/resource-groups, helm.sh/depends-on/subcharts, and helm.sh/readiness-success/helm.sh/readiness-failure annotations, the depends-on Chart.yaml field, custom JSONPath readiness semantics, helm template --wait=ordered marker behavior, and the new lint rules. Example annotation values were also corrected to valid single-quoted JSON strings.

Review: Document HIP-0025 resource sequencing (docs PR #2068)

[kunalworldwide]

This is a large but well-structured implementation of HIP-0025. The two-level DAG approach — subchart dependencies plus resource-group annotations — gives chart authors flexibility without forcing everyone to adopt sequenced installs.

A few things that stand out:

• The backward-compat tests are important: --wait=ordered is new, but existing --wait behavior must remain unchanged. The test file covers that.
• The lint rule for sequencing is a good safety net; invalid dependsOn references will be caught at helm lint time rather than at install time.
• The DAG utility in pkg/chart/v2/util/dag.go is reusable and well-scoped.

One question: the dependsOn field in Chart.yaml uses chart/subchart names. Is there validation to prevent circular subchart dependencies, and if so, where does the cycle error surface? I see DAG construction in subchart_dag.go but didn't spot an explicit cycle check in the diff.

LGTM otherwise.

[caretak3r]

@kunalworldwide

A couple things to note:

1. subchart DAGs and resource group DAGs share one implementation, and BuildSubchartDAG only constructs the graph

2. the check is in DAG.GetBatches()

   helm/pkg/chart/v2/util/dag.go

   Lines 83 to 138 in 9470c5a

   	// GetBatches performs a topological sort using Kahn's algorithm and returns
   	// the nodes grouped into deployment batches. Each batch contains nodes that
   	// can be deployed in parallel. Batches are ordered: batch 0 has no prerequisites,
   	// batch 1 depends only on batch 0, etc.
   	//
   	// Returns an error if a cycle is detected, including the names of the nodes
   	// involved in the cycle.
   	func (d *DAG) GetBatches() ([][]string, error) {
   	if len(d.nodes) == 0 {
   	return nil, nil
   	}
   	inDegree := make(map[string]int, len(d.inDegree))
   	maps.Copy(inDegree, d.inDegree)
   	var batches [][]string
   	processed := 0
   	for {
   	var batch []string
   	for node := range d.nodes {
   	if inDegree[node] == 0 {
   	batch = append(batch, node)
   	inDegree[node] = -1
   	}
   	}
   	if len(batch) == 0 {
   	break
   	}
   	sort.Strings(batch)
   	batches = append(batches, batch)
   	processed += len(batch)
   	for _, node := range batch {
   	for _, dependent := range d.edges[node] {
   	inDegree[dependent]--
   	}
   	}
   	}
   	if processed == len(d.nodes) {
   	return batches, nil
   	}
   	cycleNodes := make([]string, 0, len(d.nodes)-processed)
   	for node := range d.nodes {
   	if inDegree[node] > 0 {
   	cycleNodes = append(cycleNodes, node)
   	}
   	}
   	sort.Strings(cycleNodes)
   	return nil, fmt.Errorf("cycle detected among nodes: %s", strings.Join(cycleNodes, ", "))
   	}

3. the loop is rejected earlier

   helm/pkg/chart/v2/util/dag.go

   Lines 58 to 81 in 9470c5a

   	// AddEdge adds a directed edge: "to" depends on "from" (from is deployed before to).
   	// Returns an error if either node is unknown or if a self-loop is requested.
   	func (d *DAG) AddEdge(from, to string) error {
   	if from == to {
   	return fmt.Errorf("self-loop not allowed: %q", from)
   	}
   	if _, ok := d.nodes[from]; !ok {
   	return fmt.Errorf("unknown node %q", from)
   	}
   	if _, ok := d.nodes[to]; !ok {
   	return fmt.Errorf("unknown node %q", to)
   	}
   	key := from + "\x00" + to
   	if _, exists := d.edgeSet[key]; exists {
   	return nil
   	}
   	d.edgeSet[key] = struct{}{}
   	d.edges[from] = append(d.edges[from], to)
   	d.inDegree[to]++
   	return nil
   	}

4. the builder calls GetBatches() and wraps the error with domain/chart path to subchart and resource-group cycles are distinguishable

helm/pkg/release/v1/sequence/builder.go

Lines 166 to 170 in 9470c5a

	batches, err := dag.GetBatches()
	if err != nil {
	return fmt.Errorf("subchart circular dependency detected in %s: %w", chartPath, err)
	}

helm/pkg/release/v1/sequence/builder.go

Lines 293 to 297 in 9470c5a

	groupBatches, err := dag.GetBatches()
	if err != nil {
	return fmt.Errorf("resource-group circular dependency detected in %s: %w", chartPath, err)
	}

since the plan is built up-front this should fail fast before any resource is applied (helm install or helm upgrade or helm template —wait=ordered). the check being in dag.go versus subchart_dag.go is easy to miss - I will add a doc to BuildSubchartDAG pointing to GetBatches to make this easier for new reviewers.

[caretak3r]

I have this in the PR, but also want to call out the issue around the double / in the annotations:

helm.sh/depends-on/resource-groups fails Kubernetes annotation-key validation only because it contains two /. The other HIP-0025 annotations (helm.sh/resource-group, helm.sh/readiness-success, helm.sh/readiness-failure) are single-slash and valid. helm.sh/depends-on/subcharts is also two-slash but only ever lives in Chart.yaml, so it never reaches the API server.

I would be able to remove the “workarounds” in the code, and address the two copilot suggestions, if the helm.sh/depends-on/resource-groups could be renamed in the HIP-0025 spec to something like helm.sh/depends-on-resource-groups (open to suggestions on the rename) .

• manifest.go changes and calls in pkg/cmd/template.go
• stripSequencingAnnotations and calls in pkg/action
• stripping the key from the hook resources
• failure case in helm get manifest | kubectl apply