Skip to content

Stable weight is calculated incorrectly when canary and experiments are combined #225

Description

@southball

Checklist:

  • I've included steps to reproduce the bug.
  • I've included the version of argo rollouts.
  • I've included the version of the gateway API plugin
  • I've included the name and version of the gateway API provider I am using
  • I've included my Rollout Spec (anonymized if needed)

Describe the bug

Stable weight is calculated incorrectly when canary and experiments are combined

To Reproduce

Apply the following manifest along with the supporting resources below:

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: nginx-weight-test
  namespace: default
spec:
  replicas: 4
  revisionHistoryLimit: 2
  selector:
    matchLabels:
      app: nginx-weight-test
  strategy:
    canary:
      stableService: nginx-weight-stable
      canaryService: nginx-weight-canary
      trafficRouting:
        plugins:
          argoproj-labs/gatewayAPI:
            httpRoute: nginx-weight-route
            namespace: default
      steps:
      - setWeight: 20
      - experiment:
          duration: 10m
          templates:
          - name: baseline
            specRef: stable
            weight: 10
            service:
              name: nginx-weight-exp-baseline
          - name: candidate
            specRef: canary
            weight: 10
            service:
              name: nginx-weight-exp-candidate
      - pause: {}
  template:
    metadata:
      labels:
        app: nginx-weight-test
    spec:
      containers:
      - name: nginx
        image: nginx:1.27-alpine
        ports:
        - name: http
          containerPort: 80
Kubernetes manifest for supporting resources
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: experiment-repro-gateway
  namespace: default
spec:
  gatewayClassName: istio
  listeners:
  - name: http
    protocol: HTTP
    port: 80
    allowedRoutes:
      namespaces:
        from: Same
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-weight-stable
  namespace: default
spec:
  selector:
    app: nginx-weight-test
  ports:
  - name: http
    port: 80
    targetPort: http
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-weight-canary
  namespace: default
spec:
  selector:
    app: nginx-weight-test
  ports:
  - name: http
    port: 80
    targetPort: http
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: nginx-weight-route
  namespace: default
spec:
  parentRefs:
  - name: experiment-repro-gateway
  hostnames:
  - nginx-weight.test
  rules:
  - backendRefs:
    - name: nginx-weight-stable
      port: 80
      weight: 100
    - name: nginx-weight-canary
      port: 80
      weight: 0

Expected behavior

The weight should be

nginx-weight-stable=60
nginx-weight-canary=20
nginx-weight-exp-baseline=10
nginx-weight-exp-candidate=10

Screenshots

The actual weight is

nginx-weight-stable=80
nginx-weight-canary=20
nginx-weight-exp-baseline=10
nginx-weight-exp-candidate=10

See nginx-weight-stable has weight 80 instead of 60.

Version

Argo Rollouts v1.9.1, Gateway API plugin aa218b8fbfe6a92cf883c4aad7ff7d9334d17269 (2026-08-05, built locally)

Latest main

Additional Notes

Using Istio directly in Argo Rollouts we get this

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: nginx-istio-test
  namespace: default
spec:
  replicas: 4
  revisionHistoryLimit: 2
  selector:
    matchLabels:
      app: nginx-istio-test
  strategy:
    canary:
      stableService: nginx-istio-stable
      canaryService: nginx-istio-canary
      trafficRouting:
        istio:
          virtualService:
            name: nginx-istio-route
            routes:
            - primary
      steps:
      - setWeight: 20
      - experiment:
          duration: 10m
          templates:
          - name: baseline
            specRef: stable
            weight: 10
            service:
              name: nginx-istio-exp-baseline
          - name: candidate
            specRef: canary
            weight: 10
            service:
              name: nginx-istio-exp-candidate
      - pause: {}
  template:
    metadata:
      labels:
        app: nginx-istio-test
    spec:
      containers:
      - name: nginx
        image: nginx:1.27-alpine
        ports:
        - name: http
          containerPort: 80
Kubernetes manifest for supporting resources
apiVersion: v1
kind: Service
metadata:
  name: nginx-istio-stable
  namespace: default
spec:
  selector:
    app: nginx-istio-test
  ports:
  - name: http
    port: 80
    targetPort: http
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-istio-canary
  namespace: default
spec:
  selector:
    app: nginx-istio-test
  ports:
  - name: http
    port: 80
    targetPort: http
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: nginx-istio-route
  namespace: default
spec:
  hosts:
  - nginx-istio.test
  gateways:
  - mesh
  http:
  - name: primary
    route:
    - destination:
        host: nginx-istio-stable.default.svc.cluster.local
        port:
          number: 80
      weight: 100
    - destination:
        host: nginx-istio-canary.default.svc.cluster.local
        port:
          number: 80
      weight: 0

(Snippet from kubectl describe virtualservice nginx-istio-route)

Spec:
  Gateways:
    mesh
  Hosts:
    nginx-istio.test
  Http:
    Name:  primary
    Route:
      Destination:
        Host:  nginx-istio-stable.default.svc.cluster.local
        Port:
          Number:  80
      Weight:      60
      Destination:
        Host:  nginx-istio-canary.default.svc.cluster.local
        Port:
          Number:  80
      Weight:      20
      Destination:
        Host:  nginx-istio-exp-baseline
      Weight:  10
      Destination:
        Host:  nginx-istio-exp-candidate
      Weight:  10

Logs

# Paste the logs from the rollout controller

# Logs for the entire controller:
kubectl logs -n argo-rollouts deployment/argo-rollouts

# Logs for a specific rollout:
kubectl logs -n argo-rollouts deployment/argo-rollouts | grep rollout=<ROLLOUTNAME

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritize the issues with the most 👍.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions