Skip to content

Connection resilience: reconnection behavior after network lossΒ #2203

Description

@n-lark

Connection resilience: reconnection behavior after network loss

User story

As an operator running dashboards on always-on screens, I want a screen that lost the server to recover on its own, so that nobody has to walk the floor after a network blip or a server restart.

Product Requirements

Release headline πŸ“Ά Connection resilience
Pillar Build
Packaging All β€” open source npm package, no tier or feature flag
Dependencies None. Independent of tags and Canvas
Closes #1344, #2074, #2149, #2068, #2135
Success A dashboard left through a server outage of any length reconnects unattended, and a proxy session expiry sends you to login rather than hanging
KPI No new reports of screens needing a manual touch after an outage, across the two releases following
Out of scope Stale widgets after reconnect (#913, #1251 β€” solved by tags); chart history after a Node-RED restart (#1362); server-side scaling
Marketing No standalone blog. A section in the release blog, phrased as capability
Enablement Release-notes paragraph for support. No new config surface, so no Expert enablement
Feature brief To be written

Demo points

The problem

A dashboard that loses its server doesn't recover on its own. On a laptop you refresh. On an always-on display someone walks to the screen.

From #1344:

50 displays showing dashboards. Server maintenance takes it offline for ten minutes. All 50 screens attempt a refresh, land on ERR_NAME_NOT_RESOLVED, and each one then needs a manual touch to come back.

The complaint is the blind refresh, not the retry timing β€” "blindly refreshing the page without the server even being alive".

Three defects in reconnect():

  1. No jitter. Every client retries on the same schedule β€” 2.5s Γ—4, 5s Γ—10, 30s Γ—8 β€” so 50 screens hit the server in lockstep.
  2. It gives up. After MAX_RETRIES (22, ~5 min) forcePageReload() hard-reloads. If the server is still down the screen lands on a browser error page and stays there.
  3. parser error also force-reloads. The auth-proxy case. The code comment calls the reload a fallback β€” "it doesn't seem that we can do much other than force a reload" β€” but PR Handle redirection to login page if session has expired when websocket connection is attemptedΒ #2078 shows the redirect can be detected and followed.

Child tickets

Merge order: #2152 β†’ #2078 β†’ #1344 on top. All three edit the same ~20 lines in reconnect() and forcePageReload().

Constraint to preserve: the retry is count-based rather than elapsed-time on purpose. Android suspends the page so timers don't fire, and a count-based scheme retries promptly on wake (noted on #1344).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

epicA significant feature or piece of work that doesn't easily fit into a single release

Type

No type

Projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions