Skip to content

Commit 643b3f5

Browse files
ShaunShaun
Shaun
authored and
Shaun
committed
docs: add OpenShell Google Chat reference architecture
1 parent 02a599b commit 643b3f5

9 files changed

Lines changed: 645 additions & 27 deletions

File tree

‎Cargo.lock‎

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎docs/adr/openshell-websocket-auth.md‎

Lines changed: 306 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,306 @@
1+
# ADR: OpenShell-Compatible Gateway WebSocket Authentication
2+
3+
- **Status:** Proposed
4+
- **Date:** 2026-06-06
5+
- **Author:** OpenAB POC contributors
6+
- **Related:** [ADR: Custom Gateway](./custom-gateway.md), [OpenShell](../openshell.md), [Google Chat](../google-chat.md)
7+
8+
---
9+
10+
## 1. Context
11+
12+
OpenAB's Custom Gateway lets OAB connect outbound over WebSocket while the
13+
gateway owns inbound webhooks and platform credentials. This works well for
14+
Google Chat, LINE, Telegram, and other webhook platforms.
15+
16+
When OAB runs inside an NVIDIA OpenShell sandbox, gateway authentication has an
17+
extra constraint: long-lived secrets should not be placed in the sandbox as raw
18+
environment variables or raw config values.
19+
20+
OpenShell provider credentials appear in the sandbox as resolver placeholders,
21+
for example:
22+
23+
```text
24+
provider-OPENSHELL-RESOLVE-ENV-GATEWAY_WS_TOKEN
25+
openshell:resolve:env:...
26+
```
27+
28+
OpenShell resolves those placeholders at the network boundary only when the
29+
policy proxy can inspect the request. This allows the agent process to use a
30+
credential without ever seeing the raw value.
31+
32+
## 2. Problem
33+
34+
The original gateway auth path appended the token to the WebSocket URL:
35+
36+
```text
37+
ws://gateway/ws?token=<token>
38+
```
39+
40+
That is simple for ordinary Docker/Kubernetes deployments, but it is a poor fit
41+
for OpenShell provider credentials:
42+
43+
- The sandbox config would otherwise need a raw gateway token.
44+
- WebSocket handshake query/header credential rewrite was unreliable in the
45+
OpenShell bridge path used by this POC.
46+
- A Rust WebSocket client may not honor the injected HTTP proxy environment, so
47+
local OpenShell/Docker setups sometimes need a small local bridge to reach the
48+
host gateway.
49+
50+
During the POC, these routes were tested and rejected as the primary OpenShell
51+
path:
52+
53+
```text
54+
[gateway].token = "${GATEWAY_WS_TOKEN}"
55+
[gateway].token = "openshell:resolve:env:GATEWAY_WS_TOKEN"
56+
[gateway].token = "provider-OPENSHELL-RESOLVE-ENV-GATEWAY_WS_TOKEN"
57+
ws://.../ws?token=<placeholder>
58+
Authorization: Bearer <placeholder>
59+
```
60+
61+
OpenShell can rewrite placeholders in normal HTTP requests. For example, a
62+
`curl` request with `Authorization: Bearer <placeholder>` to the gateway's HTTP
63+
endpoint proves header rewrite works when the request is inspected as REST.
64+
However, WebSocket auth in the upgrade handshake is not the right compatibility
65+
point for the OpenShell setup.
66+
67+
OpenShell's documented WebSocket credential rewrite applies to client-to-server
68+
WebSocket text frames after the `101 Switching Protocols` upgrade.
69+
70+
## 3. Decision
71+
72+
Add a first-text-frame gateway authentication message.
73+
74+
OAB connects to the gateway WebSocket without putting the gateway token in the
75+
URL query string. Immediately after the WebSocket upgrade succeeds, OAB sends:
76+
77+
```json
78+
{
79+
"schema": "openab.gateway.auth.v1",
80+
"token": "provider-OPENSHELL-RESOLVE-ENV-GATEWAY_WS_TOKEN"
81+
}
82+
```
83+
84+
When running under OpenShell with:
85+
86+
```text
87+
websocket_credential_rewrite: true
88+
```
89+
90+
the policy proxy rewrites the provider placeholder inside that client-to-server
91+
text frame before forwarding it to the gateway.
92+
93+
The gateway behavior is:
94+
95+
1. If `GATEWAY_WS_TOKEN` is not set, accept WebSocket clients with a warning.
96+
2. If a legacy query token or `Authorization: Bearer` token is present, validate
97+
it before upgrade handling continues.
98+
3. Otherwise, accept the upgrade and require the first client text frame to be
99+
`openab.gateway.auth.v1` with the expected token.
100+
4. If the first frame is missing, malformed, or invalid, close the socket and
101+
do not forward events.
102+
103+
This keeps the old query-token path compatible while enabling a hardened
104+
OpenShell path.
105+
106+
## 4. Reference Architecture
107+
108+
```text
109+
Google Workspace
110+
+-----------------------+
111+
| Google Chat |
112+
| signed HTTPS webhook |
113+
+-----------+-----------+
114+
|
115+
v
116+
+-----------------------------------------------------------------------+
117+
| Host / trusted runtime |
118+
| |
119+
| +-----------------------------+ +---------------------------+ |
120+
| | openab-gateway | | Host secret/token broker | |
121+
| | | | | |
122+
| | - owns webhook endpoint | | - stores SA JSON | |
123+
| | - verifies Google Chat JWT | | - mints short-lived | |
124+
| | - owns Google Chat SA JSON | | Google access tokens | |
125+
| | - owns raw GATEWAY_WS_TOKEN | | - updates OpenShell | |
126+
| +--------------+--------------+ | provider credentials | |
127+
| ^ +-------------+-------------+ |
128+
| | | |
129+
| | validates raw token | stores raw |
130+
| | after rewrite | provider data |
131+
| | v |
132+
| +--------------+--------------------------------------+-------------+ |
133+
| | OpenShell provider store | |
134+
| | GATEWAY_WS_TOKEN, GOOGLE_ACCESS_TOKEN | |
135+
| +--------------+--------------------------------------+-------------+ |
136+
+-----------------+--------------------------------------+---------------+
137+
| |
138+
| WebSocket 101 + text-frame rewrite |
139+
| REST header rewrite for Google APIs |
140+
v v
141+
+-----------------------------------------------------------------------+
142+
| OpenShell sandbox |
143+
| |
144+
| +-----------------------------+ +----------------------------+ |
145+
| | openab | | curl / Google API tools | |
146+
| | | | | |
147+
| | - config has placeholders | | - Authorization header | |
148+
| | - opens ws://gateway/ws | | contains placeholder | |
149+
| | - first text frame sends | | - OpenShell rewrites | |
150+
| | openab.gateway.auth.v1 | | token at egress | |
151+
| +--------------+--------------+ +----------------------------+ |
152+
| | |
153+
| | stdio ACP |
154+
| v |
155+
| +-----------------------------+ |
156+
| | kiro-cli | |
157+
| | | |
158+
| | - agent-owned auth state | |
159+
| | remains local credential | |
160+
| | material | |
161+
| +-----------------------------+ |
162+
+-----------------------------------------------------------------------+
163+
```
164+
165+
Credential flow:
166+
167+
```text
168+
Host/OpenShell provider stores raw GATEWAY_WS_TOKEN
169+
-> sandbox config stores provider placeholder only
170+
-> OAB sends placeholder in first WebSocket text frame
171+
-> OpenShell rewrites placeholder at network boundary
172+
-> gateway validates raw token
173+
```
174+
175+
Google API credential flow:
176+
177+
```text
178+
service-account JSON stays on host/gateway side
179+
-> host broker mints short-lived GOOGLE_ACCESS_TOKEN
180+
-> OpenShell provider stores raw short-lived token
181+
-> sandbox command sends provider placeholder in HTTP Authorization header
182+
-> OpenShell rewrites placeholder at egress
183+
-> Google API receives raw short-lived token
184+
```
185+
186+
## 5. Change From Earlier `openshell.md` Guidance
187+
188+
The earlier OpenShell quick-start pattern focused on running OAB inside an
189+
OpenShell sandbox with provider placeholders for simple outbound APIs such as
190+
Discord. It did not cover webhook platforms where a Custom Gateway receives
191+
inbound traffic, owns platform credentials, and authenticates OAB over
192+
WebSocket.
193+
194+
The new recommendation keeps that provider-placeholder principle but changes
195+
where secrets live and how WebSocket auth is performed.
196+
197+
| Area | Earlier `openshell.md` pattern | New Google Chat/Kiro gateway pattern |
198+
|---|---|---|
199+
| Platform surface | Direct bot API from sandbox, for example Discord gateway/API | Google Chat HTTPS webhook terminates at `openab-gateway` outside OpenShell |
200+
| Platform credentials | Provider placeholder in sandbox config for the bot token | Google Chat service-account JSON stays in the gateway/host layer |
201+
| Gateway auth | Not applicable, or token could be appended to `ws://.../ws?token=...` in non-OpenShell deployments | OAB sends `openab.gateway.auth.v1` as first WebSocket text frame so OpenShell can rewrite the provider placeholder after upgrade |
202+
| `.env` / local secret practice | Easy to accidentally fall back to raw env/config values during local setup | Sandbox config should contain placeholders only; raw values live in OpenShell provider, gateway env, or host token broker |
203+
| Google Drive/Sheets/Docs access | Not covered | Host keeps service-account JSON, mints short-lived Google token, passes only a provider placeholder through sandbox egress |
204+
| Agent-owned auth state | Not addressed | Explicitly documented as a remaining limitation: Kiro's own login cache is still local credential material |
205+
206+
This is not a claim that OpenShell can make all agent secrets disappear. It is a
207+
more precise split:
208+
209+
```text
210+
Injected operator/platform secrets
211+
-> use provider placeholders, gateway ownership, or host token broker
212+
213+
Agent CLI's own login cache
214+
-> remains agent-owned credential material unless the CLI supports brokering
215+
```
216+
217+
## 6. Consequences
218+
219+
Positive:
220+
221+
- The sandbox does not need the raw gateway token.
222+
- OpenShell can enforce which binary may send the credential and to which
223+
endpoint.
224+
- Gateway authentication remains enabled for OpenShell deployments.
225+
- Existing query-token deployments continue to work.
226+
227+
Tradeoffs:
228+
229+
- The gateway briefly accepts the WebSocket upgrade before authentication
230+
completes. It must not send events until the first-frame auth succeeds.
231+
- A malformed or unauthenticated client consumes a socket briefly until the
232+
auth timeout expires or the gateway closes it.
233+
- Operators need a WebSocket policy with credential rewrite enabled.
234+
235+
## 7. Operational Notes
236+
237+
Minimal OpenShell endpoint shape:
238+
239+
```bash
240+
openshell policy update oab \
241+
--add-endpoint gateway-host:8080:read-write:websocket:enforce:websocket-credential-rewrite \
242+
--binary /usr/local/bin/openab \
243+
--wait
244+
245+
openshell policy update oab \
246+
--add-allow 'gateway-host:8080:GET:/ws' \
247+
--wait
248+
```
249+
250+
For local Docker Desktop setups where direct TCP to the host gateway is not
251+
available to the Rust WebSocket client, a small bridge may be used inside the
252+
sandbox. Even then, policy should classify the upstream gateway endpoint rather
253+
than treating the OpenShell HTTP proxy as the application endpoint.
254+
255+
## 8. Security Boundary
256+
257+
Do not pass long-lived platform secrets or service-account JSON files into the
258+
agent sandbox. Keep those in the host gateway or in a host-side token broker.
259+
260+
For Google Drive/Sheets/Docs access, prefer:
261+
262+
```text
263+
service-account JSON on host
264+
-> host mints short-lived OAuth access token
265+
-> OpenShell provider stores GOOGLE_ACCESS_TOKEN
266+
-> sandbox uses provider placeholder in Authorization header
267+
-> OpenShell rewrites at egress
268+
```
269+
270+
This avoids exposing the long-lived service-account private key to the agent.
271+
272+
## 9. Limitation: Agent-Owned Credential Stores
273+
274+
OpenShell provider placeholders protect credentials that the operator injects
275+
through OpenShell, such as gateway tokens or short-lived Google API access
276+
tokens. They do not automatically protect credentials that an agent CLI creates
277+
and persists for itself.
278+
279+
For example, Kiro CLI maintains its own local authentication state. In the POC,
280+
Kiro could still boot after `.aws/sso/cache/*.json` was removed, but failed when
281+
`.local/share/kiro-cli/data.sqlite3` was removed. This means the decisive Kiro
282+
auth state is not only the AWS SSO cache files; it also lives in Kiro's own
283+
SQLite state under `.local/share/kiro-cli/`.
284+
285+
Observed credential-bearing paths include:
286+
287+
```text
288+
/sandbox/.local/share/kiro-cli/data.sqlite3
289+
/sandbox/.aws/sso/cache/*.json
290+
```
291+
292+
Those files are agent-owned credential stores, not OpenShell provider
293+
credentials. Replacing their JSON fields with provider placeholder strings is
294+
not a natural fix unless the agent CLI itself supports placeholder resolution or
295+
external credential brokering. Otherwise, the CLI expects its native local token
296+
format and needs readable auth state to start.
297+
298+
Operational consequence:
299+
300+
- Provider placeholders are appropriate for injected secrets controlled by
301+
OpenAB, the gateway, or a host-side broker.
302+
- They are not sufficient to isolate a third-party agent's own login cache.
303+
- Treat agent auth state such as Kiro's `data.sqlite3` as high-risk credential
304+
material.
305+
- Use a dedicated low-privilege agent identity per bot, avoid shared sandboxes,
306+
and prefer an agent-supported auth proxy/token broker when available.

‎docs/google-chat.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,11 @@ Google Chat ──POST──▶ Gateway (:8080) ◀──WebSocket── OAB Pod
77
(OAB connects out)
88
```
99

10+
For a hardened OpenShell deployment with Kiro, gateway first-frame WebSocket
11+
auth, and Google Drive/Sheets/Docs access through provider-rewritten short-lived
12+
tokens, see
13+
[Google Chat + OpenShell + Kiro Reference Architecture](./refarch/google-chat-openshell-kiro.md).
14+
1015
## Prerequisites
1116

1217
- **A Google Workspace (Business or Enterprise) account** — required by Google to configure the Chat API. Regular `@gmail.com` consumer accounts cannot create Google Chat apps. Workspace Individual or Business Starter is the cheapest qualifying tier. See [Configure the Google Chat API](https://developers.google.com/workspace/chat/configure-chat-api).

‎docs/openshell.md‎

Lines changed: 35 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,10 @@
22

33
Run OAB inside an [NVIDIA OpenShell](https://github.com/NVIDIA/OpenShell) sandbox for isolated, policy-enforced execution.
44

5+
For webhook platforms such as Google Chat, where a Custom Gateway receives
6+
inbound HTTPS webhooks and OAB connects outbound over WebSocket, see
7+
[Google Chat + OpenShell + Kiro Reference Architecture](./refarch/google-chat-openshell-kiro.md).
8+
59
## Architecture
610

711
```
@@ -20,7 +24,7 @@ Run OAB inside an [NVIDIA OpenShell](https://github.com/NVIDIA/OpenShell) sandbo
2024
│ │ Docker Container (sandbox: "oab") │ │
2125
│ │ │ │
2226
│ │ /home/sandbox/ │ │
23-
│ │ └── config.toml ← bot token + agent config │ │
27+
│ │ └── config.toml ← provider placeholders + config │ │
2428
│ │ │ │
2529
│ │ openab run ──stdio JSON-RPC──► openab-agent │ │
2630
│ │ │ │ │ │
@@ -111,10 +115,14 @@ Open the printed URL in your browser, approve, then paste the `localhost:1455/au
111115

112116
### 5. Create config and run
113117

118+
Provider credentials appear in the sandbox as OpenShell resolver placeholders.
119+
Do not copy raw long-lived secrets into `config.toml` when a provider placeholder
120+
can be used instead.
121+
114122
```bash
115123
sandbox$ cat > config.toml <<'EOF'
116124
[discord]
117-
bot_token = "your-bot-token"
125+
bot_token = "provider-OPENSHELL-RESOLVE-ENV-DISCORD_BOT_TOKEN"
118126
allow_all_channels = true
119127
120128
[agent]
@@ -133,6 +141,31 @@ EOF
133141
sandbox$ openab run --config config.toml
134142
```
135143

144+
## Provider Credential Handling
145+
146+
OpenShell provider credentials are safer than ordinary container `.env` values
147+
because the sandbox receives placeholder strings, not raw secret material. The
148+
OpenShell policy proxy can rewrite those placeholders at approved egress
149+
boundaries.
150+
151+
For ordinary HTTP APIs, use the placeholder in headers, query parameters, or
152+
paths according to OpenShell policy. For example:
153+
154+
```bash
155+
curl -H "Authorization: Bearer provider-OPENSHELL-RESOLVE-ENV-API_TOKEN" \
156+
https://api.example.com/v1/resource
157+
```
158+
159+
For Custom Gateway WebSocket authentication, prefer
160+
`openab.gateway.auth.v1` first-frame auth instead of putting the token in the
161+
WebSocket URL. See
162+
[ADR: OpenShell-Compatible Gateway WebSocket Authentication](./adr/openshell-websocket-auth.md).
163+
164+
Avoid mounting long-lived service-account JSON files into the sandbox. For
165+
Google Drive, Sheets, and Docs access, use a host-side token refresher that
166+
stores a short-lived `GOOGLE_ACCESS_TOKEN` in the OpenShell provider, then call
167+
Google APIs from the sandbox with the provider placeholder.
168+
136169
## Network Policy
137170

138171
OpenShell sandboxes have **default-deny egress**. Required endpoints by backend:

0 commit comments

Comments
 (0)