Skip to content

Commit 9ad444a

Browse files
jableyjabley
committed
Add architecture documentation
This is possibly a moving target but I wanted to get capture my understanding whilst preparing to take this to the Technical Review Group - https://architecture.digital.nhs.uk/trg
1 parent 6e0c6c9 commit 9ad444a

2 files changed

Lines changed: 195 additions & 24 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -126,30 +126,7 @@ The gateway is designed to run as a set of Windows Services on-premises.
126126

127127
## Architecture
128128

129-
This gateway implements a lightweight DICOM service architecture:
130-
131-
1. **DICOM Worklist Server** - Provides scheduled procedure information to modalities
132-
2. **DICOM PACS Server** - Receives and stores medical images ([docs](docs/pacs/README.md))
133-
3. **Event Processing** - Processes MPPS status updates and image metadata
134-
4. **Azure Relay Communication** - Bidirectional communication with cloud service
135-
136-
### PACS Server
137-
138-
The PACS server provides C-STORE functionality for receiving medical images:
139-
- Hash-based storage for scalability
140-
- SQLite metadata indexing
141-
- Thread-safe concurrent access
142-
- Docker containerized deployment
143-
144-
See [PACS documentation](docs/pacs/README.md) for detailed information.
145-
146-
### Relay Listener
147-
148-
The Relay Listener handles incoming messages from the cloud service via Azure Relay:
149-
- Listens on configured Hybrid Connection
150-
- Processes worklist actions (e.g., create worklist item)
151-
152-
See [Relay Listener documentation](docs/relay-listener/README.md) for details.
129+
See [docs/architecture.md](docs/architecture.md)
153130

154131
## Testing
155132

‎docs/architecture.md‎

Lines changed: 194 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,194 @@
1+
# Gateway Architecture and Data Flows
2+
3+
This document summarises the architecture of the Rubie Gateway, its protocols and data flows.
4+
5+
## Overview
6+
7+
This gateway implements a lightweight DICOM service architecture:
8+
9+
1. **DICOM Worklist Server** - Provides scheduled procedure information to modalities ([docs](mwl/README.md))
10+
2. **DICOM PACS Server** - Receives and stores medical images ([docs](pacs/README.md))
11+
3. **Event Processing** - Processes appointment status updates and image metadata ([docs](upload/README.md))
12+
4. **Azure Relay Communication** - Bidirectional communication with cloud service ([docs](relay-listener/README.md))
13+
14+
### Modality Worklist (MWL) Server
15+
16+
The MWL server provides `C-FIND` functionality for modalities to query the scheduled procedure information.
17+
18+
See [MWL documentation](mwl/README.md) for detailed information.
19+
20+
### PACS Server
21+
22+
The PACS server provides `C-STORE` functionality for receiving medical images:
23+
24+
- Content-addressable storage
25+
- SQLite metadata indexing
26+
- Thread-safe concurrent access
27+
28+
See [PACS documentation](pacs/README.md) for detailed information.
29+
30+
### Relay Listener
31+
32+
The Relay Listener handles incoming messages from the cloud service via Azure Relay:
33+
34+
- Listens on configured Hybrid Connection
35+
- Processes worklist actions (e.g., create worklist item)
36+
37+
See [Relay Listener documentation](relay-listener/README.md) for details.
38+
39+
The gateway bridges on-premises breast screening modalities with the cloud-based Rubie platform.
40+
41+
Core runtime services:
42+
43+
- PACS Server (`C-STORE` receiver)
44+
- Modality Worklist (MWL) Server (`C-FIND` responder)
45+
- Relay Listener (Azure Relay WebSocket listener over outbound HTTPS)
46+
- Upload Service (background poller and HTTP uploader)
47+
48+
## Topology
49+
50+
```mermaid
51+
graph TB
52+
subgraph "On-Premises"
53+
subgraph "Trust IT"
54+
MOD["Modality\nMammography System"]
55+
end
56+
57+
subgraph "Gateway"
58+
PACS["PACS Server\nPort 4244\nDICOM C-STORE SCP\nListening"]
59+
MWL["MWL Server\nPort 4243\nDICOM C-FIND SCP\nListening"]
60+
RELAY["Relay Listener\nAzure Relay\nListening over outbound HTTPS"]
61+
UPLOAD["Upload Service\nPolling loop"]
62+
63+
PACSDB[("PACS SQLite DB\n+ DICOM file storage")]
64+
MWLDB[("MWL SQLite DB")]
65+
end
66+
end
67+
68+
subgraph "Cloud"
69+
RUBIE["Rubie API"]
70+
AR["Azure Relay"]
71+
end
72+
73+
MOD -->|"C-STORE Images"| PACS
74+
MOD -->|"C-FIND Worklist Query"| MWL
75+
76+
PACS --> PACSDB
77+
MWL --> MWLDB
78+
79+
AR -->|"Worklist actions"| RELAY
80+
RELAY --> MWLDB
81+
82+
UPLOAD -->|"Poll pending uploads"| PACSDB
83+
UPLOAD -->|"Lookup source_message_id"| MWLDB
84+
UPLOAD -->|"HTTP multipart upload"| RUBIE
85+
```
86+
87+
## Protocols, Inputs, Outputs, and Transfer Direction
88+
89+
| Service | Input | Output | Protocol(s) | Listening or Initiating? |
90+
| --- | --- | --- | --- | --- |
91+
| PACS Server | DICOM image objects from modality | Compressed/stored DICOM + PACS metadata rows | DICOM `C-STORE` (+ `C-ECHO`) | **Listening** on default port `4244` |
92+
| MWL Server | Worklist queries from modality | Matching worklist records | DICOM `C-FIND` (+ MPPS updates) | **Listening** on default port `4243` |
93+
| Relay Listener | GatewayAction-style worklist messages from cloud service | Created/updated MWL worklist items | Azure Relay WebSocket over outbound HTTPS (`443`) | **Listening** via persistent outbound connection |
94+
| Upload Service | Pending PACS rows + DICOM files + MWL linkage data | Uploaded image payloads and status updates | HTTPS `POST` multipart/form-data | **Initiating** outbound HTTP requests |
95+
96+
## End-to-End Data Flows
97+
98+
### 1. Worklist down to modality
99+
100+
This happens via 2 flows:
101+
102+
1. Rubie service emits worklist action via [Azure Relay websocket](https://learn.microsoft.com/en-us/azure/azure-relay/relay-hybrid-connections-protocol).
103+
2. Gateway relay listener receives message and writes MWL item to MWL SQLite DB.
104+
3. Separately, the Modality sends DICOM `C-FIND` query to MWL server.
105+
4. MWL server responds with matching scheduled procedure items.
106+
107+
```mermaid
108+
graph TB
109+
subgraph "Cloud"
110+
RUBIE["Rubie API"]
111+
AR["Azure Relay"]
112+
end
113+
114+
subgraph "On Premises"
115+
subgraph "Trust IT"
116+
MOD["Modality\nMammography System"]
117+
end
118+
119+
subgraph "Gateway"
120+
MWL["MWL Server\nPort 4243\nDICOM C-FIND SCP\nListening"]
121+
RELAY["Relay Listener\nAzure Relay\nListening over outbound HTTPS"]
122+
123+
MWLDB[("MWL SQLite DB")]
124+
end
125+
end
126+
127+
RUBIE -->|"1. Emit worklist actions"| AR
128+
AR -->|"Emit worklist actions"| RELAY
129+
RELAY -->|"2. Store MWL item"| MWLDB
130+
131+
MOD -->|"3. C-FIND"| MWL
132+
MWL -->|"4. Retrieve worklist items"| MWLDB
133+
```
134+
135+
### 2. Images up to cloud
136+
137+
This also happens via 2 flows:
138+
139+
1. Modality sends DICOM `C-STORE` to PACS server.
140+
2. PACS validates tags and pixel data, decompresses if required, resizes, and recompresses (JPEG 2000 lossy).
141+
3. PACS stores file in content-addressable filesystem layout and records metadata in PACS SQLite DB.
142+
4. In parallel, the Upload service polls for `PENDING` items, reads the DICOM file, and finds `source_message_id` via MWL data.
143+
5. Upload service sends multipart HTTPS upload to Rubie API with `X-Source-Message-ID`.
144+
6. Upload status is updated (`COMPLETE` or retried/fails after max retries).
145+
146+
```mermaid
147+
graph TB
148+
subgraph "Cloud"
149+
RUBIE["Rubie API"]
150+
AR["Azure Relay"]
151+
end
152+
153+
subgraph "On Premises"
154+
subgraph "Trust IT"
155+
MOD["Modality\nMammography System"]
156+
end
157+
158+
subgraph "Gateway"
159+
PACS["PACS Server\nPort 4244\nDICOM C-STORE SCP\nListening"]
160+
UPLOAD["Upload Service\nPolling loop"]
161+
162+
PACSDB[("PACS SQLite DB\n+ DICOM file storage")]
163+
MWLDB[("MWL SQLite DB")]
164+
end
165+
end
166+
167+
MOD -->|"1. C-STORE"| PACS
168+
PACS -->|"2. Validate"| PACS
169+
PACS -->|"3. Store"| PACSDB
170+
171+
UPLOAD -->|"4. Poll"| MWLDB
172+
UPLOAD -->|"4. Read DICOM file"| PACSDB
173+
UPLOAD -->|"5. upload"| RUBIE
174+
UPLOAD --> |"6. Update status"| MWLDB
175+
```
176+
177+
### 3. Worklist status progression
178+
179+
- `SCHEDULED` when created from relay message.
180+
- Moves to `IN PROGRESS` when image activity begins (`C-STORE` observed).
181+
- Finalised by MPPS events (`COMPLETED` or `DISCONTINUED`).
182+
- Daily backup-and-clear is run by scheduled task (`reset_main.py`), preparing next day’s workload.
183+
184+
## Storage and Processing Notes
185+
186+
- PACS file storage uses content-addressable paths derived from `SOPInstanceUID`.
187+
- PACS metadata and MWL data are stored in separate SQLite databases.
188+
- Compression and resizing are intentional to support thumbnail/display workflows in the cloud UI; full-resolution clinical images remain in BSU internal PACS.
189+
190+
## Deployment Characteristics
191+
192+
- PACS and MWL are designed to run as separate services/containers.
193+
- Gateway can operate behind strict firewalls because relay communication is maintained via outbound HTTPS.
194+
- Upload is asynchronous and retry-based (polling + exponential backoff).

0 commit comments

Comments
 (0)