Skip to content

Commit 52f8728

Browse files
nicosuavenicosuave
authored
Add Docker setup with PG and MCP servers (#99)
* Add Docker production setup with PostgreSQL and MCP servers - Dockerfile: Multi-stage build with all database drivers and optional dependencies (serve, mcp, all-databases). 781MB runtime image. - docker-entrypoint.sh: Single container with SIDEMANTIC_MODE env var to control serve/mcp/both modes - .dockerignore: Excludes unnecessary files for faster builds - Add --host parameter to sidemantic serve CLI and config for container networking (0.0.0.0 support) - README: Comprehensive Docker usage section with examples Server fully tested: psql connections work, semantic layer tables queryable, demo mode verified. * Add Docker CI workflow Builds the image, starts the PG server in demo mode, and runs psql queries to verify semantic layer tables are accessible. * Test semantic layer metric aggregation in Docker CI Replace passthrough SELECT * queries with GROUP BY queries that exercise metric computation (product_count, avg_price, customer_count). * Add Docker Hub push to CI workflow Pushes sidequery/sidemantic:latest and :version tags on main after tests pass. Uses GHA build cache and buildx. Requires DOCKERHUB_USERNAME and DOCKERHUB_TOKEN repo secrets. * Run Docker CI after PyPI publish or on manual dispatch No longer triggers on every push/PR. Runs automatically after a successful PyPI publish, or manually via workflow_dispatch. * Fix entrypoint shell splitting and mcp-serve arg mismatch Quote all env var expansions to handle special characters in connection strings and credentials. Only pass --db to mcp-serve (it doesn't accept --connection, --username, --password, or --port).
1 parent f7f44b7 commit 52f8728

8 files changed

Lines changed: 292 additions & 5 deletions

File tree

‎.dockerignore‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
.git
2+
.github
3+
.venv
4+
.mypy_cache
5+
.pytest_cache
6+
.ruff_cache
7+
__pycache__
8+
*.egg-info
9+
*.pyc
10+
11+
# Rust/DuckDB extension builds (not needed for Python image)
12+
sidemantic-rs/
13+
sidemantic-duckdb/
14+
15+
# Docs and examples not needed at runtime
16+
docs/
17+
examples/superset_demo/
18+
examples/rill_demo/
19+
examples/cube_demo/
20+
21+
# Dev/test artifacts
22+
Dockerfile.test
23+
docker-compose.yml
24+
*.md
25+
!README.md
26+
.context/

‎.github/workflows/docker.yml‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
name: Docker
2+
3+
on:
4+
workflow_dispatch:
5+
workflow_run:
6+
workflows: ["Publish to PyPI"]
7+
types: [completed]
8+
9+
env:
10+
IMAGE: sidequery/sidemantic
11+
12+
jobs:
13+
docker:
14+
name: Docker build, test, and push
15+
runs-on: ubuntu-latest
16+
if: ${{ github.event_name == 'workflow_dispatch' || github.event.workflow_run.conclusion == 'success' }}
17+
18+
steps:
19+
- uses: actions/checkout@v4
20+
with:
21+
ref: main
22+
23+
- name: Set up Docker Buildx
24+
uses: docker/setup-buildx-action@v3
25+
26+
- name: Login to Docker Hub
27+
uses: docker/login-action@v3
28+
with:
29+
username: ${{ secrets.DOCKERHUB_USERNAME }}
30+
password: ${{ secrets.DOCKERHUB_TOKEN }}
31+
32+
- name: Extract version
33+
id: version
34+
run: |
35+
VERSION=$(grep '^version = ' pyproject.toml | sed 's/version = "\(.*\)"/\1/')
36+
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
37+
echo "Version: $VERSION"
38+
39+
- name: Build image
40+
uses: docker/build-push-action@v6
41+
with:
42+
context: .
43+
load: true
44+
tags: ${{ env.IMAGE }}:test
45+
cache-from: type=gha
46+
cache-to: type=gha,mode=max
47+
48+
- name: Start server (demo mode)
49+
run: |
50+
docker run -d --name sidemantic-test -p 5433:5433 ${{ env.IMAGE }}:test --demo
51+
for i in $(seq 1 30); do
52+
if docker logs sidemantic-test 2>&1 | grep -q "Listening on"; then
53+
echo "Server ready"
54+
break
55+
fi
56+
echo "Waiting for server... ($i/30)"
57+
sleep 1
58+
done
59+
60+
- name: Verify server logs
61+
run: docker logs sidemantic-test 2>&1
62+
63+
- name: Install psql
64+
run: sudo apt-get update && sudo apt-get install -y postgresql-client
65+
66+
- name: Test connection
67+
run: PGPASSWORD=any psql -h localhost -p 5433 -U any -d sidemantic -c "SELECT 1 AS test"
68+
69+
- name: Test metric aggregation (products)
70+
run: |
71+
PGPASSWORD=any psql -h localhost -p 5433 -U any -d sidemantic -c \
72+
"SELECT category, product_count, avg_price, total_catalog_value FROM semantic_layer.products GROUP BY category ORDER BY category"
73+
74+
- name: Test metric aggregation (customers)
75+
run: |
76+
PGPASSWORD=any psql -h localhost -p 5433 -U any -d sidemantic -c \
77+
"SELECT region, customer_count FROM semantic_layer.customers GROUP BY region ORDER BY region"
78+
79+
- name: Stop server
80+
if: always()
81+
run: docker stop sidemantic-test || true
82+
83+
- name: Push to Docker Hub
84+
uses: docker/build-push-action@v6
85+
with:
86+
context: .
87+
push: true
88+
tags: |
89+
${{ env.IMAGE }}:latest
90+
${{ env.IMAGE }}:${{ steps.version.outputs.version }}
91+
cache-from: type=gha

‎Dockerfile‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
FROM python:3.12-slim AS builder
2+
3+
# Install build deps for riffq (Rust/maturin) and other native extensions
4+
RUN apt-get update && apt-get install -y --no-install-recommends \
5+
build-essential \
6+
cmake \
7+
pkg-config \
8+
libssl-dev \
9+
&& rm -rf /var/lib/apt/lists/*
10+
11+
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
12+
13+
WORKDIR /app
14+
15+
COPY pyproject.toml README.md LICENSE ./
16+
COPY sidemantic/ sidemantic/
17+
COPY examples/ examples/
18+
19+
RUN uv pip install --system --no-cache ".[serve,mcp,all-databases]"
20+
21+
# --- Runtime stage (no build tools) ---
22+
FROM python:3.12-slim
23+
24+
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
25+
26+
# Copy installed packages from builder
27+
COPY --from=builder /usr/local/lib/python3.12/site-packages /usr/local/lib/python3.12/site-packages
28+
COPY --from=builder /usr/local/bin/sidemantic /usr/local/bin/sidemantic
29+
30+
WORKDIR /app
31+
32+
COPY docker-entrypoint.sh /docker-entrypoint.sh
33+
RUN chmod +x /docker-entrypoint.sh
34+
35+
RUN mkdir -p /app/models
36+
WORKDIR /app/models
37+
38+
EXPOSE 5433
39+
40+
ENTRYPOINT ["/docker-entrypoint.sh"]
41+
# Mode is controlled by SIDEMANTIC_MODE env var (serve, mcp, both)

‎README.md‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -252,6 +252,77 @@ load_from_directory(layer, "my_models/") # Auto-detects formats
252252
| Databricks | ✅ | `uv add sidemantic[databricks]` |
253253
| Spark SQL | ✅ | `uv add sidemantic[spark]` |
254254

255+
## Docker
256+
257+
Build the image (includes all database drivers, PG server, and MCP server):
258+
259+
```bash
260+
docker build -t sidemantic .
261+
```
262+
263+
### PostgreSQL server (default)
264+
265+
Mount your models directory and expose port 5433:
266+
267+
```bash
268+
docker run -p 5433:5433 -v ./models:/app/models sidemantic
269+
```
270+
271+
Connect with any PostgreSQL client:
272+
273+
```bash
274+
psql -h localhost -p 5433 -U any -d sidemantic
275+
```
276+
277+
With a backend database connection:
278+
279+
```bash
280+
docker run -p 5433:5433 \
281+
-v ./models:/app/models \
282+
-e SIDEMANTIC_CONNECTION="postgres://user:pass@host:5432/db" \
283+
sidemantic
284+
```
285+
286+
### MCP server
287+
288+
```bash
289+
docker run -v ./models:/app/models -e SIDEMANTIC_MODE=mcp sidemantic
290+
```
291+
292+
### Both servers simultaneously
293+
294+
Runs the PG server in the background and MCP on stdio:
295+
296+
```bash
297+
docker run -p 5433:5433 -v ./models:/app/models -e SIDEMANTIC_MODE=both sidemantic
298+
```
299+
300+
### Demo mode
301+
302+
```bash
303+
docker run -p 5433:5433 sidemantic --demo
304+
```
305+
306+
### Baking models into the image
307+
308+
Create a `Dockerfile` that copies your models at build time:
309+
310+
```dockerfile
311+
FROM sidemantic
312+
COPY my_models/ /app/models/
313+
```
314+
315+
### Environment variables
316+
317+
| Variable | Description |
318+
|----------|-------------|
319+
| `SIDEMANTIC_MODE` | `serve` (default), `mcp`, or `both` |
320+
| `SIDEMANTIC_CONNECTION` | Database connection string |
321+
| `SIDEMANTIC_DB` | Path to DuckDB file (inside container) |
322+
| `SIDEMANTIC_USERNAME` | PG server auth username |
323+
| `SIDEMANTIC_PASSWORD` | PG server auth password |
324+
| `SIDEMANTIC_PORT` | PG server port (default 5433) |
325+
255326
## Testing
256327

257328
```bash

‎docker-entrypoint.sh‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
#!/bin/sh
2+
set -e
3+
4+
# SIDEMANTIC_MODE: "serve" (default), "mcp", or "both"
5+
MODE="${SIDEMANTIC_MODE:-serve}"
6+
7+
# Build arg arrays for each command.
8+
# serve accepts: --connection, --db, --host, --port, --username, --password
9+
# mcp-serve accepts: --db only
10+
11+
# Serve args
12+
SERVE_ARGS="--host 0.0.0.0"
13+
if [ -n "$SIDEMANTIC_CONNECTION" ]; then
14+
SERVE_ARGS="$SERVE_ARGS --connection \"$SIDEMANTIC_CONNECTION\""
15+
fi
16+
if [ -n "$SIDEMANTIC_DB" ]; then
17+
SERVE_ARGS="$SERVE_ARGS --db \"$SIDEMANTIC_DB\""
18+
fi
19+
if [ -n "$SIDEMANTIC_USERNAME" ]; then
20+
SERVE_ARGS="$SERVE_ARGS --username \"$SIDEMANTIC_USERNAME\""
21+
fi
22+
if [ -n "$SIDEMANTIC_PASSWORD" ]; then
23+
SERVE_ARGS="$SERVE_ARGS --password \"$SIDEMANTIC_PASSWORD\""
24+
fi
25+
if [ -n "$SIDEMANTIC_PORT" ]; then
26+
SERVE_ARGS="$SERVE_ARGS --port \"$SIDEMANTIC_PORT\""
27+
fi
28+
29+
# MCP args (only --db is supported)
30+
MCP_ARGS=""
31+
if [ -n "$SIDEMANTIC_DB" ]; then
32+
MCP_ARGS="$MCP_ARGS --db \"$SIDEMANTIC_DB\""
33+
fi
34+
35+
case "$MODE" in
36+
serve)
37+
eval exec sidemantic serve $SERVE_ARGS "$@"
38+
;;
39+
mcp)
40+
eval exec sidemantic mcp-serve $MCP_ARGS "$@"
41+
;;
42+
both)
43+
eval sidemantic serve $SERVE_ARGS &
44+
SERVE_PID=$!
45+
trap "kill $SERVE_PID 2>/dev/null" EXIT
46+
eval exec sidemantic mcp-serve $MCP_ARGS "$@"
47+
;;
48+
*)
49+
echo "Unknown SIDEMANTIC_MODE: $MODE (use serve, mcp, or both)" >&2
50+
exit 1
51+
;;
52+
esac

‎sidemantic/cli.py‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -422,6 +422,7 @@ def serve(
422422
None, "--connection", help="Database connection string (e.g., postgres://host/db, bigquery://project/dataset)"
423423
),
424424
db: Path = typer.Option(None, "--db", help="Path to DuckDB database file (shorthand for duckdb:/// connection)"),
425+
host: str = typer.Option(None, "--host", "-H", help="Host/IP to bind to (overrides config, default 127.0.0.1)"),
425426
port: int = typer.Option(None, "--port", "-p", help="Port to listen on (overrides config)"),
426427
username: str = typer.Option(None, "--username", "-u", help="Username for authentication (overrides config)"),
427428
password: str = typer.Option(None, "--password", help="Password for authentication (overrides config)"),
@@ -482,7 +483,8 @@ def serve(
482483
# Use connection from config
483484
connection_str = build_connection_string(_loaded_config)
484485

485-
# Resolve port, username, password from args or config
486+
# Resolve host, port, username, password from args or config
487+
host_resolved = host or (_loaded_config.pg_server.host if _loaded_config else "127.0.0.1")
486488
port_resolved = port if port is not None else (_loaded_config.pg_server.port if _loaded_config else 5433)
487489
username_resolved = username or (_loaded_config.pg_server.username if _loaded_config else None)
488490
password_resolved = password or (_loaded_config.pg_server.password if _loaded_config else None)
@@ -535,7 +537,7 @@ def serve(
535537
layer.conn.executemany(f"INSERT INTO {table} VALUES ({placeholders})", rows)
536538

537539
# Start the server
538-
start_server(layer, port=port_resolved, username=username_resolved, password=password_resolved)
540+
start_server(layer, host=host_resolved, port=port_resolved, username=username_resolved, password=password_resolved)
539541

540542

541543
@app.command(hidden=True)

‎sidemantic/config.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,7 @@ class PostgresServerConfig(BaseModel):
104104
This feature is experimental and may change.
105105
"""
106106

107+
host: str = Field(default="127.0.0.1", description="Host/IP to bind to (use 0.0.0.0 for Docker)")
107108
port: int = Field(default=5433, description="Port to listen on")
108109
username: str | None = Field(default=None, description="Username for authentication (optional)")
109110
password: str | None = Field(default=None, description="Password for authentication (optional)")

‎sidemantic/server/server.py‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ def map_type(duckdb_type: str) -> str:
3333

3434
def start_server(
3535
layer: SemanticLayer,
36+
host: str = "127.0.0.1",
3637
port: int = 5433,
3738
username: str | None = None,
3839
password: str | None = None,
@@ -41,6 +42,7 @@ def start_server(
4142
4243
Args:
4344
layer: Semantic layer instance
45+
host: Host/IP to bind to (use 0.0.0.0 for Docker)
4446
port: Port to listen on
4547
username: Username for authentication (optional)
4648
password: Password for authentication (optional)
@@ -52,7 +54,7 @@ def __init__(self, connection_id, executor):
5254
super().__init__(connection_id, executor, layer, username, password)
5355

5456
# Start server
55-
server = riffq.RiffqServer(f"127.0.0.1:{port}", connection_cls=BoundConnection)
57+
server = riffq.RiffqServer(f"{host}:{port}", connection_cls=BoundConnection)
5658

5759
# Register catalog
5860
typer.echo("Registering semantic layer catalog...", err=True)
@@ -134,12 +136,13 @@ def __init__(self, connection_id, executor):
134136
server._server.register_table("sidemantic", "semantic_layer", "metrics", metric_columns)
135137
typer.echo(" Registered table: semantic_layer.metrics", err=True)
136138

137-
typer.echo(f"\nStarting PostgreSQL-compatible server on 127.0.0.1:{port}", err=True)
139+
typer.echo(f"\nStarting PostgreSQL-compatible server on {host}:{port}", err=True)
138140
if username:
139141
typer.echo(f"Authentication: username={username}", err=True)
140142
else:
141143
typer.echo("Authentication: disabled (any username/password accepted)", err=True)
142-
typer.echo(f"\nConnect with: psql -h 127.0.0.1 -p {port} -U {username or 'any'} -d sidemantic\n", err=True)
144+
connect_host = "localhost" if host == "0.0.0.0" else host
145+
typer.echo(f"\nConnect with: psql -h {connect_host} -p {port} -U {username or 'any'} -d sidemantic\n", err=True)
143146

144147
# Disable catalog_emulation and handle catalog queries manually in our Python handler
145148
# This prevents riffq from intercepting queries with its DataFusion parser (which fails on multi-statement queries)

0 commit comments

Comments
 (0)