Skip to content

Commit c356732

Browse files
committed
Add BigQuery emulator support and fix docker-compose
- Add emulator detection in BigQueryAdapter - Use AnonymousCredentials when BIGQUERY_EMULATOR_HOST is set - Set API endpoint to emulator using ClientOptions - Fix docker-compose.yml - Remove pre-created dataset (was causing errors) - Remove grpc_health_probe healthcheck (not available in image) - Use service_started condition for bigquery dependency Note: BigQuery integration tests currently hang on emulator connection and need further debugging. Postgres integration tests pass (10/10).
1 parent 96a6fc0 commit c356732

2 files changed

Lines changed: 19 additions & 8 deletions

File tree

docker-compose.yml

Lines changed: 2 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -20,12 +20,7 @@ services:
2020
platform: linux/amd64
2121
ports:
2222
- "9050:9050"
23-
command: ["--project=test-project", "--dataset=test_dataset"]
24-
healthcheck:
25-
test: ["CMD", "grpc_health_probe", "-addr=:9050"]
26-
interval: 5s
27-
timeout: 5s
28-
retries: 5
23+
command: ["--project=test-project"]
2924

3025
test:
3126
build:
@@ -35,7 +30,7 @@ services:
3530
postgres:
3631
condition: service_healthy
3732
bigquery:
38-
condition: service_healthy
33+
condition: service_started
3934
environment:
4035
POSTGRES_TEST: "1"
4136
POSTGRES_URL: "postgres://test:test@postgres:5432/sidemantic_test"

sidemantic/db/bigquery.py

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,23 @@ def __init__(
7777
"Install with: pip install sidemantic[bigquery] or pip install google-cloud-bigquery"
7878
) from e
7979

80-
self.client = bigquery.Client(project=project_id, credentials=credentials, location=location, **kwargs)
80+
# Check if using emulator
81+
import os
82+
83+
emulator_host = os.getenv("BIGQUERY_EMULATOR_HOST")
84+
if emulator_host:
85+
# Use anonymous credentials for emulator
86+
from google.auth.credentials import AnonymousCredentials
87+
from google.api_core.client_options import ClientOptions
88+
89+
# Set API endpoint to emulator
90+
client_options = ClientOptions(api_endpoint=f"http://{emulator_host}")
91+
credentials = AnonymousCredentials()
92+
self.client = bigquery.Client(
93+
project=project_id, credentials=credentials, location=location, client_options=client_options, **kwargs
94+
)
95+
else:
96+
self.client = bigquery.Client(project=project_id, credentials=credentials, location=location, **kwargs)
8197
self.project_id = project_id or self.client.project
8298
self.dataset_id = dataset_id
8399

0 commit comments

Comments
 (0)