fix: redirect browser visits to the base domain, not /mcp

BrowserRedirectMiddleware now matches human GET/HEAD navigations to the
base path (/) instead of /mcp, so the live MCP streamable-HTTP endpoint
(/mcp) and /health are left untouched while someone opening the bare
domain in a browser is 302'd to the docs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: VinciGit00

.agent/system/project_architecture.md

@@ -570,12 +570,13 @@ scrapegraph-mcp
   served at `/mcp`; health check at `/health`. Used by the Render deployment at
   `mcp.scrapegraphai.com`.
 
-**Browser redirect (`/mcp`):**
+**Browser redirect (base domain `/`):**
 - `/mcp` is the live MCP streamable-HTTP endpoint (JSON-RPC `POST`, SSE `GET`
-  with `Accept: text/event-stream`).
+  with `Accept: text/event-stream`) and is left untouched.
 - `BrowserRedirectMiddleware` (in `server.py`, HTTP mode only) redirects **only**
-  human browser navigations — `GET`/`HEAD` on `/mcp` (or `/mcp/`) with
-  `Accept: text/html` — to the docs (`302`). Real MCP traffic is untouched.
+  human browser navigations to the base domain — `GET`/`HEAD` on `/` with
+  `Accept: text/html` — to the docs (`302`). Real MCP traffic (`/mcp`) and the
+  `/health` endpoint are untouched.
 - Target is `https://docs.scrapegraphai.com/services/mcp-server/introduction`,
   overridable via the `MCP_DOCS_URL` env var.
 

src/scrapegraph_mcp/server.py

@@ -94,21 +94,21 @@
 # Matches scrapegraph-py v2 (env.py): https://v2-api.scrapegraphai.com/api
 DEFAULT_API_BASE_URL = "https://v2-api.scrapegraphai.com/api"
 
-# Where to send humans who open the /mcp endpoint in a browser.
+# Where to send humans who open the base domain in a browser.
 DOCS_URL = os.getenv(
     "MCP_DOCS_URL",
     "https://docs.scrapegraphai.com/services/mcp-server/introduction",
 )
 
 
 class BrowserRedirectMiddleware:
-    """Redirect browser visits to the MCP endpoint to the docs.
+    """Redirect browser visits to the base domain to the docs.
 
-    The ``/mcp`` path is the real MCP streamable-HTTP endpoint: clients POST
-    JSON-RPC there and open ``GET`` streams with ``Accept: text/event-stream``.
-    A person pasting ``https://mcp.scrapegraphai.com/mcp`` into a browser sends
-    ``GET`` with ``Accept: text/html`` instead — that (and only that) is
-    redirected to the documentation so real MCP traffic is left untouched.
+    The ``/mcp`` path is the real MCP streamable-HTTP endpoint and is left
+    untouched. A person pasting the base URL ``https://mcp.scrapegraphai.com/``
+    into a browser sends ``GET`` with ``Accept: text/html`` — that (and only
+    that) is redirected to the documentation so real MCP traffic and the
+    ``/health`` endpoint are left untouched.
     """
 
     def __init__(self, app, docs_url: str = DOCS_URL) -> None:
@@ -125,7 +125,7 @@ async def __call__(self, scope, receive, send) -> None:
     def _is_browser_navigation(self, scope) -> bool:
         if scope["method"] not in ("GET", "HEAD"):
             return False
-        if scope["path"].rstrip("/") != "/mcp":
+        if scope["path"].rstrip("/") != "":
             return False
         accept = ""
         for name, value in scope.get("headers", []):