feat: Add ui.page_html() and express.ui.page_opts(html=) for a complete, author-owned HTML document as the UI (#2475)

Co-authored-by: Carson Sievert <cpsievert1@gmail.com>

Author: schloerke

.claude/references/architecture.md

@@ -82,6 +82,30 @@ HTML is generated using the `htmltools` package:
 Asset vendoring (bslib CSS/JS, theme presets, `make upgrade-html-deps`) is
 covered in `.claude/references/assets.md`.
 
+### App UI types
+
+`App(ui=)` accepts a `Tag`/`TagList`, a `Path` to a complete HTML file, a
+`ui.page_html()` result (a complete HTML document with Shiny's dependencies
+prefixed onto the app author's), or a function taking a `Request` and returning
+any of those (except for `Path`, see below).
+
+**Every UI *value* type (except for `Path`) must work in both positions: passed
+directly, and returned by a UI function.** The function form is what bookmarking requires --
+`App._init_bookmarking()` rejects a static UI, since the UI has to be
+reconstructed from the bookmarked state -- so a type supported only when passed
+directly is silently unavailable to bookmark-enabled apps.
+
+The page-level dependency set (requirejs, jQuery, Shiny) has one definition,
+`html_dependencies._page_deps()`. Both page forms -- the tag tree and the
+complete document -- splat it, so they cannot drift.
+
+### `Path` exception
+
+`Path` is the deliberate exception: it names a file to read once at startup
+rather than a UI value, so it is handled in `App.__init__()` and rejected in
+`_render_page()`. A UI function that wants to serve a file returns
+`ui.page_html(path)`, which makes the per-pageview file read explicit.
+
 ## Input/Output Bindings
 
 Client-server communication works through bindings:

CHANGELOG.md

@@ -17,6 +17,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 * Added `session.allow_reconnect()`, the Python counterpart to Shiny for R's `session$allowReconnect()`. Call it with `True` to let the browser reconnect to its session (showing a countdown dialog instead of the "Disconnected from server" overlay) when the hosting environment keeps sessions alive after a client disconnects, or with `"force"` to attempt the reconnect anywhere. (#2441)
 
+* Added `ui.page_html()`, for apps whose UI is a complete HTML document they own (e.g. the `index.html` a JS bundler emits) rather than one built from `ui.page_*()` components. It takes the document as a string or a `Path`, plus optional `extra_deps=`. Pass the result as `App(ui=)`, or return it from a UI function (`App(ui=lambda request: ...)`, which is what bookmarking requires): the document is served as-is, with Shiny's own HTML dependencies -- plus any in `extra_deps=` -- inserted at `<meta name="shiny-dependency-placeholder" content="">` (or a custom `deps_replace_pattern=`), and their files served by the app. In Express, use `ui.page_opts(html=)`, which routes the whole app through `ui.page_html()`: top-level UI markup is dropped (the document already is the page) but its HTML dependencies are kept. This is the Python counterpart to Shiny for R's `shinyApp(ui = htmlTemplate("index.html", document_ = TRUE))` with `attachDependencies()`. (#2462)
+
 ### Improvements
 
 * The README and the `shiny skills` CLI help now explain that [`library-skills`](https://library-skills.io) must be run from your own project directory, since it installs the bundled Agent Skills of the packages that project has installed. The previous wording left that precondition implicit, so running the command from an empty directory or from a clone of py-shiny silently installed nothing. (#2447)

docs/_quartodoc-core.yml

@@ -22,6 +22,7 @@ quartodoc:
         - ui.page_bootstrap
         - ui.page_auto
         - ui.page_output
+        - ui.page_html
     - title: UI Layouts
       desc: Control the layout of multiple UI components.
       contents:

docs/_quartodoc-express.yml

@@ -59,6 +59,7 @@ quartodoc:
       desc: Tools for creating, arranging, and styling UI components.
       contents:
         - express.ui.page_opts
+        - express.ui.page_html
         - express.ui.sidebar
         - express.ui.layout_columns
         - express.ui.layout_column_wrap

shiny/_app.py

@@ -31,6 +31,7 @@
 
 if TYPE_CHECKING:
     from htmltools import Tagified
+
 from starlette.requests import Request
 from starlette.responses import HTMLResponse, JSONResponse, Response
 from starlette.types import ASGIApp, Message, Receive, Scope, Send
@@ -48,10 +49,11 @@
     BookmarkSaveDirFn,
     BookmarkStore,
 )
-from .html_dependencies import jquery_deps, require_deps, shiny_deps
+from .html_dependencies import _page_deps
 from .http_staticfiles import FileResponse, StaticFiles
 from .session._session import AppSession, Inputs, Outputs, Session, session_context
 from .types import MISSING, MISSING_TYPE
+from .ui._page import DEPS_PLACEHOLDER, PageHtmlDocument, page_html
 
 T = TypeVar("T")
 
@@ -75,7 +77,19 @@ class App:
         similar, with layouts and controls nested inside). You can
         also pass a function that takes a :class:`~starlette.requests.Request` and
         returns a UI definition, if you need the UI definition to be created dynamically
-        for each pageview.
+        for each pageview -- which is also what bookmarking requires. Finally, it can
+        be a complete HTML document that you own: either a :class:`~pathlib.Path` to an
+        HTML file, or a call to :func:`~shiny.ui.page_html`, which additionally lets
+        you attach your own :class:`~htmltools.HTMLDependency` objects. Such a document
+        is served as-is, and must contain
+        ``<meta name="shiny-dependency-placeholder" content="">`` (or the
+        ``deps_replace_pattern=`` passed to ``ui.page_html()``) to
+        mark where Shiny's HTML dependencies are inserted.
+
+        A ``Tag``, ``TagList``, or ``ui.page_html()`` result may equally be *returned
+        by* the function above, for a UI that varies per pageview. A ``Path`` may not:
+        it names a file to read once at startup, so a function that wants to serve a
+        file should return ``ui.page_html(path)``, which reads it per call.
     server
         A function which is called once for each session, ensuring that each session is
         independent.
@@ -140,7 +154,7 @@ def server(input: Inputs, output: Outputs, session: Session):
     ``SafeException`` messages bypass sanitization regardless of this setting.
     """
 
-    ui: RenderedHTML | Callable[[Request], Tag | TagList]
+    ui: RenderedHTML | Callable[[Request], Tag | TagList | PageHtmlDocument]
     server: Callable[[Inputs, Outputs, Session], None]
 
     _bookmark_save_dir_fn: BookmarkSaveDirFn | None | MISSING_TYPE
@@ -153,8 +167,9 @@ def __init__(
             Tag
             | TagList
             | Tagified
-            | Callable[[Request], Tag | TagList | Tagified]
+            | Callable[[Request], Tag | TagList | Tagified | PageHtmlDocument]
             | Path
+            | PageHtmlDocument
         ),
         server: (
             Callable[[Inputs], None] | Callable[[Inputs, Outputs, Session], None] | None
@@ -243,17 +258,20 @@ def __init__(
             if is_async_callable(cast(Callable[[Request], Any], ui)):
                 raise TypeError("App UI cannot be a coroutine function")
             # Dynamic UI: just store the function for later
-            self.ui = cast("Callable[[Request], Tag | TagList]", ui)
+            self.ui = cast("Callable[[Request], Tag | TagList | PageHtmlDocument]", ui)
         elif isinstance(ui, Path):
             if not ui.is_absolute():
                 raise ValueError("Path to UI must be absolute")
 
-            self.ui = self._render_page_from_file(ui, lib_prefix=self.lib_prefix)
+            # Read once, here: a `Path` names a file to serve, not a per-pageview UI
+            # value, so it is not something a UI function may return.
+            self.ui = self._render_page(page_html(ui), lib_prefix=self.lib_prefix)
 
         else:
             # Static UI: render the UI now and save the results
             self.ui = self._render_page(
-                cast("Tag | TagList", ui), lib_prefix=self.lib_prefix
+                cast("Tag | TagList | PageHtmlDocument", ui),
+                lib_prefix=self.lib_prefix,
             )
 
     def init_starlette_app(self) -> starlette.applications.Starlette:
@@ -499,38 +517,74 @@ def _register_web_dependency(self, dep: HTMLDependency) -> None:
 
         self._registered_dependencies[dep_name] = dep
 
-    def _render_page(self, ui: Tag | TagList, lib_prefix: str) -> RenderedHTML:
-        # Use presence of the Bootstrap dependency as a signal that the UI uses a
-        # shiny.ui.page_*() function, in which case the Shiny CSS is already included.
-        has_bootstrap = any(dep.name == "bootstrap" for dep in ui.get_dependencies())
-        # Make sure requirejs, jQuery, and Shiny come before any other dependencies.
-        # (see require_deps() for a comment about why we even include it)
-        # Compose a new TagList so this works for any UI input shape, including
-        # pre-tagified (and immutable) TagifiedTag/TagifiedTagList values that
-        # express mode produces (`run_express(...).tagify()` in `express/_run.py`).
-        ui_res = TagList(
-            require_deps(),
-            jquery_deps(),
-            *shiny_deps(include_css=not has_bootstrap),
-            ui,
-        )
-        rendered = HTMLDocument(ui_res).render(lib_prefix=lib_prefix)
-        self._ensure_web_dependencies(rendered["dependencies"])
-        return rendered
+    def _render_page(
+        self,
+        ui: Tag | TagList | PageHtmlDocument,
+        lib_prefix: str,
+    ) -> RenderedHTML:
+        # Every UI *value* type must be handled here, and nowhere else. This is the one
+        # place both `App(ui=)` and a UI function's return value are rendered, so
+        # handling a type here is what makes it work in both positions. Adding a UI
+        # type without doing so silently supports it in only one of them.
+        #
+        # `Path` is the exception, and is handled in `__init__()`: it names a file to
+        # read once at startup, not a UI value, so a UI function may not return one.
+        if isinstance(ui, Path):
+            raise TypeError(
+                "A UI function cannot return a `Path`. Return `ui.page_html(path)`"
+                " instead, which makes it clear the file is read on every call."
+            )
+
+        if isinstance(ui, HTMLDocument):
+            # Not supported: `HTMLDocument` builds a document out of a tag tree and
+            # hoists the dependencies it finds there into <head> in tree order, so
+            # Shiny's could only be appended after the app author's -- jQuery would
+            # load after the scripts that need it. Pass the tags themselves instead
+            # and let Shiny build the document.
+            raise TypeError(
+                "An `HTMLDocument` cannot be used as a UI. Pass its contents (a `Tag`"
+                " or `TagList`) instead, and Shiny will build the document -- or, for"
+                " a complete HTML document of your own, use `ui.page_html()`."
+            )
 
-    def _render_page_from_file(self, file: Path, lib_prefix: str) -> RenderedHTML:
-        with open(file, "r") as f:
-            page_html = f.read()
+        if isinstance(ui, HTMLTextDocument):
+            if not isinstance(ui, PageHtmlDocument):
+                raise TypeError(
+                    "A complete HTML document used as a UI must be created with"
+                    " `ui.page_html()`, which is what prefixes Shiny's own"
+                    " HTML dependencies onto the app author's."
+                )
 
-        doc = HTMLTextDocument(
-            page_html,
-            deps=[require_deps(), jquery_deps(), *shiny_deps(include_css=True)],
-            deps_replace_pattern='<meta name="shiny-dependency-placeholder" content="">',
-        )
+            # Render the document as-is: wrapping it in an HTMLDocument would nest
+            # <html> inside <html>.
+            rendered = ui.render(lib_prefix=lib_prefix)
 
-        rendered = doc.render(lib_prefix=lib_prefix)
-        self._ensure_web_dependencies(rendered["dependencies"])
+            # `render()` replaces `deps_replace_pattern` with the dependency markup,
+            # or silently does nothing if the pattern isn't in the document. The
+            # document always has dependencies, so a missing manifest means the
+            # pattern wasn't found -- i.e. we'd serve a page with no Shiny on it.
+            if "application/html-dependencies" not in rendered["html"]:
+                raise ValueError(
+                    "The UI document does not contain the string that marks where"
+                    " Shiny's HTML dependencies are inserted, so they could not be"
+                    " inserted. Add the `deps_replace_pattern=` passed to"
+                    " `ui.page_html()` (by default,"
+                    f" `{DEPS_PLACEHOLDER}`) to the document."
+                )
+        else:
+            # Use presence of the Bootstrap dependency as a signal that the UI uses a
+            # shiny.ui.page_*() function, in which case the Shiny CSS is already
+            # included.
+            has_bootstrap = any(
+                dep.name == "bootstrap" for dep in ui.get_dependencies()
+            )
+            # Compose a new TagList so this works for any UI input shape, including
+            # pre-tagified (and immutable) TagifiedTag/TagifiedTagList values that
+            # express mode produces (`run_express(...).tagify()` in `express/_run.py`).
+            ui_res = TagList(*_page_deps(include_css=not has_bootstrap), ui)
+            rendered = HTMLDocument(ui_res).render(lib_prefix=lib_prefix)
 
+        self._ensure_web_dependencies(rendered["dependencies"])
         return rendered
 
     # ==========================================================================
@@ -559,7 +613,14 @@ def set_bookmark_restore_dir_fn(self, bookmark_restore_dir_fn: BookmarkDirFn):
 
 
 def is_uifunc(
-    x: Path | Tag | TagList | Tagified | Callable[[Request], Tag | TagList | Tagified],
+    x: (
+        Path
+        | Tag
+        | TagList
+        | Tagified
+        | Callable[[Request], Tag | TagList | Tagified | PageHtmlDocument]
+        | PageHtmlDocument
+    ),
 ) -> bool:
     if (
         isinstance(x, Path)

shiny/api-examples/page_html/app-core.py

@@ -0,0 +1,14 @@
+from pathlib import Path
+
+from shiny import App, Inputs, Outputs, Session, render, ui
+
+app_ui = ui.page_html(Path(__file__).parent / "index.html")
+
+
+def server(input: Inputs, output: Outputs, session: Session):
+    @render.text
+    def greeting():
+        return "Hello from the server!"
+
+
+app = App(app_ui, server)

shiny/api-examples/page_html/app-express.py

@@ -0,0 +1,10 @@
+from pathlib import Path
+
+from shiny.express import render, ui
+
+ui.page_opts(html=Path(__file__).parent / "index.html")
+
+
+@render.text
+def greeting():
+    return "Hello from the server!"

shiny/api-examples/page_html/index.html

@@ -0,0 +1,12 @@
+<!DOCTYPE html>
+<html>
+  <head>
+    <title>A page Shiny did not build</title>
+    <!-- A meta tag that is replaced with shiny's dependencies (and any provided `extra_deps=`) -->
+    <meta name="shiny-dependency-placeholder" content="">
+  </head>
+  <body>
+    <h1>A complete HTML document, served as-is</h1>
+    <div id="greeting" class="shiny-text-output"></div>
+  </body>
+</html>

shiny/express/_run.py

@@ -19,6 +19,7 @@
 from ..bookmark._types import BookmarkStore
 from ..session import Inputs, Outputs, Session, get_current_session, session_context
 from ..types import MISSING, MISSING_TYPE
+from ..ui._page import PageHtmlDocument
 from ._is_express import find_magic_comment_mode
 from ._recall_context import RecallContextManager
 from ._stub_session import ExpressStubSession
@@ -131,7 +132,9 @@ def create_express_app(file: Path, package_name: str) -> App:
             # catch them here and convert them to a different type of error, because uvicorn
             # specifically catches AttributeErrors and prints an error message that is
             # misleading for Shiny Express. https://github.com/posit-dev/py-shiny/issues/937
-            app_ui = run_express(file, package_name).tagify()
+            ui_res = run_express(file, package_name)
+            # A `PageHtmlDocument` (from `ui.page_opts(html=)`) has no `.tagify()`.
+            app_ui = ui_res if isinstance(ui_res, PageHtmlDocument) else ui_res.tagify()
 
     except AttributeError as e:
         raise RuntimeError(e) from e
@@ -143,7 +146,10 @@ def create_express_app(file: Path, package_name: str) -> App:
         def app_ui_wrapper(request: Request):
             # Stub session used to pass `app_opts()` checks.
             with session_context(ExpressStubSession()):
-                return run_express(file, package_name).tagify()
+                wrapper_ui_res = run_express(file, package_name)
+                if isinstance(wrapper_ui_res, PageHtmlDocument):
+                    return wrapper_ui_res
+                return wrapper_ui_res.tagify()
 
         app_ui = app_ui_wrapper
 
@@ -176,7 +182,10 @@ def express_server(input: Inputs, output: Outputs, session: Session):
     return app
 
 
-def run_express(file: Path, package_name: str | None = None) -> Tag | TagList:
+def run_express(
+    file: Path,
+    package_name: str | None = None,
+) -> Tag | TagList | PageHtmlDocument:
     """
     Run the code in a Shiny Express app file and return the UI. This is to be run in
     both the UI-rendering phase and the server-rendering phase of a Shiny Express app.
@@ -198,7 +207,7 @@ def run_express(file: Path, package_name: str | None = None) -> Tag | TagList:
     tree = DisplayFuncsTransformer().visit(tree)
     tree = ast.fix_missing_locations(tree)
 
-    ui_result: Tag | TagList = TagList()
+    ui_result: Tag | TagList | PageHtmlDocument = TagList()
 
     def set_result(x: object):
         nonlocal ui_result
@@ -271,7 +280,9 @@ def set_result(x: object):
         sys.displayhook = prev_displayhook
 
 
-_top_level_recall_context_manager: RecallContextManager[Tag] | None = None
+_top_level_recall_context_manager: (
+    RecallContextManager[Tag | PageHtmlDocument] | None
+) = None
 
 
 def reset_top_level_recall_context_manager() -> None:
@@ -281,7 +292,9 @@ def reset_top_level_recall_context_manager() -> None:
     _top_level_recall_context_manager = page_auto_cm()
 
 
-def get_top_level_recall_context_manager() -> RecallContextManager[Tag]:
+def get_top_level_recall_context_manager() -> (
+    RecallContextManager[Tag | PageHtmlDocument]
+):
     if _top_level_recall_context_manager is None:
         raise RuntimeError("No top-level recall context manager has been set.")
 

shiny/express/ui/__init__.py

@@ -50,6 +50,7 @@
     hover_opts,
     include_css,
     include_js,
+    page_html,
     hide_offcanvas,
     input_action_button,
     input_action_link,
@@ -246,6 +247,7 @@
     "bind_task_button",
     "input_task_button",
     "input_text",
+    "page_html",
     "panel_title",
     "input_text_area",
     "insert_accordion_panel",