fix: use shared memory for virtual files when running with app isolation #9181
Merged
fix: use shared memory for virtual files when running with app isolation #9181
Conversation
This change updates isolated apps to use shared memory for virtual file storage. Isolated apps run in different processes from the main server. This means they need to place virtual file data in shared memory, but historically run-mode has used run in-process and used in-memory storage. The change looks large because we're lifting the choice of storage out of the kernel context, letting the caller decide what storage backend to use. This required touching many files.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
akshayka
commented
Apr 14, 2026
Comment on lines
-161
to
170
| # Use shared memory in edit mode, | ||
| # in-memory storage in run mode (same process) | ||
| storage = ( | ||
| # Storage is chosen explicitly by the caller. None means virtual files | ||
| # are not supported; we still construct an (inert) InMemoryStorage so | ||
| # the registry has a backend, but ctx.virtual_files_supported is False. | ||
| storage: VirtualFileStorage = ( | ||
| SharedMemoryStorage() | ||
| if mode == SessionMode.EDIT | ||
| if virtual_file_storage == "shared_memory" | ||
| else InMemoryStorage() | ||
| ) |
Contributor
Author
There was a problem hiding this comment.
This is the main change in this PR.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates marimo’s virtual file subsystem to work correctly when apps are run in isolated subprocesses by making the virtual-file storage backend an explicit choice (shared memory vs in-process memory vs disabled).
Changes:
- Replace the boolean
virtual_files_supportedflag withvirtual_file_storage: "in_memory" | "shared_memory" | Noneand thread it through session/kernel/app-host/runtime creation paths. - Configure AppHost (isolated run-mode) kernels to use shared-memory virtual-file storage so the parent server can serve
/@file/...requests. - Add a process-isolation smoke test for virtual files and update docs/tests to use the new parameter.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/conftest.py | Updates kernel/test config to pass virtual_file_storage based on session mode. |
| tests/_session/app_host/test_main.py | Switches app-host tests to pass virtual_file_storage="shared_memory". |
| tests/_session/app_host/test_app_host.py | Switches app-host tests to pass virtual_file_storage="shared_memory". |
| tests/_server/test_sessions.py | Updates session/kernel-manager tests to new virtual_file_storage arg. |
| tests/_runtime/test_runtime.py | Updates runtime tests to use virtual_file_storage. |
| marimo/_smoke_tests/process_isolation/virtual_files.py | Adds a smoke-test notebook for virtual files under process isolation. |
| marimo/_smoke_tests/process_isolation/README.md | Documents how to run the process-isolation virtual-file smoke test. |
| marimo/_session/session.py | Threads virtual_file_storage into the “original kernel” session creation path. |
| marimo/_session/managers/kernel.py | KernelManager now stores and passes virtual_file_storage into runtime.launch_kernel. |
| marimo/_session/managers/ipc.py | Removes virtual_files_supported from IPC manager args (IPC kernels always disable virtual files). |
| marimo/_session/managers/app_host.py | Forces AppHost kernels to use virtual_file_storage="shared_memory". |
| marimo/_session/app_host/main.py | App-host kernel launch now forwards virtual_file_storage. |
| marimo/_session/app_host/host.py | AppHost create_kernel now accepts virtual_file_storage. |
| marimo/_session/app_host/commands.py | Updates CreateKernelCmd to include virtual_file_storage. |
| marimo/_server/session_manager.py | Chooses default virtual_file_storage based on session mode (edit vs run). |
| marimo/_server/export/init.py | Export path disables virtual files via virtual_file_storage=None. |
| marimo/_runtime/virtual_file/storage.py | Introduces VirtualFileStorageType and keeps storage backends. |
| marimo/_runtime/virtual_file/init.py | Re-exports VirtualFileStorageType. |
| marimo/_runtime/runtime.py | Updates launch_kernel signature to accept virtual_file_storage. |
| marimo/_runtime/context/kernel_context.py | Selects virtual-file backend based on virtual_file_storage and sets virtual_files_supported. |
| marimo/_runtime/app/kernel_runner.py | Uses shared-memory storage for embedded/edit-like runner context. |
| marimo/_pyodide/pyodide_session.py | Disables virtual files via virtual_file_storage=None. |
| marimo/_ipc/types.py | Removes virtual_files_supported from KernelArgs. |
| marimo/_ipc/launch_kernel.py | IPC kernel launch disables virtual files via virtual_file_storage=None. |
Thanks copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
akshayka
commented
Apr 14, 2026
| # Whether to use run-mode config (autorun) vs edit-mode config (lazy) | ||
| is_run_mode: bool = False | ||
| # Runtime behavior flags | ||
| virtual_files_supported: bool = True |
Contributor
Author
There was a problem hiding this comment.
@manzt I can keep this argument and just ignore it to prevent a breaking change, lmk. It looks like it was ignored before
Collaborator
There was a problem hiding this comment.
We extend the KernelManager impl in the VS Code extension, so we probably just need to bump the minimum marimo version in the next release.
https://github.com/marimo-team/marimo/actions/runs/24527579413
dmadisetti
previously approved these changes
Apr 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This change updates isolated apps to use shared memory for virtual
file storage.
Isolated apps run in different processes from the main server. This
means they need to place virtual file data in shared memory, but
historically run-mode has run in-process and used in-memory
storage.
The change looks large because we're lifting the choice of storage
out of the kernel context, letting the caller decide what storage
backend to use. This required touching many files. In practice the change is small.
This fixes the issue identified in the following comment: #9127 (comment)