Open Reality: phone video to a splat scene your AI assistant can measure (uses gsplat)

[bdavidzhang]

Hi Nerfstudio folks. Sharing a project that uses gsplat, with credit where it is due.

Open Reality (https://github.com/reality-opened/openreality, BSD-2-Clause) turns a phone walkthrough video into a persistent 3D scene that an AI assistant can query over MCP (the tool protocol Claude, Codex, and Cursor speak). Reconstruction is VGGT-SLAM based (MIT SPARK lineage); the splat export uses gsplat for the optional GPU refinement pass, and it is listed in the Acknowledgements section of the README.

What the assistant can do with a scene once it exists:

• measure distances and angles (the tools report relative units until the user calibrates with one known real-world distance, and only then say metres; the reasoning is written up here: https://github.com/reality-opened/openreality/blob/main/docs/posts/relative-units-are-not-metres.md)
• find floor and wall planes, plan a path through the free space
• list the detected objects (a closed world, so the model cannot invent furniture)
• export the scene as robot-training data (LeRobot / GR00T style datasets, Isaac Sim USD, the latter refused without a metric anchor)
• pull the splat (.ply / .spz LOD levels) and point cloud to disk

The whole server is self-hostable on your own GPU (the SLAM library and the broker are in the same repo), and there is an offline simulator so you can try the tool surface with no account and no GPU:

claude mcp add openreality -- npx -y openreality-mcp serve

A 60-second terminal demo is at the top of the README. If anyone here wants to try it on a scan of their own, I am happy to help in the thread. Thank you for gsplat; it is a pleasure to build on.