graph LR
Simulation_Core_Engine["Simulation Core Engine"]
Environment_Orchestrator["Environment Orchestrator"]
Observation_Generator["Observation Generator"]
Task_Manager["Task Manager"]
Camera_Manager["Camera Manager"]
Demonstration_Management["Demonstration Management"]
Environment_Orchestrator -- "sends commands to" --> Simulation_Core_Engine
Simulation_Core_Engine -- "provides data to" --> Observation_Generator
Environment_Orchestrator -- "invokes" --> Task_Manager
Environment_Orchestrator -- "invokes" --> Camera_Manager
Observation_Generator -- "requests data from" --> Simulation_Core_Engine
Task_Manager -- "manipulates objects via" --> Simulation_Core_Engine
Camera_Manager -- "configures" --> Simulation_Core_Engine
Demonstration_Management -- "orchestrates" --> Environment_Orchestrator
Demonstration_Management -- "utilizes" --> Observation_Generator
Demonstration_Management -- "interacts with" --> Task_Manager
The RLBench simulation environment is orchestrated by the Environment Orchestrator, which manages the overall lifecycle, including scene setup and episode advancement. It delegates specific responsibilities such as task object placement to the Task Manager and camera configuration to the Camera Manager. The Simulation Core Engine serves as the direct interface to the underlying CoppeliaSim/PyRep backend, executing commands from the Environment Orchestrator and Task Manager, and producing raw simulation data based on configurations set by the Camera Manager. The Observation Generator is responsible for retrieving and processing this raw data from the Simulation Core Engine to form structured observations for the agent. Finally, Demonstration Management oversees the entire process for recording or replaying expert trajectories, coordinating with the Environment Orchestrator, Observation Generator, and Task Manager.
The lowest-level interface to the CoppeliaSim/PyRep backend. It directly handles physics updates, rendering, robot joint control, and scene graph manipulation. It is the direct executor of simulation commands.
Related Classes/Methods:
Manages the overall lifecycle of the simulation environment. This includes initial setup of the scene, preparing for new learning episodes (e.g., resetting the environment, initializing tasks), and advancing the simulation by single time steps.
Related Classes/Methods:
rlbench.backend.scene.Scene:__init__rlbench.backend.scene.Scene:init_episoderlbench.backend.scene.Scene:step
Responsible for gathering and processing all types of sensory data from the simulation, such as RGB images, depth maps, semantic masks, and miscellaneous robot/scene state data. It aggregates raw simulation outputs into structured observations for the agent.
Related Classes/Methods:
rlbench.backend.scene.Scene:get_observationrlbench.backend.scene.Scene:retrieve_rgb_depthrlbench.backend.scene.Scene:retrieve_misc_datarlbench.backend.scene.Scene:retrieve_maskrlbench.backend.scene.Scene:retrieve_camera_data
Handles the initialization and placement of task-specific objects within the virtual scene. It ensures that the environment is correctly configured for a given learning task at the start of each episode.
Related Classes/Methods:
Manages the configuration and properties of virtual cameras within the simulation. This includes setting camera positions, orientations, and specific rendering properties (e.g., for RGB, depth, or mask generation).
Related Classes/Methods:
rlbench.backend.scene.Scene:set_camera_propertiesrlbench.backend.scene.Scene:set_rgb_propertiesrlbench.backend.scene.Scene:set_mask_properties
Facilitates the recording and handling of expert demonstrations. It orchestrates the simulation steps, observation capture, and state logging required to generate or replay pre-recorded trajectories.
Related Classes/Methods: