Skip to content

Execution_Layer.md

Latest commit

 

History

History
112 lines (66 loc) · 6.09 KB

Execution_Layer.md

File metadata and controls

112 lines (66 loc) · 6.09 KB
graph LR
    Execution_Layer["Execution Layer"]
    Executor_Base["Executor Base"]
    Local_Executor["Local Executor"]
    Dry_Run_Executor["Dry Run Executor"]
    Deployment_Management["Deployment Management"]
    Conda_Environment_Manager["Conda Environment Manager"]
    Container_Manager["Container Manager"]
    Script_Execution["Script Execution"]
    Execution_Layer -- "delegates job execution to" --> Local_Executor
    Execution_Layer -- "delegates job execution to" --> Dry_Run_Executor
    Execution_Layer -- "relies on" --> Deployment_Management
    Executor_Base -- "is inherited by" --> Local_Executor
    Executor_Base -- "is inherited by" --> Dry_Run_Executor
    Executor_Base -- "defines the interface for" --> Script_Execution
    Local_Executor -- "invokes" --> Script_Execution
    Dry_Run_Executor -- "simulates interactions with" --> Script_Execution
    Dry_Run_Executor -- "simulates interactions with" --> Deployment_Management
    Deployment_Management -- "is utilized by" --> Local_Executor
    Deployment_Management -- "is utilized by" --> Dry_Run_Executor
    Conda_Environment_Manager -- "is a part of" --> Deployment_Management
    Container_Manager -- "is a part of" --> Deployment_Management
    Script_Execution -- "is invoked by" --> Local_Executor
    Script_Execution -- "is invoked by" --> Dry_Run_Executor
    Script_Execution -- "relies on the environment prepared by" --> Deployment_Management
    click Execution_Layer href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/snakemake/Execution_Layer.md" "Details"
Loading

CodeBoardingDemoContact

Details

The Execution Layer in Snakemake is a critical subsystem responsible for the actual running of workflow jobs and managing their required software environments. It embodies the Strategy Pattern, allowing Snakemake to adapt to various execution backends (local, dry run, cluster, cloud) and ensures reproducibility through robust environment management.

Execution Layer [Expand]

This is the top-level conceptual component that orchestrates the execution of jobs. It acts as a facade, providing a unified interface for the core workflow engine to initiate job execution, abstracting away the complexities of different execution strategies and environment provisioning.

Related Classes/Methods:

  • snakemake.executors.Executor

Executor Base

This abstract base class defines the common interface and core logic for all job executors. It establishes the contract for how jobs are handled, including methods for submitting, monitoring, and finalizing jobs. This is where the Strategy Pattern is implemented, with concrete subclasses providing specific execution strategies.

Related Classes/Methods:

  • snakemake.executors.Executor

Local Executor

A concrete implementation of the Executor base class, responsible for executing jobs directly on the local machine where Snakemake is running. It handles the spawning of local processes and managing their lifecycle.

Related Classes/Methods:

  • snakemake.executors.LocalExecutor

Dry Run Executor

Another concrete Executor implementation that simulates the execution of a workflow without actually running any commands or creating output files. It's crucial for validating workflow logic, dependency resolution, and identifying potential issues before actual execution.

Related Classes/Methods:

  • snakemake.executors.DryrunExecutor

Deployment Management

This component is responsible for provisioning and managing the software environments (e.g., Conda, Singularity/Docker containers) required for each job. It ensures that jobs run in isolated and reproducible environments, handling environment creation, activation, and caching.

Related Classes/Methods:

Conda Environment Manager

A specialized part of Deployment Management that specifically handles Conda environments. It manages the creation, activation, and caching of Conda environments based on the conda directive in Snakemake rules, ensuring that the correct software dependencies are available for each job.

Related Classes/Methods:

Container Manager

A specialized part of Deployment Management that handles the use of Singularity (and implicitly Docker) containers for job execution. It manages pulling, building, and running jobs within isolated container environments, further enhancing reproducibility and portability across different systems.

Related Classes/Methods:

Script Execution

This component, along with its subclasses (e.g., BashScript, PythonScript, RScript), is responsible for executing the actual user-defined scripts or shell commands specified within a Snakemake rule. It provides the necessary environment and context for these scripts to run correctly within the job's allocated resources and prepared environment.

Related Classes/Methods:

  • snakemake.script.ScriptBase

FAQ