graph LR
Concurrency_Decorator["Concurrency Decorator"]
AST_Instrumentation_Analysis["AST Instrumentation & Analysis"]
Concurrent_Task_Execution_Engine["Concurrent Task Execution Engine"]
Asynchronous_Result_Management["Asynchronous Result Management"]
Inter_Process_Data_Proxy_Serialization["Inter-Process Data Proxy & Serialization"]
Concurrency_Decorator -- "Initiates analysis" --> AST_Instrumentation_Analysis
Concurrency_Decorator -- "Triggers runtime execution" --> Concurrent_Task_Execution_Engine
AST_Instrumentation_Analysis -- "Provides task definitions" --> Concurrent_Task_Execution_Engine
AST_Instrumentation_Analysis -- "Informs about synchronization requirements" --> Asynchronous_Result_Management
Concurrent_Task_Execution_Engine -- "Produces raw results" --> Asynchronous_Result_Management
Concurrent_Task_Execution_Engine -- "Exchanges proxied data" --> Inter_Process_Data_Proxy_Serialization
Inter_Process_Data_Proxy_Serialization -- "Prepares results for management" --> Asynchronous_Result_Management
Asynchronous_Result_Management -- "Returns final results" --> Concurrency_Decorator
click Concurrency_Decorator href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/deco/Concurrency_Decorator.md" "Details"
click AST_Instrumentation_Analysis href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/deco/AST_Instrumentation_Analysis.md" "Details"
click Concurrent_Task_Execution_Engine href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/deco/Concurrent_Task_Execution_Engine.md" "Details"
click Inter_Process_Data_Proxy_Serialization href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/deco/Inter_Process_Data_Proxy_Serialization.md" "Details"
The deco project implements an automatic parallelization framework centered around a Concurrency Decorator. This decorator orchestrates the entire process, starting with AST Instrumentation & Analysis to statically identify parallelizable operations and synchronization points within decorated functions. The insights from this analysis guide the Concurrent Task Execution Engine, which manages the actual multiprocessing. Inter-Process Data Proxy & Serialization ensures efficient data exchange between the main and worker processes. Asynchronous Result Management then handles the collection, processing, and implicit synchronization of results, leveraging the synchronization requirements identified during AST analysis, before returning the final outcomes to the Concurrency Decorator. This architecture ensures a seamless transition from decorated function definition to parallel execution with automated result handling and synchronization.
Concurrency Decorator [Expand]
The primary user-facing interface, acting as a facade via the @concurrent decorator to simplify parallel execution. It's the entry point for initiating concurrent operations.
Related Classes/Methods:
AST Instrumentation & Analysis [Expand]
Statically analyzes the AST of decorated functions to identify concurrent calls, implicit synchronization points, and data dependencies, crucial for automatic parallelization.
Related Classes/Methods:
deco.astutil.visit_FunctionDef:154-158deco.astutil.get_waits:64-65deco.astutil.is_concurrent_call:54-55deco.astutil.generic_visit:73-76
Concurrent Task Execution Engine [Expand]
The core component managing the submission and lifecycle of tasks to a multiprocessing pool, leveraging Python's multiprocessing.pool for parallel execution.
Related Classes/Methods:
Handles asynchronous results from concurrent operations, providing mechanisms for retrieval and implicit synchronization by blocking until results are available, ensuring data consistency.
Related Classes/Methods:
deco.conc.ConcurrentResult:154-162deco.conc.result:161-162deco.conc.wait:136-150deco.conc.get:158-159
Inter-Process Data Proxy & Serialization [Expand]
Facilitates seamless and efficient data transfer (arguments and results) between the main process and worker processes using proxy objects and managing serialization/deserialization.
Related Classes/Methods: