graph LR
Client_Application["Client Application"]
Pool_Manager["Pool Manager"]
Inter_Process_Task_Queue["Inter-Process Task Queue"]
Inter_Process_Result_Queue["Inter-Process Result Queue"]
Worker_Process["Worker Process"]
Worker_s_Asyncio_Event_Loop["Worker's Asyncio Event Loop"]
Process_Context_Manager["Process Context Manager"]
Client_Application -- "submits tasks to" --> Pool_Manager
Pool_Manager -- "returns results to" --> Client_Application
Client_Application -- "configures context via" --> Process_Context_Manager
Pool_Manager -- "queues work into" --> Inter_Process_Task_Queue
Pool_Manager -- "spawns and manages" --> Worker_Process
Pool_Manager -- "uses" --> Process_Context_Manager
Inter_Process_Result_Queue -- "provides results to" --> Pool_Manager
Inter_Process_Task_Queue -- "distributes tasks to" --> Worker_Process
Worker_Process -- "fetches tasks from" --> Inter_Process_Task_Queue
Worker_Process -- "executes via" --> Worker_s_Asyncio_Event_Loop
Worker_Process -- "sends results to" --> Inter_Process_Result_Queue
click Pool_Manager href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/aiomultiprocess/Pool_Manager.md" "Details"
click Worker_Process href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/aiomultiprocess/Worker_Process.md" "Details"
The aiomultiprocess library facilitates asynchronous multiprocessing by orchestrating tasks between a Client Application and a pool of Worker Processes. The Client Application initiates parallel tasks by submitting them to the Pool Manager, which acts as the central coordinator. The Pool Manager leverages the Process Context Manager to configure and manage the lifecycle of Worker Processes. Tasks are dispatched from the Pool Manager to Worker Processes via the Inter-Process Task Queue. Each Worker Process fetches tasks from this queue and executes them asynchronously, utilizing its internal Worker's Asyncio Event Loop for concurrent I/O operations. Upon completion, results are sent back from the Worker Processes to the Pool Manager through the Inter-Process Result Queue, which then returns the aggregated results to the Client Application. This architecture ensures efficient, non-blocking parallel execution of tasks.
The user's code that defines and submits parallel processing tasks, and consumes their results. It serves as the entry point for interacting with the aiomultiprocess pool. This component represents external user-defined code and does not have a specific source reference within the aiomultiprocess library.
Related Classes/Methods: None
Pool Manager [Expand]
The central orchestrator of the worker pool, running in the main process. It manages the lifecycle of worker processes, dispatches tasks, and aggregates results.
Related Classes/Methods:
A dedicated communication channel (implemented using multiprocessing.Queue) responsible for holding tasks submitted by the Pool Manager, awaiting consumption by Worker Processes. This component utilizes a standard Python library class, hence no specific file reference within the project.
Related Classes/Methods: None
A dedicated communication channel (implemented using multiprocessing.Queue) for Worker Processes to send their completed task results and exceptions back to the Pool Manager. This component utilizes a standard Python library class, hence no specific file reference within the project.
Related Classes/Methods: None
Worker Process [Expand]
An independent Python process spawned by the Pool Manager, designed to execute tasks. Each worker maintains its own asyncio event loop for internal concurrency.
Related Classes/Methods:
The asynchronous event loop running within each Worker Process. This enables a single worker process to concurrently handle multiple I/O-bound coroutines, maximizing efficiency. This component represents the standard asyncio module, hence no specific file reference within the project.
Related Classes/Methods: None
The underlying mechanism for configuring the multiprocessing start method (e.g., 'fork', 'spawn', 'forkserver') and managing the process creation context.
Related Classes/Methods: