graph LR
bonobo_config_configurables["bonobo.config.configurables"]
bonobo_config_options["bonobo.config.options"]
bonobo_config_processors["bonobo.config.processors"]
bonobo_config_services["bonobo.config.services"]
bonobo_config_configurables_PartiallyConfigured["bonobo.config.configurables.PartiallyConfigured"]
bonobo_config_processors_ContextProcessor["bonobo.config.processors.ContextProcessor"]
bonobo_config_services_get_lock["bonobo.config.services.get_lock"]
bonobo_config_configurables -- "retrieves options from" --> bonobo_config_options
bonobo_config_configurables -- "creates instances of" --> bonobo_config_configurables_PartiallyConfigured
bonobo_config_options -- "provides default values to" --> bonobo_config_configurables
bonobo_config_processors -- "leverages for managing execution context" --> bonobo_config_processors_ContextProcessor
bonobo_config_services -- "uses for concurrency control" --> bonobo_config_services_get_lock
bonobo_config_configurables_PartiallyConfigured -- "manages configuration for" --> bonobo_config_configurables
bonobo_config_processors_ContextProcessor -- "modifies behavior of" --> bonobo_config_processors
bonobo_config_services_get_lock -- "ensures thread safety for" --> bonobo_config_services
The bonobo.config subsystem is central to defining and managing the configurable aspects, execution context, and external dependencies within Bonobo ETL graphs. It provides a structured approach to making ETL nodes reusable and adaptable. bonobo.config.configurables allows nodes to declare their configuration needs, leveraging bonobo.config.options to define individual settings. The subsystem also manages dynamic behavior through bonobo.config.processors, which includes bonobo.config.processors.ContextProcessor for execution context management. Furthermore, bonobo.config.services handles the injection and lifecycle of external dependencies, utilizing mechanisms like bonobo.config.services.get_lock for concurrency control. bonobo.config.configurables.PartiallyConfigured facilitates flexible configuration by allowing incremental definition of node settings.
Manages the declaration and application of configurable options for Bonobo nodes (extractors, transformers, loaders). It enables nodes to expose their configuration needs, facilitating dynamic customization and reusability across different ETL scenarios.
Related Classes/Methods:
Represents individual configurable options, managing their default values, retrieval mechanisms, and validation rules. It ensures that configuration values are consistent and reliable.
Related Classes/Methods:
Manages the lifecycle and application of "processors," which are functions or objects that can modify node behavior based on configuration or execution context. This enables dynamic adjustments during ETL execution.
Related Classes/Methods:
Provides a robust mechanism for defining, validating, resolving, and injecting external dependencies (services) into Bonobo nodes. It also handles concurrency control for shared services.
Related Classes/Methods:
Represents instances of nodes or components where some, but not all, configuration has been defined. It supports deferred or staged configuration, enabling incremental or placeholder-based ETL graph definitions. Its implementation as a _functools.partial subclass allows it to simulate option values from its keywords and args.
Related Classes/Methods:
A specialized type of processor that manages the execution context for ETL processes. It functions as a yielding context manager, allowing setup and teardown of runtime dependencies, enabling nodes to adapt their behavior based on the current execution environment.
Related Classes/Methods:
Provides a locking mechanism for concurrency control when accessing shared resources or services within ETL pipelines, ensuring thread safety by managing threading.RLock instances.
Related Classes/Methods: