graph LR
I_O_Nodes["I/O Nodes"]
Flow_Control_Nodes["Flow Control Nodes"]
Basic_Utility_Nodes["Basic Utility Nodes"]
bonobo_nodes_io_csv -- "extends" --> bonobo_nodes_io_base
bonobo_nodes_io_json -- "extends" --> bonobo_nodes_io_base
bonobo_nodes_throttle -- "relies on" --> bonobo_nodes_throttle_RateLimitBucket
bonobo_nodes_basics -- "provides" --> bonobo_nodes_basics_print_console
bonobo_nodes_basics -- "provides" --> bonobo_nodes_basics_print_quiet
bonobo_nodes_basics -- "provides" --> bonobo_nodes_basics_format_console
bonobo_nodes_basics -- "provides" --> bonobo_nodes_basics_format_quiet
bonobo_nodes_basics -- "provides" --> bonobo_nodes_basics_indent
I_O_Nodes -- "Provides Data To" --> Flow_Control_Nodes
Flow_Control_Nodes -- "Regulated By" --> I_O_Nodes
Flow_Control_Nodes -- "Forwards Data To" --> Basic_Utility_Nodes
Basic_Utility_Nodes -- "Processes Data From" --> Flow_Control_Nodes
I_O_Nodes -- "Provides Data To" --> Basic_Utility_Nodes
Basic_Utility_Nodes -- "Consumes Data From" --> I_O_Nodes
The bonobo.nodes subsystem forms the operational core of Bonobo's data processing capabilities. It is structured around three primary abstract components: I/O Nodes, Flow Control Nodes, and Basic Utility Nodes. I/O Nodes are responsible for the fundamental tasks of data extraction and loading, offering specialized implementations for various formats that extend a common base. Flow Control Nodes act as crucial intermediaries, regulating the flow of data through the pipeline to ensure efficient and controlled processing, leveraging internal mechanisms like RateLimitBucket. Finally, Basic Utility Nodes provide essential, often terminal, operations for data inspection, formatting, and console output. Data typically flows from I/O Nodes, potentially through Flow Control Nodes for regulation, and then to Basic Utility Nodes for final display or simple transformations, establishing a clear and manageable data pipeline architecture.
This component provides a foundational set of nodes for handling data input (extraction) from various sources and data output (loading) to different destinations. It abstracts common I/O patterns through a bonobo.nodes.io.base class, which is extended by specialized nodes like bonobo.nodes.io.csv and bonobo.nodes.io.json to support diverse formats.
Related Classes/Methods:
This component implements mechanisms to control the rate at which data flows through a Bonobo graph. The bonobo.nodes.throttle node within this component relies on bonobo.nodes.throttle.RateLimitBucket to manage and enforce data processing limits, acting as a specialized transformer to prevent downstream operations from being overwhelmed.
Related Classes/Methods:
This component offers fundamental utility operations primarily focused on displaying processed data to the console and basic data formatting. The bonobo.nodes.basics module provides functions such as print_console, print_quiet, format_console, format_quiet, and indent, which are often used for debugging, monitoring, or simple output tasks within a pipeline.
Related Classes/Methods: