Skip to content

Graph_Definition_API.md

Latest commit

 

History

History
36 lines (20 loc) · 2.08 KB

Graph_Definition_API.md

File metadata and controls

36 lines (20 loc) · 2.08 KB
graph LR
    Graph["Graph"]
    GraphCursor["GraphCursor"]
    Graph -- "provides" --> GraphCursor
    GraphCursor -- "builds" --> Graph
    GraphCursor -- "modifies" --> Graph
Loading

CodeBoardingDemoContact

Details

The Graph Definition API subsystem is responsible for programmatically defining and structuring data processing pipelines as Directed Acyclic Graphs (DAGs). Its primary function is to declare the structure of ETL pipelines by managing interconnected nodes that represent atomic data operations.

Graph

The Graph component is the foundational element for constructing and representing the entire data processing pipeline. It serves as the primary orchestrator, holding the collection of nodes (ETL operations) and defining their interconnections to establish the complete data flow. It is the concrete representation of the DAG.

Related Classes/Methods:

GraphCursor

The GraphCursor component provides a fluent and intuitive API for incrementally building and connecting nodes within a Graph. It simplifies the process of defining complex data flows, particularly through operator overloading (e.g., >>), acting as a temporary builder or pointer that facilitates the sequential linking of operations.

Related Classes/Methods:

FAQ