graph LR
Nimbus_Host_Port_Locator["Nimbus Host/Port Locator"]
Nimbus_Client_Provider["Nimbus Client Provider"]
SSH_Tunnel_Manager["SSH Tunnel Manager"]
Project_Configuration_Loader["Project Configuration Loader"]
Storm_Worker_Information_Retriever["Storm Worker Information Retriever"]
Environment_Activation_Orchestrator["Environment Activation Orchestrator"]
Environment_Configuration_Loader["Environment Configuration Loader"]
Nimbus_SSH_Requirement_Checker["Nimbus SSH Requirement Checker"]
Environment_Activation_Orchestrator -- "orchestrates" --> Environment_Configuration_Loader
Environment_Activation_Orchestrator -- "orchestrates" --> Storm_Worker_Information_Retriever
Environment_Configuration_Loader -- "depends on" --> Project_Configuration_Loader
Nimbus_Client_Provider -- "depends on" --> Nimbus_Host_Port_Locator
Nimbus_Host_Port_Locator -- "retrieves from" --> Project_Configuration_Loader
SSH_Tunnel_Manager -- "establishes" --> Nimbus_Host_Port_Locator
SSH_Tunnel_Manager -- "checks" --> Nimbus_SSH_Requirement_Checker
The streamparse utility subsystem is designed to facilitate seamless interaction with Apache Storm clusters. Its core functionality revolves around a set of specialized components responsible for environment setup, Nimbus service discovery, and secure communication. The Environment Activation Orchestrator serves as the central entry point for preparing the application's operational context, relying on the Environment Configuration Loader to retrieve deployment-specific settings and the Storm Worker Information Retriever to gather cluster worker details. For connecting to the Storm Nimbus service, the Nimbus Client Provider leverages the Nimbus Host/Port Locator to identify the correct service endpoint. In scenarios requiring secure access, the SSH Tunnel Manager dynamically establishes SSH tunnels, guided by the Nimbus SSH Requirement Checker and also utilizing the Nimbus Host/Port Locator for target information. All configuration-related components ultimately depend on the Project Configuration Loader to access the foundational config.json settings, ensuring a unified and flexible configuration management approach.
Identifies the network address (host and port) of the Storm Nimbus service, which is the central entry point for cluster management.
Related Classes/Methods:
Creates and provides a Thrift RPC client object for programmatic interaction with the Storm Nimbus service, enabling remote calls for cluster operations.
Related Classes/Methods:
Establishes and manages secure SSH tunnels to the Storm cluster, ensuring secure communication, especially when use_ssh_for_nimbus is enabled.
Related Classes/Methods:
Parses and loads the main config.json file for the streamparse project, providing access to global project settings.
Related Classes/Methods:
Gathers and provides information about the active worker nodes within the Storm cluster, essential for understanding resource allocation and health.
Related Classes/Methods:
Orchestrates the retrieval of necessary Storm cluster configurations and worker details to prepare the streamparse environment for operations, acting as a setup coordinator.
Related Classes/Methods:
Fetches environment-specific configurations from the loaded project configuration, allowing for flexible deployment across different setups.
Related Classes/Methods:
Determines if an SSH connection is required for communicating with the Nimbus service based on environment configuration, guiding the connection strategy.
Related Classes/Methods: