Skip to content

Data_Input_Output_IO_.md

Latest commit

 

History

History
95 lines (59 loc) · 7.03 KB

Data_Input_Output_IO_.md

File metadata and controls

95 lines (59 loc) · 7.03 KB
graph LR
    File_Based_Data_Ingestors["File-Based Data Ingestors"]
    API_Based_Data_Fetchers["API-Based Data Fetchers"]
    Internal_File_Parsing_Logic["Internal File Parsing Logic"]
    Internal_API_Communication_Logic["Internal API Communication Logic"]
    Shared_Data_Transformation_Utilities["Shared Data Transformation Utilities"]
    Error_Handling_and_Validation["Error Handling and Validation"]
    File_Based_Data_Ingestors -- "delegates to" --> Internal_File_Parsing_Logic
    File_Based_Data_Ingestors -- "sends data to" --> Shared_Data_Transformation_Utilities
    API_Based_Data_Fetchers -- "delegates to" --> Internal_API_Communication_Logic
    API_Based_Data_Fetchers -- "sends data to" --> Shared_Data_Transformation_Utilities
    Internal_File_Parsing_Logic -- "sends data to" --> Shared_Data_Transformation_Utilities
    Internal_File_Parsing_Logic -- "interacts with" --> Error_Handling_and_Validation
    Internal_API_Communication_Logic -- "sends data to" --> Shared_Data_Transformation_Utilities
    Internal_API_Communication_Logic -- "interacts with" --> Error_Handling_and_Validation
    Error_Handling_and_Validation -- "validates" --> Internal_File_Parsing_Logic
    Error_Handling_and_Validation -- "validates" --> Internal_API_Communication_Logic
Loading

CodeBoardingDemoContact

Details

The Data Input/Output (IO) subsystem is a critical part of pvlib-python, responsible for ingesting diverse meteorological and PV system data. It adheres to the project's architectural bias towards a modular, functional design, providing clear interfaces for data acquisition while encapsulating the complexities of various data formats and external APIs.

File-Based Data Ingestors

This component aggregates the functionality for reading and initially structuring meteorological and PV system data from various local file formats, such as EnergyPlus Weather (EPW), Typical Meteorological Year (TMY), Photovoltaic Array Network Operations Data (PANOND), and file-based aspects of Baseline Surface Radiation Network (BSRN) and Measurement and Instrumentation Data Center (MIDC) data. It acts as a facade for specific file parsing logic.

Related Classes/Methods:

API-Based Data Fetchers

This component is responsible for retrieving meteorological and PV system data from external web services and APIs, including Applied Climate Information System (ACIS), Physical Solar Model 3 (PSM3), Physical Solar Model 4 (PSM4), Photovoltaic Geographical Information System (PVGIS), Solcast, and API-based aspects of BSRN and MIDC. It abstracts the complexities of network communication and API interactions.

Related Classes/Methods:

Internal File Parsing Logic

This foundational component provides the core routines for parsing raw data from various file formats into an initial structured format, typically pandas.DataFrame objects. It handles the low-level details of file reading and data extraction.

Related Classes/Methods:

Internal API Communication Logic

This component manages the low-level HTTP requests, API authentication, and communication protocols required to interact with external web services. It ensures reliable data retrieval from remote sources.

Related Classes/Methods:

Shared Data Transformation Utilities

This component standardizes the ingested data, performing crucial transformations such as data type conversions, time zone handling, column renaming, and ensuring the data conforms to pvlib's internal pandas.DataFrame conventions. This prepares the data for subsequent processing by core PV models.

Related Classes/Methods:

Error Handling and Validation

This component provides robust mechanisms for identifying and reporting issues such as malformed input files, API errors, or data quality problems. It includes validation routines to ensure the integrity and usability of the ingested data.

Related Classes/Methods:

FAQ