diff --git a/README.md b/README.md index 0d086c6..d7d19e5 100644 --- a/README.md +++ b/README.md @@ -22,9 +22,14 @@ The dataframe libraries currently supported are: - [Polars](https://pola.rs) The library offers various custom [Converters](https://docs.haystack.deepset.ai/docs/converters) components to transform dataframes into Haystack [`Document`](https://docs.haystack.deepset.ai/docs/data-classes#document) objects: +- `DataFrameFileToDocument` is a main generic converter that reads files using a dataframe backend and converts them into `Document` objects. - `FileToPandasDataFrame` and `FileToPolarsDataFrame` read files and convert them into dataframes. - `PandasDataFrameConverter` or `PolarsDataFrameConverter` convert data stored in dataframes into Haystack `Document`objects. +`dataframes-haystack` supports reading files in various formats: +- _csv_, _json_, _parquet_, _excel_, _html_, _xml_, _orc_, _pickle_, _fixed-width format_ for `pandas`. See the [pandas documentation](https://pandas.pydata.org/pandas-docs/stable/user_guide/io.html) for more details. +- _csv_, _json_, _parquet_, _excel_, _avro_, _delta_, _ipc_ for `polars`. See the [polars documentation](https://docs.pola.rs/api/python/stable/reference/io.html) for more details. + ## ๐Ÿ› ๏ธ Installation ```sh @@ -40,8 +45,31 @@ pip install "dataframes-haystack[polars]" > [!TIP] > See the [Example Notebooks](./notebooks) for complete examples. +## DataFrameFileToDocument + +[Complete example](https://github.com/EdAbati/dataframes-haystack/blob/main/notebooks/dataframe-file-to-doc-example.ipynb) + +You can leverage both `pandas` and `polars` backends (thanks to [`narwhals`](https://github.com/narwhals-dev/narwhals)) to read your data! + +```python +from dataframes_haystack.components.converters import DataFrameFileToDocument + +converter = DataFrameFileToDocument(content_column="text_str") +documents = converter.run(files=["file1.csv", "file2.csv"]) +``` + +```python +>>> documents +{'documents': [ + Document(id=0, content: 'Hello world', meta: {}), + Document(id=1, content: 'Hello everyone', meta: {}) +]} +``` + ### Pandas +[Complete example](https://github.com/EdAbati/dataframes-haystack/blob/main/notebooks/pandas-example.ipynb) + #### FileToPandasDataFrame ```python @@ -87,6 +115,8 @@ Result: ### Polars +[Complete example](https://github.com/EdAbati/dataframes-haystack/blob/main/notebooks/polars-example.ipynb) + #### FileToPolarsDataFrame ```python diff --git a/notebooks/dataframe-file-to-doc-example.ipynb b/notebooks/dataframe-file-to-doc-example.ipynb new file mode 100644 index 0000000..43c5a0c --- /dev/null +++ b/notebooks/dataframe-file-to-doc-example.ipynb @@ -0,0 +1,467 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Using `DataFrameFileToDocument` with `dataframe-haystack`\n", + "\n", + "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/EdAbati/dataframes-haystack/blob/main/notebooks/dataframe-file-to-doc-example.ipynb)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "# when running in Google Colab (or similar), install the following packages\n", + "# !pip install dataframe-haystack arxiv 'polars[timezone]' pyarrow" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Downloading the dataset\n", + "\n", + "We are using a dataset that contains abstracts of papers uploaded on arXiv. We are using the arXiv API to get the data." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "# we import polars just to create and store an example dataframe\n", + "import polars as pl\n", + "\n", + "\n", + "def get_arxiv_data(search_query: str, max_num_rows: int = 10) -> pl.DataFrame:\n", + " \"\"\"Get data using the arXiv API.\"\"\"\n", + " import arxiv\n", + "\n", + " arxiv_client = arxiv.Client()\n", + "\n", + " search = arxiv.Search(query=search_query, max_results=max_num_rows, sort_by=arxiv.SortCriterion.Relevance)\n", + " results_list = [\n", + " {\n", + " \"title\": result.title,\n", + " \"authors\": [author.name for author in result.authors],\n", + " \"summary\": result.summary,\n", + " \"published\": result.published,\n", + " \"primary_category\": result.primary_category,\n", + " \"categories\": result.categories,\n", + " \"pdf_url\": result.pdf_url,\n", + " \"entry_id\": result.entry_id,\n", + " }\n", + " for result in arxiv_client.results(search)\n", + " ]\n", + " return pl.DataFrame(results_list)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We will have a dataset of 10 rows with papers about LLMs." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "shape: (5, 8)
titleauthorssummarypublishedprimary_categorycategoriespdf_urlentry_id
strlist[str]strdatetime[ฮผs, UTC]strlist[str]strstr
"Large Language Models as Softwโ€ฆ["Irene Weber"]"Large Language Models (LLMs) hโ€ฆ2024-06-13 21:32:56 UTC"cs.SE"["cs.SE", "cs.CL", โ€ฆ "A.1; I.2.7; D.2.11"]"http://arxiv.org/pdf/2406.1030โ€ฆ"http://arxiv.org/abs/2406.1030โ€ฆ
"Parrot: Efficient Serving of Lโ€ฆ["Chaofan Lin", "Zhenhua Han", โ€ฆ "Lili Qiu"]"The rise of large language modโ€ฆ2024-05-30 09:46:36 UTC"cs.LG"["cs.LG", "cs.AI"]"http://arxiv.org/pdf/2405.1988โ€ฆ"http://arxiv.org/abs/2405.1988โ€ฆ
"A Survey of Large Language Modโ€ฆ["Zibin Zheng", "Kaiwen Ning", โ€ฆ "Jiachi Chen"]"General large language models โ€ฆ2023-11-17 07:55:16 UTC"cs.SE"["cs.SE"]"http://arxiv.org/pdf/2311.1037โ€ฆ"http://arxiv.org/abs/2311.1037โ€ฆ
"A Survey of Large Language Modโ€ฆ["Wenbo Shang", "Xin Huang"]"A graph is a fundamental data โ€ฆ2024-04-23 07:39:24 UTC"cs.CL"["cs.CL", "cs.AI", "cs.DB"]"http://arxiv.org/pdf/2404.1480โ€ฆ"http://arxiv.org/abs/2404.1480โ€ฆ
"TEST: Text Prototype Aligned Eโ€ฆ["Chenxi Sun", "Hongyan Li", โ€ฆ "Shenda Hong"]"This work summarizes two ways โ€ฆ2023-08-16 09:16:02 UTC"cs.CL"["cs.CL", "cs.AI"]"http://arxiv.org/pdf/2308.0824โ€ฆ"http://arxiv.org/abs/2308.0824โ€ฆ
" + ], + "text/plain": [ + "shape: (5, 8)\n", + "โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”\n", + "โ”‚ title โ”† authors โ”† summary โ”† published โ”† primary_c โ”† categorie โ”† pdf_url โ”† entry_id โ”‚\n", + "โ”‚ --- โ”† --- โ”† --- โ”† --- โ”† ategory โ”† s โ”† --- โ”† --- โ”‚\n", + "โ”‚ str โ”† list[str] โ”† str โ”† datetime[ โ”† --- โ”† --- โ”† str โ”† str โ”‚\n", + "โ”‚ โ”† โ”† โ”† ฮผs, UTC] โ”† str โ”† list[str] โ”† โ”† โ”‚\n", + "โ•žโ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•ชโ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•ชโ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•ชโ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•ชโ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•ชโ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•ชโ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•ชโ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•ก\n", + "โ”‚ Large โ”† [\"Irene โ”† Large โ”† 2024-06-1 โ”† cs.SE โ”† [\"cs.SE\", โ”† http://ar โ”† http://ar โ”‚\n", + "โ”‚ Language โ”† Weber\"] โ”† Language โ”† 3 โ”† โ”† \"cs.CL\", โ”† xiv.org/p โ”† xiv.org/a โ”‚\n", + "โ”‚ Models as โ”† โ”† Models โ”† 21:32:56 โ”† โ”† โ€ฆ \"A.1; โ”† df/2406.1 โ”† bs/2406.1 โ”‚\n", + "โ”‚ Softwโ€ฆ โ”† โ”† (LLMs) hโ€ฆ โ”† UTC โ”† โ”† I.2โ€ฆ โ”† 030โ€ฆ โ”† 030โ€ฆ โ”‚\n", + "โ”‚ Parrot: โ”† [\"Chaofan โ”† The rise โ”† 2024-05-3 โ”† cs.LG โ”† [\"cs.LG\", โ”† http://ar โ”† http://ar โ”‚\n", + "โ”‚ Efficient โ”† Lin\", โ”† of large โ”† 0 โ”† โ”† \"cs.AI\"] โ”† xiv.org/p โ”† xiv.org/a โ”‚\n", + "โ”‚ Serving of โ”† \"Zhenhua โ”† language โ”† 09:46:36 โ”† โ”† โ”† df/2405.1 โ”† bs/2405.1 โ”‚\n", + "โ”‚ Lโ€ฆ โ”† Han\",โ€ฆ โ”† modโ€ฆ โ”† UTC โ”† โ”† โ”† 988โ€ฆ โ”† 988โ€ฆ โ”‚\n", + "โ”‚ A Survey โ”† [\"Zibin โ”† General โ”† 2023-11-1 โ”† cs.SE โ”† [\"cs.SE\"] โ”† http://ar โ”† http://ar โ”‚\n", + "โ”‚ of Large โ”† Zheng\", โ”† large โ”† 7 โ”† โ”† โ”† xiv.org/p โ”† xiv.org/a โ”‚\n", + "โ”‚ Language โ”† \"Kaiwen โ”† language โ”† 07:55:16 โ”† โ”† โ”† df/2311.1 โ”† bs/2311.1 โ”‚\n", + "โ”‚ Modโ€ฆ โ”† Ning\",โ€ฆ โ”† models โ€ฆ โ”† UTC โ”† โ”† โ”† 037โ€ฆ โ”† 037โ€ฆ โ”‚\n", + "โ”‚ A Survey โ”† [\"Wenbo โ”† A graph is โ”† 2024-04-2 โ”† cs.CL โ”† [\"cs.CL\", โ”† http://ar โ”† http://ar โ”‚\n", + "โ”‚ of Large โ”† Shang\", โ”† a fundamen โ”† 3 โ”† โ”† \"cs.AI\", โ”† xiv.org/p โ”† xiv.org/a โ”‚\n", + "โ”‚ Language โ”† \"Xin โ”† tal data โ€ฆ โ”† 07:39:24 โ”† โ”† \"cs.DB\"] โ”† df/2404.1 โ”† bs/2404.1 โ”‚\n", + "โ”‚ Modโ€ฆ โ”† Huang\"] โ”† โ”† UTC โ”† โ”† โ”† 480โ€ฆ โ”† 480โ€ฆ โ”‚\n", + "โ”‚ TEST: Text โ”† [\"Chenxi โ”† This work โ”† 2023-08-1 โ”† cs.CL โ”† [\"cs.CL\", โ”† http://ar โ”† http://ar โ”‚\n", + "โ”‚ Prototype โ”† Sun\", โ”† summarizes โ”† 6 โ”† โ”† \"cs.AI\"] โ”† xiv.org/p โ”† xiv.org/a โ”‚\n", + "โ”‚ Aligned Eโ€ฆ โ”† \"Hongyan โ”† two ways โ€ฆ โ”† 09:16:02 โ”† โ”† โ”† df/2308.0 โ”† bs/2308.0 โ”‚\n", + "โ”‚ โ”† Li\", โ€ฆโ€ฆ โ”† โ”† UTC โ”† โ”† โ”† 824โ€ฆ โ”† 824โ€ฆ โ”‚\n", + "โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df = get_arxiv_data(\"llm\", max_num_rows=10)\n", + "df.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Saving the data in a temporary file." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "import tempfile\n", + "\n", + "# create a temporary parquet file\n", + "temp_file_parquet = tempfile.NamedTemporaryFile(delete=False, suffix=\".parquet\")\n", + "\n", + "# write the dataframe to the temporary file\n", + "with open(temp_file_parquet.name, \"w\") as f:\n", + " df.write_parquet(f)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Loading the data in `haystack`\n", + "\n", + "We can read the data and convert the rows into `Document`s using the `DataFrameFileToDocument` component. It can be used to convert a DataFrame into a list of `Document`s." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/Users/edoardoabati/Library/Application Support/hatch/env/virtual/dataframes-haystack/Gal9cSh8/dataframes-haystack/lib/python3.8/site-packages/haystack/core/errors.py:34: DeprecationWarning: PipelineMaxLoops is deprecated and will be remove in version '2.7.0'; use PipelineMaxComponentRuns instead.\n", + " warnings.warn(\n" + ] + } + ], + "source": [ + "from dataframes_haystack.components.converters import DataFrameFileToDocument" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "file_converter = DataFrameFileToDocument(\n", + " content_column=\"summary\",\n", + " meta_columns=[\"title\", \"authors\", \"published\", \"primary_category\", \"categories\", \"pdf_url\"],\n", + " file_format=\"parquet\",\n", + " backend=\"polars\",\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'documents': [Document(id=b550a0163e880995f1895a14ae3410ef88bb9d312bd69a149c125a2d9087afe1, content: 'Large Language Models (LLMs) have become widely adopted recently. Research\n", + " explores their use both a...', meta: {'title': 'Large Language Models as Software Components: A Taxonomy for LLM-Integrated Applications', 'authors': ['Irene Weber'], 'published': datetime.datetime(2024, 6, 13, 21, 32, 56, tzinfo=backports.zoneinfo.ZoneInfo(key='UTC')), 'primary_category': 'cs.SE', 'categories': ['cs.SE', 'cs.CL', 'cs.LG', 'A.1; I.2.7; D.2.11'], 'pdf_url': 'http://arxiv.org/pdf/2406.10300v1'}),\n", + " Document(id=fc64aeb2ac791e14cd3e1ec24d74bee04ce2c8cb8428e70fb040cafbab76365e, content: 'The rise of large language models (LLMs) has enabled LLM-based applications\n", + " (a.k.a. AI agents or co-...', meta: {'title': 'Parrot: Efficient Serving of LLM-based Applications with Semantic Variable', 'authors': ['Chaofan Lin', 'Zhenhua Han', 'Chengruidong Zhang', 'Yuqing Yang', 'Fan Yang', 'Chen Chen', 'Lili Qiu'], 'published': datetime.datetime(2024, 5, 30, 9, 46, 36, tzinfo=backports.zoneinfo.ZoneInfo(key='UTC')), 'primary_category': 'cs.LG', 'categories': ['cs.LG', 'cs.AI'], 'pdf_url': 'http://arxiv.org/pdf/2405.19888v1'}),\n", + " Document(id=479b45f57528bf4dd23e1c01314b399bca4b125a2065f978ad139b59d5f1f21c, content: 'General large language models (LLMs), represented by ChatGPT, have\n", + " demonstrated significant potentia...', meta: {'title': 'A Survey of Large Language Models for Code: Evolution, Benchmarking, and Future Trends', 'authors': ['Zibin Zheng', 'Kaiwen Ning', 'Yanlin Wang', 'Jingwen Zhang', 'Dewu Zheng', 'Mingxi Ye', 'Jiachi Chen'], 'published': datetime.datetime(2023, 11, 17, 7, 55, 16, tzinfo=backports.zoneinfo.ZoneInfo(key='UTC')), 'primary_category': 'cs.SE', 'categories': ['cs.SE'], 'pdf_url': 'http://arxiv.org/pdf/2311.10372v2'}),\n", + " Document(id=de8b9ac5619193f037562f2f38e4052df96d0e7ff3d8ff4a387898e9fc7fd79e, content: 'A graph is a fundamental data model to represent various entities and their\n", + " complex relationships in...', meta: {'title': 'A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applications', 'authors': ['Wenbo Shang', 'Xin Huang'], 'published': datetime.datetime(2024, 4, 23, 7, 39, 24, tzinfo=backports.zoneinfo.ZoneInfo(key='UTC')), 'primary_category': 'cs.CL', 'categories': ['cs.CL', 'cs.AI', 'cs.DB'], 'pdf_url': 'http://arxiv.org/pdf/2404.14809v1'}),\n", + " Document(id=4d0bdfbea62bfc9b2ab68ae10d0cd51ebc4728debabb9758a1ebd5e7a37b879b, content: 'This work summarizes two ways to accomplish Time-Series (TS) tasks in today's\n", + " Large Language Model (...', meta: {'title': \"TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Series\", 'authors': ['Chenxi Sun', 'Hongyan Li', 'Yaliang Li', 'Shenda Hong'], 'published': datetime.datetime(2023, 8, 16, 9, 16, 2, tzinfo=backports.zoneinfo.ZoneInfo(key='UTC')), 'primary_category': 'cs.CL', 'categories': ['cs.CL', 'cs.AI'], 'pdf_url': 'http://arxiv.org/pdf/2308.08241v2'}),\n", + " Document(id=dbc7fa154f875811cab410c9ed24e88c1ffb5813d9ae0f0d8be96ff7958be2ba, content: 'With the rise of large language models (LLMs), researchers are increasingly\n", + " exploring their applicat...', meta: {'title': 'From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future', 'authors': ['Haolin Jin', 'Linghan Huang', 'Haipeng Cai', 'Jun Yan', 'Bo Li', 'Huaming Chen'], 'published': datetime.datetime(2024, 8, 5, 14, 1, 15, tzinfo=backports.zoneinfo.ZoneInfo(key='UTC')), 'primary_category': 'cs.SE', 'categories': ['cs.SE', 'cs.AI', 'cs.CL'], 'pdf_url': 'http://arxiv.org/pdf/2408.02479v1'}),\n", + " Document(id=98ffd575fed718fbd6458fd46d785d188cb41396fef346da603decff998a4720, content: 'Large language models (LLMs) can label data faster and cheaper than humans\n", + " for various NLP tasks. De...', meta: {'title': 'MEGAnno+: A Human-LLM Collaborative Annotation System', 'authors': ['Hannah Kim', 'Kushan Mitra', 'Rafael Li Chen', 'Sajjadur Rahman', 'Dan Zhang'], 'published': datetime.datetime(2024, 2, 28, 4, 58, 7, tzinfo=backports.zoneinfo.ZoneInfo(key='UTC')), 'primary_category': 'cs.CL', 'categories': ['cs.CL', 'cs.HC'], 'pdf_url': 'http://arxiv.org/pdf/2402.18050v1'}),\n", + " Document(id=a04fd3e32828716a495b7575b24f62343f6033b54e3fa671ee7bf38aaa4c27b2, content: 'Large Language Model (LLM) assistants, such as ChatGPT, have emerged as\n", + " potential alternatives to se...', meta: {'title': 'Why and When LLM-Based Assistants Can Go Wrong: Investigating the Effectiveness of Prompt-Based Interactions for Software Help-Seeking', 'authors': ['Anjali Khurana', 'Hari Subramonyam', 'Parmit K Chilana'], 'published': datetime.datetime(2024, 2, 12, 19, 49, 58, tzinfo=backports.zoneinfo.ZoneInfo(key='UTC')), 'primary_category': 'cs.HC', 'categories': ['cs.HC', 'cs.AI', 'cs.LG'], 'pdf_url': 'http://arxiv.org/pdf/2402.08030v1'}),\n", + " Document(id=5a34f6971b2877a873b03fcf886acdfba8b60328239408b1d7874b1c7c7879bd, content: 'Alignment approaches such as RLHF and DPO are actively investigated to align\n", + " large language models (...', meta: {'title': 'Systematic Evaluation of LLM-as-a-Judge in LLM Alignment Tasks: Explainable Metrics and Diverse Prompt Templates', 'authors': ['Hui Wei', 'Shenghua He', 'Tian Xia', 'Andy Wong', 'Jingyang Lin', 'Mei Han'], 'published': datetime.datetime(2024, 8, 23, 11, 49, 1, tzinfo=backports.zoneinfo.ZoneInfo(key='UTC')), 'primary_category': 'cs.CL', 'categories': ['cs.CL'], 'pdf_url': 'http://arxiv.org/pdf/2408.13006v1'}),\n", + " Document(id=8ab28c8637feaba2772674b7e1b8d30ae1675a9aa617c5213e3c99d65bb44fc7, content: 'Since late 2022, Large Language Models (LLMs) have become very prominent with\n", + " LLMs like ChatGPT and ...', meta: {'title': 'On the Origin of LLMs: An Evolutionary Tree and Graph for 15,821 Large Language Models', 'authors': ['Sarah Gao', 'Andrew Kean Gao'], 'published': datetime.datetime(2023, 7, 19, 7, 17, 43, tzinfo=backports.zoneinfo.ZoneInfo(key='UTC')), 'primary_category': 'cs.DL', 'categories': ['cs.DL', 'cs.CL', 'I.2.1; H.5.0'], 'pdf_url': 'http://arxiv.org/pdf/2307.09793v1'})]}" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "file_converter.run(file_paths=[temp_file_parquet.name])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If we want to use a column of the dataframe as unique identifier for documents, we can set the `index_column` parameter." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'documents': [Document(id=http://arxiv.org/abs/2406.10300v1, content: 'Large Language Models (LLMs) have become widely adopted recently. Research\n", + " explores their use both a...', meta: {'title': 'Large Language Models as Software Components: A Taxonomy for LLM-Integrated Applications', 'authors': ['Irene Weber'], 'published': datetime.datetime(2024, 6, 13, 21, 32, 56, tzinfo=backports.zoneinfo.ZoneInfo(key='UTC')), 'primary_category': 'cs.SE', 'categories': ['cs.SE', 'cs.CL', 'cs.LG', 'A.1; I.2.7; D.2.11'], 'pdf_url': 'http://arxiv.org/pdf/2406.10300v1'}),\n", + " Document(id=http://arxiv.org/abs/2405.19888v1, content: 'The rise of large language models (LLMs) has enabled LLM-based applications\n", + " (a.k.a. AI agents or co-...', meta: {'title': 'Parrot: Efficient Serving of LLM-based Applications with Semantic Variable', 'authors': ['Chaofan Lin', 'Zhenhua Han', 'Chengruidong Zhang', 'Yuqing Yang', 'Fan Yang', 'Chen Chen', 'Lili Qiu'], 'published': datetime.datetime(2024, 5, 30, 9, 46, 36, tzinfo=backports.zoneinfo.ZoneInfo(key='UTC')), 'primary_category': 'cs.LG', 'categories': ['cs.LG', 'cs.AI'], 'pdf_url': 'http://arxiv.org/pdf/2405.19888v1'}),\n", + " Document(id=http://arxiv.org/abs/2311.10372v2, content: 'General large language models (LLMs), represented by ChatGPT, have\n", + " demonstrated significant potentia...', meta: {'title': 'A Survey of Large Language Models for Code: Evolution, Benchmarking, and Future Trends', 'authors': ['Zibin Zheng', 'Kaiwen Ning', 'Yanlin Wang', 'Jingwen Zhang', 'Dewu Zheng', 'Mingxi Ye', 'Jiachi Chen'], 'published': datetime.datetime(2023, 11, 17, 7, 55, 16, tzinfo=backports.zoneinfo.ZoneInfo(key='UTC')), 'primary_category': 'cs.SE', 'categories': ['cs.SE'], 'pdf_url': 'http://arxiv.org/pdf/2311.10372v2'}),\n", + " Document(id=http://arxiv.org/abs/2404.14809v1, content: 'A graph is a fundamental data model to represent various entities and their\n", + " complex relationships in...', meta: {'title': 'A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applications', 'authors': ['Wenbo Shang', 'Xin Huang'], 'published': datetime.datetime(2024, 4, 23, 7, 39, 24, tzinfo=backports.zoneinfo.ZoneInfo(key='UTC')), 'primary_category': 'cs.CL', 'categories': ['cs.CL', 'cs.AI', 'cs.DB'], 'pdf_url': 'http://arxiv.org/pdf/2404.14809v1'}),\n", + " Document(id=http://arxiv.org/abs/2308.08241v2, content: 'This work summarizes two ways to accomplish Time-Series (TS) tasks in today's\n", + " Large Language Model (...', meta: {'title': \"TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Series\", 'authors': ['Chenxi Sun', 'Hongyan Li', 'Yaliang Li', 'Shenda Hong'], 'published': datetime.datetime(2023, 8, 16, 9, 16, 2, tzinfo=backports.zoneinfo.ZoneInfo(key='UTC')), 'primary_category': 'cs.CL', 'categories': ['cs.CL', 'cs.AI'], 'pdf_url': 'http://arxiv.org/pdf/2308.08241v2'}),\n", + " Document(id=http://arxiv.org/abs/2408.02479v1, content: 'With the rise of large language models (LLMs), researchers are increasingly\n", + " exploring their applicat...', meta: {'title': 'From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future', 'authors': ['Haolin Jin', 'Linghan Huang', 'Haipeng Cai', 'Jun Yan', 'Bo Li', 'Huaming Chen'], 'published': datetime.datetime(2024, 8, 5, 14, 1, 15, tzinfo=backports.zoneinfo.ZoneInfo(key='UTC')), 'primary_category': 'cs.SE', 'categories': ['cs.SE', 'cs.AI', 'cs.CL'], 'pdf_url': 'http://arxiv.org/pdf/2408.02479v1'}),\n", + " Document(id=http://arxiv.org/abs/2402.18050v1, content: 'Large language models (LLMs) can label data faster and cheaper than humans\n", + " for various NLP tasks. De...', meta: {'title': 'MEGAnno+: A Human-LLM Collaborative Annotation System', 'authors': ['Hannah Kim', 'Kushan Mitra', 'Rafael Li Chen', 'Sajjadur Rahman', 'Dan Zhang'], 'published': datetime.datetime(2024, 2, 28, 4, 58, 7, tzinfo=backports.zoneinfo.ZoneInfo(key='UTC')), 'primary_category': 'cs.CL', 'categories': ['cs.CL', 'cs.HC'], 'pdf_url': 'http://arxiv.org/pdf/2402.18050v1'}),\n", + " Document(id=http://arxiv.org/abs/2402.08030v1, content: 'Large Language Model (LLM) assistants, such as ChatGPT, have emerged as\n", + " potential alternatives to se...', meta: {'title': 'Why and When LLM-Based Assistants Can Go Wrong: Investigating the Effectiveness of Prompt-Based Interactions for Software Help-Seeking', 'authors': ['Anjali Khurana', 'Hari Subramonyam', 'Parmit K Chilana'], 'published': datetime.datetime(2024, 2, 12, 19, 49, 58, tzinfo=backports.zoneinfo.ZoneInfo(key='UTC')), 'primary_category': 'cs.HC', 'categories': ['cs.HC', 'cs.AI', 'cs.LG'], 'pdf_url': 'http://arxiv.org/pdf/2402.08030v1'}),\n", + " Document(id=http://arxiv.org/abs/2408.13006v1, content: 'Alignment approaches such as RLHF and DPO are actively investigated to align\n", + " large language models (...', meta: {'title': 'Systematic Evaluation of LLM-as-a-Judge in LLM Alignment Tasks: Explainable Metrics and Diverse Prompt Templates', 'authors': ['Hui Wei', 'Shenghua He', 'Tian Xia', 'Andy Wong', 'Jingyang Lin', 'Mei Han'], 'published': datetime.datetime(2024, 8, 23, 11, 49, 1, tzinfo=backports.zoneinfo.ZoneInfo(key='UTC')), 'primary_category': 'cs.CL', 'categories': ['cs.CL'], 'pdf_url': 'http://arxiv.org/pdf/2408.13006v1'}),\n", + " Document(id=http://arxiv.org/abs/2307.09793v1, content: 'Since late 2022, Large Language Models (LLMs) have become very prominent with\n", + " LLMs like ChatGPT and ...', meta: {'title': 'On the Origin of LLMs: An Evolutionary Tree and Graph for 15,821 Large Language Models', 'authors': ['Sarah Gao', 'Andrew Kean Gao'], 'published': datetime.datetime(2023, 7, 19, 7, 17, 43, tzinfo=backports.zoneinfo.ZoneInfo(key='UTC')), 'primary_category': 'cs.DL', 'categories': ['cs.DL', 'cs.CL', 'I.2.1; H.5.0'], 'pdf_url': 'http://arxiv.org/pdf/2307.09793v1'})]}" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "file_converter = DataFrameFileToDocument(\n", + " content_column=\"summary\",\n", + " meta_columns=[\"title\", \"authors\", \"published\", \"primary_category\", \"categories\", \"pdf_url\"],\n", + " index_column=\"entry_id\",\n", + " file_format=\"parquet\",\n", + " backend=\"polars\",\n", + ")\n", + "file_converter.run(file_paths=[temp_file_parquet.name])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If you want to use `pandas` as the backend to read the data, you can set `backend=\"pandas\"`." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'documents': [Document(id=http://arxiv.org/abs/2406.10300v1, content: 'Large Language Models (LLMs) have become widely adopted recently. Research\n", + " explores their use both a...', meta: {'title': 'Large Language Models as Software Components: A Taxonomy for LLM-Integrated Applications', 'authors': array(['Irene Weber'], dtype=object), 'published': Timestamp('2024-06-13 21:32:56+0000', tz='UTC'), 'primary_category': 'cs.SE', 'categories': array(['cs.SE', 'cs.CL', 'cs.LG', 'A.1; I.2.7; D.2.11'], dtype=object), 'pdf_url': 'http://arxiv.org/pdf/2406.10300v1'}),\n", + " Document(id=http://arxiv.org/abs/2405.19888v1, content: 'The rise of large language models (LLMs) has enabled LLM-based applications\n", + " (a.k.a. AI agents or co-...', meta: {'title': 'Parrot: Efficient Serving of LLM-based Applications with Semantic Variable', 'authors': array(['Chaofan Lin', 'Zhenhua Han', 'Chengruidong Zhang', 'Yuqing Yang',\n", + " 'Fan Yang', 'Chen Chen', 'Lili Qiu'], dtype=object), 'published': Timestamp('2024-05-30 09:46:36+0000', tz='UTC'), 'primary_category': 'cs.LG', 'categories': array(['cs.LG', 'cs.AI'], dtype=object), 'pdf_url': 'http://arxiv.org/pdf/2405.19888v1'}),\n", + " Document(id=http://arxiv.org/abs/2311.10372v2, content: 'General large language models (LLMs), represented by ChatGPT, have\n", + " demonstrated significant potentia...', meta: {'title': 'A Survey of Large Language Models for Code: Evolution, Benchmarking, and Future Trends', 'authors': array(['Zibin Zheng', 'Kaiwen Ning', 'Yanlin Wang', 'Jingwen Zhang',\n", + " 'Dewu Zheng', 'Mingxi Ye', 'Jiachi Chen'], dtype=object), 'published': Timestamp('2023-11-17 07:55:16+0000', tz='UTC'), 'primary_category': 'cs.SE', 'categories': array(['cs.SE'], dtype=object), 'pdf_url': 'http://arxiv.org/pdf/2311.10372v2'}),\n", + " Document(id=http://arxiv.org/abs/2404.14809v1, content: 'A graph is a fundamental data model to represent various entities and their\n", + " complex relationships in...', meta: {'title': 'A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applications', 'authors': array(['Wenbo Shang', 'Xin Huang'], dtype=object), 'published': Timestamp('2024-04-23 07:39:24+0000', tz='UTC'), 'primary_category': 'cs.CL', 'categories': array(['cs.CL', 'cs.AI', 'cs.DB'], dtype=object), 'pdf_url': 'http://arxiv.org/pdf/2404.14809v1'}),\n", + " Document(id=http://arxiv.org/abs/2308.08241v2, content: 'This work summarizes two ways to accomplish Time-Series (TS) tasks in today's\n", + " Large Language Model (...', meta: {'title': \"TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Series\", 'authors': array(['Chenxi Sun', 'Hongyan Li', 'Yaliang Li', 'Shenda Hong'],\n", + " dtype=object), 'published': Timestamp('2023-08-16 09:16:02+0000', tz='UTC'), 'primary_category': 'cs.CL', 'categories': array(['cs.CL', 'cs.AI'], dtype=object), 'pdf_url': 'http://arxiv.org/pdf/2308.08241v2'}),\n", + " Document(id=http://arxiv.org/abs/2408.02479v1, content: 'With the rise of large language models (LLMs), researchers are increasingly\n", + " exploring their applicat...', meta: {'title': 'From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future', 'authors': array(['Haolin Jin', 'Linghan Huang', 'Haipeng Cai', 'Jun Yan', 'Bo Li',\n", + " 'Huaming Chen'], dtype=object), 'published': Timestamp('2024-08-05 14:01:15+0000', tz='UTC'), 'primary_category': 'cs.SE', 'categories': array(['cs.SE', 'cs.AI', 'cs.CL'], dtype=object), 'pdf_url': 'http://arxiv.org/pdf/2408.02479v1'}),\n", + " Document(id=http://arxiv.org/abs/2402.18050v1, content: 'Large language models (LLMs) can label data faster and cheaper than humans\n", + " for various NLP tasks. De...', meta: {'title': 'MEGAnno+: A Human-LLM Collaborative Annotation System', 'authors': array(['Hannah Kim', 'Kushan Mitra', 'Rafael Li Chen', 'Sajjadur Rahman',\n", + " 'Dan Zhang'], dtype=object), 'published': Timestamp('2024-02-28 04:58:07+0000', tz='UTC'), 'primary_category': 'cs.CL', 'categories': array(['cs.CL', 'cs.HC'], dtype=object), 'pdf_url': 'http://arxiv.org/pdf/2402.18050v1'}),\n", + " Document(id=http://arxiv.org/abs/2402.08030v1, content: 'Large Language Model (LLM) assistants, such as ChatGPT, have emerged as\n", + " potential alternatives to se...', meta: {'title': 'Why and When LLM-Based Assistants Can Go Wrong: Investigating the Effectiveness of Prompt-Based Interactions for Software Help-Seeking', 'authors': array(['Anjali Khurana', 'Hari Subramonyam', 'Parmit K Chilana'],\n", + " dtype=object), 'published': Timestamp('2024-02-12 19:49:58+0000', tz='UTC'), 'primary_category': 'cs.HC', 'categories': array(['cs.HC', 'cs.AI', 'cs.LG'], dtype=object), 'pdf_url': 'http://arxiv.org/pdf/2402.08030v1'}),\n", + " Document(id=http://arxiv.org/abs/2408.13006v1, content: 'Alignment approaches such as RLHF and DPO are actively investigated to align\n", + " large language models (...', meta: {'title': 'Systematic Evaluation of LLM-as-a-Judge in LLM Alignment Tasks: Explainable Metrics and Diverse Prompt Templates', 'authors': array(['Hui Wei', 'Shenghua He', 'Tian Xia', 'Andy Wong', 'Jingyang Lin',\n", + " 'Mei Han'], dtype=object), 'published': Timestamp('2024-08-23 11:49:01+0000', tz='UTC'), 'primary_category': 'cs.CL', 'categories': array(['cs.CL'], dtype=object), 'pdf_url': 'http://arxiv.org/pdf/2408.13006v1'}),\n", + " Document(id=http://arxiv.org/abs/2307.09793v1, content: 'Since late 2022, Large Language Models (LLMs) have become very prominent with\n", + " LLMs like ChatGPT and ...', meta: {'title': 'On the Origin of LLMs: An Evolutionary Tree and Graph for 15,821 Large Language Models', 'authors': array(['Sarah Gao', 'Andrew Kean Gao'], dtype=object), 'published': Timestamp('2023-07-19 07:17:43+0000', tz='UTC'), 'primary_category': 'cs.DL', 'categories': array(['cs.DL', 'cs.CL', 'I.2.1; H.5.0'], dtype=object), 'pdf_url': 'http://arxiv.org/pdf/2307.09793v1'})]}" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pandas_file_converter = DataFrameFileToDocument(\n", + " content_column=\"summary\",\n", + " meta_columns=[\"title\", \"authors\", \"published\", \"primary_category\", \"categories\", \"pdf_url\"],\n", + " index_column=\"entry_id\",\n", + " file_format=\"parquet\",\n", + " backend=\"pandas\",\n", + ")\n", + "pandas_file_converter.run(file_paths=[temp_file_parquet.name])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Using the components in a pipeline" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "from haystack import Pipeline\n", + "from haystack.components.writers import DocumentWriter\n", + "from haystack.document_stores.in_memory import InMemoryDocumentStore" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAANKCAYAAABRV1FZAAAAAXNSR0IArs4c6QAAIABJREFUeJzs3XlYVHX///HnsAiyKbizigKKgKKo5JZbai6RpJWWuWZp3dV9fzUr00oz0TS1bs01tcW6LW8190xNszIXUlmGRUARQRBxAdlhzu+Pmzk/RkTBGBF4P66L63LOnPM57zkz5zWfc854PiCEEEIIIYQQQgghhBBCCCGEEEIIIUTNp6nuAkTNsWbNmjwTExNNvXr1HMaOHZtd3fUIIWqJNWvW5F25ckW5du2asm7duvyvvvrKurprEkLUAqXDRf8nISMqQg6RxF2tWbMmb/jw4RZmZmZlntu6dWuBHC6Ju5GAEeW6W7joSciIu5GAEXdUkXDRk5AR5ZGAEWVUJlz0JGTEnZhUdwHi4XI/4QLw1FNP1cvPz78mJ35FadKDEar7DZfS/vvf/xZYWFhIT0aABIzQq4pw0ZOQEXoSMKJKw0VPQkYgASOMES56EjJCAqYOM2a46EnI1G0SMHXUgwgXPQmZuksCpg56kOGiJyFTN0nA1DHVES56EjJ1jwRMHVKd4aInIVO3SMDUEQ9DuOhJyNQdEjB1wMMULnoSMnWDBEwt9zCGi56ETO0nAVOLPczhoichU7tJwNRSNSFc9CRkai8JmFpo7dq1uU8++aRlTQgXPQmZ2knuB1PLlIRLPWOES+/evZk7d26VtwswYsQIuZ9MLSQ9mFqkVLhU6RdHVFQUP/74I6mpqXTr1g2tVsvMmTOxsLCoytWA9GRqHQmYWsJY4QKwY8cOFi1aRFpaGo0aNcLU1JR169bRtm3bql4VSMjUKnKIVAsYM1wAgoKCCA4Oxt/fH41GwwcffGC0cEEOl2oVCZgaztjhApCXl8dvv/3GzJkzmTBhAocOHTLWqlQSMrWDHCLVYA8iXG6nKAoazYP72MjhUs0mAVMDrVixwsbS0jLxiSeeaPggw6W6bN++PbO4uNhr8uTJadVdi6icWv/hrI1effXVW6ampjnZ2bX/Sz0/Px+dTmcm4VIzSQ+mBtu4cWNS7969HRs0aFArvyjy8/PZvXt3zosvvijnYWooCZgarraGjIRL7SABUwvUtpCRcKk9JGBqidoSMhIutYsETC1S00NGwqX2kYCpZWpqyEi41E416kMo7m38+PEuR44cSbl582Z1l1JhEi61l/RgaqmNGzem9O7du0WDBg2qu5S7knCp3SRgarGHPWQkXGo/CZha7mENGQmXukECpg542EJGwqXukICpIx6WkJFwqVskYOqQ6g4ZCZe6RwKmjqmukJFwqZskYOqgBx0yEi51lwRMHfWgQkbCpW6TgKnDjB0yEi5CAqaOM1bISLgIJGAERggZCRehJwEjoApDRsJFlCYBI1R/N2QkXMTtJGCEgfsNmfz8fPbs2ZM7adIkK6MVJ2ocCRhRRmVDRsJFlEcCRtxRRUNGwkUIcV82btyYcv78eeXatWt3/Lt8+bLyxRdf5FR3nUKIGqq8kJFwEUJUidtDRsJFCFGl9CEj4SKEMIpNmzbFfPnll1eruw4hRC01efLkVS+99JJ5ddchagYZF0lU1jgLCwv53IgKkQ+KqBSNRmMaHh5eXN11iJpBAkZUlunhw4clYESFSMCIyjIBlOouQtQMEjCiwj744AMTRVGk9yIqTAJGVFhKSoopIAEjKkwCRlRYQUGBqUajkYARFSYBIyosOztboyjKr9Vdh6g5JGBEhdna2pppNJpu1V2HqDkkYESFmZmZmSiKoqvuOkTNIQEjKqygoEADXKjuOkTNIQEjKqxevXpmgFN11yFqDgkYUWE6nU6uIolKkYARFabT6eR3MKJSJGBEhRUVFWl0Ot3F6q5D1BwSMKLCrKyszDUaTfPqrkPUHBIwosLkEElUlgSMqDBFUeQkr6gUCRhRYRqNRn5oJypFAkZUlra6CxA1hwSMqLDi4mIzjUbjVd11iJpDAkZUmEajMQHkEElUmASMqDAJGFFZEjCiwnQ6nUbuxysqQ1PdBYi7UxTlFWBFddfxkHpco9H8VN1FiPJJD0YIYTQSMEIIo5GAEUIYjQSMEMJoJGBqCJ1Ox6xZs3BycqJRo0bs2bOHoqIivLy8mD59ujqfr68vo0aNqtZa7+bEiRPk5eUZTFu2bBkajYZbt25VW13COCRgaoi1a9eycOFCpk+fzldffUXPnj3RaDTY29tjZWVV3eVVyMaNGwkMDJQgqUPMqrsAUTF79+6lX79+/Otf/zKYfvz48WqrqbJu77mI2k96MDWAmZkZP/74I/v370ej0bB8+XIuXLiARqNBo9Ewa9asuy6fk5PDv/71L5o1a4adnR1dunRh8+bNFV7/mTNn0Gg0jBs3jjZt2mBpaYmfnx/fffedOk9+fj7vvvsurVq1wtzcHFdXV2bNmkVx8f/u7rBx40amTp0KQJMmTdBoNGzcuNFgPVu3bsXb2xtbW1sGDx5McnKy+tyePXvw8/PDysoKHx8fli9fXuH6RfWRgKkB9Dtex44d2bZtG0OHDqVp06Zs27YNc3Pzuy6r0+kICgpix44dvPPOO6xatQp/f39GjRrF+vXrK1XHhQsXWL16NTt37sTT05PnnnuOLVu2QEkI/vzzzwQFBbFkyRIee+wxPvroI5YtWwbAkCFD1HNFu3bt4ujRowwZMsSg/Tlz5vD6668zZ84c/vjjD8aOHQvArVu3GDlyJJaWlqxdu5YnnniClJSUStUuhLgDRVFeURRF6dGjhzJo0CDldhYWFsq7776rPvbx8VGeffZZ9fH333+v1KtXT0lOTjZYbtSoUUr79u3LtHcnp0+fVgBl586d6rTi4mLF29tb6dSpkzpNp9MZLPfoo48q3bt3Vx+vXLlSAZT09HSD+ZYuXaoAyoULF9RpCxYsUADl6tWrSlxcnAIo8+fPv720QdX9/oi7k3MwtdyePXsoLCykVatWBtOLiopo0KDBfbdrYmLCgAEDWL58OQUFBdSrV48rV64wZ84cfv75Z65duwZAw4YNK9xmo0aN1H/7+voCkJSURIcOHejevTsfffQRNjY2vPTSS1hYWNx37eLBkYCp5VJTU2nevDmHDh0q81y9evX+Vtv29vbodDqys7O5fv06nTp1wsbGhg8//JDWrVvz7rvvEhsbe19tm5qaQkkQajQa9u7dyzvvvMP06dNZvHgxX3/9NY8++ujfql8YnwRMLWdvb096ejotW7bE0tKyStu+dOkS1tbW2NvbM2fOHNLS0jh27Biurq4AuLq63jFgFKXy/yHbzs6OFStWMH36dJ588kmefPJJEhISTKvkhQijkZO8tYyFhYV6eALw2GOPUVRUxMqVKw3my87O/lvruXHjBlu3bqV79+4AXL16lSZNmqjhop+m0/3/28dYW1sD3NcJWv0lbnd3d15//XVu3LhBZGSk9d96EcLopAdTy3Ts2JH169fzf//3fyxYsIAxY8awevVq3nzzTc6fP0+nTp04e/Ys27ZtIyoqivr161e47fnz55OSkkJWVhYrV64kMzOTuXPnAtC3b1+WL1/O7Nmz6dGjB//973/Zs2cPOp2OjIwMGjVqRPfu3TEzM+ONN95g4sSJ5Obm8vLLL99zvQUFBbRp04Znn30WHx8fPv/8cxo2bEj79u3lF3tC/B2VvYqUlpamPPXUU0rDhg2VGzduKIqiKDdv3lSmTJmiNG7cWLGwsFB8fX2VkJAQpaCgoFJXkQYPHqy0bNlSsbCwUDp37qzs2bPHYL7Zs2crDg4Oir29vTJ27FglNDRU8fHxUd5//311nvXr1yvNmzdXGjZsqPTr18/gKlJWVpY63969exVAOXnypHLt2jVlwoQJSosWLRQrKyslMDBQOXr0qFxFEuLv0gdMdbrTZeqHhATMQ07OwdRxvXr1omHDhuX+jRs3rrpLFDWYnIOp4zZv3kxBQUG5z1tbWxv8ZF+IypCAqeMcHR3vOU+TJk3u69KyEHKIJIQwGgkYIYTRSMAIIYxGAkYIYTQSMEIIo5GAEUIYjQSMEMJoJGCEEEYjASOEMBoJGCGE0UjAiArT6XTUr19fHS4lLS2tStodOXKk2ubtN8YSNZsETC3VunVrFi5cWGZ6eno6Li4u9xwX6fLlywwfPpykpCR1WlxcHHl5eezbt4/U1FSaNWtW7vK//fYbzzzzTIVqXbNmDceOHQPAx8enQsuImkH+s2MtlJWVxfnz5/H39y/znJ2dHcOGDcPLy+uubRw6dIgTJ07g4uKiTouIiECj0dCjRw9sbGzuuvyXX36JmdndP17FxcWYmpri4OBAeno6lBpNQAjxANzPDad+/fVXBVDS0tIMpl+6dEkBFEC5efOmopSMVeTm5qZYWVkpAwYMUAoLC5VNmzYpZmZmirm5uWJtba288cYbiqIoyty5c5WWLVsatHmn5adMmaJoNBrF0tJSsba2VrZs2aIoiqJMnz5d6d27t/LCCy8ozZs3V9avX6+2ExISorRo0UJuOCXEg3Q/AfPZZ5+Vu7OuXbtWDYnw8HAFUPbs2aOkpaUp27dvV+fr1q2b8vHHHxss+8wzzyhDhw5VH5e3fHZ2tmJmZqacOHHCYPnHH39cady4sXLmzBmluLhYyc3NVZ97/vnnlccee6yyL1UC5iEn52BqoTNnztCxY8c7PhcREYGfnx+UjDkEkJCQQJMmTXjyyScBKCws5PTp0wQGBpZZtvQhTHnLnzp1ClNTUzp06GCwfHh4OLNmzaJDhw6YmJgYDKMSGRkph0e1kARMLXTmzJk7nn+hZCfXB4y/vz/fffcdH330EZ07dyYxMVFdvrCwkICAAHW5wsJCzp07Z3AStrzljx8/TseOHQ0Gdrtx4wbJycn079+/TE06nY7o6GgJmFpIAqaWKSoqIjIystweTOmAARg1ahQxMTHk5OSwaNEiKAmINm3aqOMYAcTExFBYWFjmKk95y3fq1KnMes3NzWnTpk2ZmuLj48nLy5MrSLWQXEWqZaKiosjPz8fa2pro6Gh1eosWLcjNzSU9PV0NmJUrV9K3b19MTU3JycnBw8MDgCtXrpCens758+dRFIVWrVoRERGBiYkJ3t7eapt3Wz4rK4vLly+j0+lwcnIiIiKCtm3bYm5uXqZm/dUpCRghHrDKnuT9+uuv1StFpf9++uknZf/+/Uq9evWUwsJCJTs7W+nTp49iZWWltGjRQnnzzTeVoqIiRVEUJSIiQnF1dVXMzc2Vp59+WlEURZk1a5bi4eGhruduy3/77beKra2tUr9+feXTTz9VFEVRpk6dqowePfqONX/44YeKm5tbZU/wykneGkBT3QWIu1MU5RVgRXXXERwcjKIobN++vcrbHj16NJmZmezevbuyiz6u0Wh+qvKCRJWRczCiQiIiInB1dSU1NZXi4uIqaTMzM5PU1FTCw8PlBK8Q1eFhGNkxNzdXMTExUQ+3rl+/XiXtDh06VG3zq6++up8m5BDpIScnecU9WVpaVlmvpbRdu3ZVeZvi4SKHSEIIo5GAEUIYjRwiPfxuAderuwj+92teC0Bjbm6eV921lMit7gKEEFXkpZdemvbSSy8tru46RM0hh0hCCKORQyRRYUVFRZkmJiZlf+svRDkkYESFmZmZ2QGNq7sOUXPIIZIQwmgkYERl5CmKkl3dRYiaQwJGVIalRqOxrsB8QoAEjKikWzqd7qH4TY6oGSRgRGXYmJiY2Fd3EaLmkIARQhiNBIyoMI1GU6Aoivw8X1SYBIyoMEVR6mk0mvrVXYeoOeSHdqLCiouLb2k0mmvVXYeoOSRgRIWZmpraAA7VXYeoOSRgRIWV9GDkMrWoMAkYUWElPRi5TC0qTE7yisrQKYpS9TfnFbWWjIsk7urFF19MMjExceZ/V5Hgf5erKXmcvnbt2qbVW6F4mEkPRtyViYnJV4qi6CgJFn24lNhbbYWJGkECRtyVTqdbCSTcPl1RlIs6nW5R9VQlagoJGHFX69atu6Qoyi794ZGeoigHv/jii4hqK0zUCBIw4p5yc3M/AeL1j3U63aXCwsJPqrcqURNIwIh72rRp0yWdTrej1EneX7788svI6q5LPPwkYESFmJubf6IoSoKiKGkFBQULq7seUTNU6DL1qlWrHq9Xr94kzW2XEETdkpWV1UlRlPp2dna/V3ctonrl5+f/MGXKlM33mq+iv+T1tbe3D3ZzczP9+6WJWmBEdRcgqk9qaiqXL19OBqosYGjQoIGuZcuWEjBC1HEFBQVcvny5QvPKORghhNFIwAghjEYCRghhNBIwQgijkYARQhiNBIwQwmgkYIQQRiMBI4QwGgkYIYTRSMAIIYxGAkYIYTQSMEIIo5GAEUIYjQSMEMJoJGCEEEYjASOEMBoJGCGE0UjAPCROnTrFzJkz+e2336q7FCGqjASMkaWkpDB+/Hjc3d3x8PBgx44dpKWl0b59eyZNmqTO99VXX7Fq1SquXLlSrfXWBNevX2fXrl3VXYaogArfk1fcn3HjxhEaGkpAQACWlpYEBASQnJzMpUuXqru0Gik5OZnOnTvj6enJsGHDqrsccQ8SMEYUHx9PaGgoXbt2Zd++fep0JycnfvjhB1xcXKq1vpooPz+f/Pz86i5DVJAcIhnJsmXL6NKlCwAnTpzAwcGBDRs28Oeff+Lg4MDTTz/Niy++eNc2ioqKWLRoEf7+/jRv3pxOnTqxaNEiioqKKl2PVqtl/PjxeHp64uzsTL9+/dixY4f6fGxsLM899xxubm44OzsTFBTE8ePH1edXrlyJg4MDq1aton///ri4uNClSxfWrFmjzhMUFISDgwP/+c9/1GmKotCpUyccHBwIDw8HID09nddffx1PT09atGhB37592b59e5l1jR07lqCgIJydnfHy8iImJobOnTsDEBERgYODAw4ODiQnJ1d4e7Vs2RIHBwdCQkLw8/OjadOmLF68uNLbU1SMBIyRuLu7qztDkyZNGDhwIC4uLtjb2/Poo4/ec3lFUZg4cSIhISHk5ubSuXNnMjMzCQkJ4ZVXXqlULSdPnmTAgAHs2LGDBg0a4OPjQ2xsLGFhYQBcvHiRxx9/nH379uHu7o6vry+//fYbTz75JH/99ZdBWzNnzsTKyorhw4eTnp7O22+/zZYtWwDUwCwdMIcPH+bChQt069YNPz8/rl+/zuOPP84333xDgwYN6NixI9HR0UycOJGvvvrKYF27du3i6tWrBAcHM3bsWJo2bcrAgQPhf8PoEBwcTHBwMFZWVpXeXkuXLqV79+707NmT0aNHV2p7ioqTQyQjefLJJ2nUqBFBQUF07NjRYKcLCQmhR48ed11+79697Nq1i/bt27Nnzx6srKzIysqif//+bNmyhddeew0/P78K1TJt2jRyc3N58803eeedd6DkXIadnR0AH3/8MTdu3GDChAl88sn/xrRfsmQJ8+bNIyQkhB9++EFt6+mnn2b16tUADBs2jNGjR7Np0yZGjhzJ0KFDcXR05OjRo1y6dAlnZ2c1NF566SUAFi9ezPnz55kwYQKLFy9Go9EQFRVFnz59mDt3Ls8//7y6Ljc3Nw4ePEj9+vXVafPnz2f//v24uLjwxRdfqNP37NlTqe21cOFCJkyYUKHtJ+6fBMxDau/evQDY2NgQEhKiTtfvbH/99VeFAubSpUtERERga2vL9OnT1elOTk7qvw8fPgzA5MmT1WnPP/888+bN49ixYwbtubq6qv/29/cH4MKFCwCYmpoyfvx45s+fz+bNmxk3bhx79+7FyclJPSG7Z88eAG7dusV7772ntmVra8u1a9c4f/68Ou3xxx83CJe7qez2Cg4OrlC74u+RgHlIpaamAvDHH3/wxx9/lHne0tKyUu04OTlhbm5+x3kyMjIAaNasmTqtcePGAOTk5JR7UlVfQ2FhoTpt3LhxLF68mP/85z+Ym5tTUFDAiy++iKnp/wYFTUtLAzDoFZVWOlBsbGwq9Bq5j+1VmbbF/ZOAeUjpD1+WLFnC+PHj77udBg0aAHDlyhUURUGj0ZSZp1GjRqSkpJCeno69vT2U2mFtbGywsLCo8PqaNGlCUFAQW7ZsYcmSJdSvX5+xY8cavK68vDyOHz+Op6fnfb8unU5n8LiqtpeoWnKS9yGj7y3oz9GsXr2aq1evqs+XvrJTEa1bt6ZFixZcu3aNzz//XJ1+5coV9dCmZ8+eAHz55Zfq8/rzLL169ar0a9AfamVmZjJy5Eg1tAC6d+8OwKJFiygoKICSHtDtJ5PLY2trCyWHfrm5uVAyVnJVbS9RtSRgHhL6LvuBAwcAGDVqFG3atCEmJoZOnToxcOBAOnXqxODBg9WrPxVhYmKinuuYPXu2QVsfffQRlJwEtra2ZuXKlfTt25d+/fqxfPlyLC0tefvttyv9Wrp06UKHDh0AePnllw2emzFjBtbW1mzZsoUOHTrw+OOP4+Pjw/PPP09eXt49227SpAnu7u5kZmbStWtXevbsyebNm6tse4mqJQHzkBg+fDgNGzYkNTWVzMxMrKys2LVrF2PHjsXKyorTp0+Tk5PDU089pR72VNSzzz7L119/TefOnUlNTSU6OprWrVvTt29fADw9Pdm1axd9+/YlLi6O2NhYevXqxY4dOyp8pep2kyZNomfPnrRr185getu2bdmzZw8DBw4kJyeH06dPY2Njw9NPP13msKc8a9eupX379qSnp3P58mUaNWpUpdtLVJ2yB+R3sGrVquleXl7z/f3973yWUAhRZ8TGxhIZGfnZpEmT3rjXvHKSt4bKzs5m3Lhx95xvwoQJDB069IHUJMTtJGBqqKKiIg4dOnTP+fr37/9A6hHiTiRgaqgGDRpw7dq16i5DiLuSk7xCCKORgBFCGI0EjBDCaCRghBBGIwEjhDAaCRghhNFIwAghjEYCRghhNBIwQgijkYARQhiNBIyokLfffptnnnmmussQNUydDJgpU6bg7OyMu7s7ffr0YeXKlRW62ZHejBkz2LlzZ4XnHzBggDqGT+m/uLi4+3wFf192djaNGzcuU5P+/i1BQUHMnDlTnf/s2bPqTb7vZvPmzXd8rfq/+Pj4Oy4XFhaGl5cXLVq0wN/fnwkTJqjjKNV0qampjBkzRh2/qS6pk//Z8ddff2XKlCkMHTqUP/74gwULFnDw4EF1fJ+7SUlJYd26deowHPei0+mIioriww8/ZMCAAQbPtWrVyuBxcXGxenNsY9Nqteh0Onbv3k2jRo3U6fqbY/fr108dQUCn0xEZGck//vGPe7Y7cOBA/vzzTwBWrFjB8ePHDcY7uv01650+fRqNRsNPP/1Eeno6K1as4LHHHmPbtm3qbTZrqqNHj/LXX38ZjORQV9S5Hkx6ejqpqakMGjSIjh078uqrr/Lpp59y6NAhTp8+DUBiYiJjxozB1dUVNzc3BgwYQGxsLMnJyQQEBGBiYkLfvn3VO8JREloDBgzA0dERDw8PRo0aRWFhIefOnSMnJ4cnn3wSLy8vg7+cnBwaNWrEwoUL6dOnD127doWS3sXMmTNp06YNTk5OdOnShZ07d6IoCl26dOG9996jZ8+eODo60r9/fzZv3kyfPn1wdHTkySefJDs7G0ruVTtv3jz8/Pxo0aIF/fr1Q6vVAhAeHo6bmxvdunUzqMnV1ZWAgADmzp2LtbU1AOfOnePWrVsGPZiTJ08yfPhwnJyc8PDw4MMPPwTA3t5ebSs1NRV/f3/1sU6nY+TIkTg7O+Pt7W0womJ4eDh+fn60b99efU1t2rThs88+g5KQW7JkCe3bt8fZ2ZkhQ4YQGxurLr9q1SoCAgJo3rw5vXr1QlEUtm3bhrOzs8Gd8jp06MDKlSsJDw/Hx8eHhQsX4u/vj6OjIxMmTGDNmjW0b98eFxcXgx7c3bZlcHAwCxcuZPjw4Tg6OtK1a1f1uS1btvDqq6+SkZFRps26oM4FTFhYGCYmJga3cuzTpw+UBEt6ejpDhw6ldevWREVFceDAAUJDQzEzM8PJyYlXXnmF/v37k5SUxC+//ALAb7/9xpgxY5gyZQoXL15k9uzZREREYG5urnbzu3XrhouLCy4uLuo3fHR0NIqikJ6ezoEDB/j1119RFIVx48YRHR3N4cOHiY+Pp7CwkBs3bqDRaLh58yahoaF89913/PLLL2i1Wr755hu+/fZbtm3bxtGjRzly5AgA48ePZ8+ePXz11VfExsbSvHlz9cbfYWFhXLp0Sa2pVatW6hCr33//PYC6jcLCwmjcuLH6DXzixAmCgoLo2bMnERERbNq0iaVLl5KUlGSwrSMiItRbbl64cIEhQ4bw2GOPce7cOdasWcOCBQvUcZfCwsIMbs9pbm5Ojx49SExMBODdd9/lxx9/ZMuWLURHR9OwYUNmzJgBJYOxLV26lHnz5hETE8PatWvRaDRotVratWuHicn/PuaZmZkkJSXh6+uLubk5ly9f5vr16xw+fJglS5bw448/kpiYyNGjR5k2bRqrVq0iMzPzntvy5s2b7Nu3j3nz5nHmzBl0Oh0bN24EYOTIkXTs2JFZs2aRlJTE/Pnzq+yzXBPUuUOkiIgIWrdurX47A1y/fh1Khr4ICQnB2dmZOXPmQMmwqvb29mrX/tSpUwajMiqKwvTp05k8eTIjRoyAkp2pU6dOUPLN3LNnT5YtW6Yuox/0PioqisaNGxMSEoKZmRlmZmZs3bqVEydOcPbsWezt7cnOziY5OZlOnTqRm5vLtWvXmD17Ni4uLhQUFFBcXMy0adNwdHSkadOmmJiYUL9+fQ4dOsS+ffvYv38/HTt2JDExkfPnz6u1R0RE8I9//IMXXngBQF0/JcFna2ur1nnmzBn1Jt6U3Dy8V69eTJ8+ncLCQkJDQ7G3tzcYVykjI4PLly/j6+sLwLx58+jZsydTp06FktEKHB0diYyMJDAwkKioKKZMmWLwXl2/fh07OztiY2NZs2YNv/76K15eXgAMHTqUDz74gLS0ND799FO+/PJLHn/8cSg1VItWq8XHx0dtT9+r8PHx4fTp01hYWDBv3jyXBxpCAAAgAElEQVTMzMxwdHTE1NSU2bNnY2lpSadOnTAxMaFevXr33JYJCQmsXr1afa1ubm5qqBUWFhIeHs77779fyU9q7VDnejD6rnFp+/fvx9TUFH9/f7Zt26budJQESseOHaGkm37mzBkCAgLU5yMiIoiNjTUY++fUqVMGAdO+fXtatWql/ukHQIuKiqJ79+4GA6Jt3bqVYcOGqUN9nD59GktLS9q2bUtUVBQ6nU7tWcTExFBUVKS+nnPnzqHT6fD29ub333/HxsaGESNG0LJlS/r168fgwYOZOnUqxcXFaLVaAgMD1ZpKj9io1Wrx9vY22Gb6gMnPz+fUqVOcPXsWNzc33Nzc2LFjB1u2bKFevXrqMvo7+et7JQcOHKB3797q84qicO3aNRo3bkx8fDzZ2dkG70tBQQFHjhyhc+fOHDx4kEaNGhk8n5GRQePGjTl27BimpqbqmNWlabVadacHiIyMxNHREQcHB7RaLW3atFFDNTIyEg8PD/UcVExMDK1atcLS0vKu2zIpKYnMzEyD2uLi4tQgDA8Pp7Cw0CCg65I614MJDw9n1KhR6uMLFy6waNEinnvuOUxMTLh58ybu7u7q83v37lVPzsbExJCVlWXwYUlMTMTMzEz9ts/IyODEiRNMmzYNSgLoqaeeumMtWq2Wbt26GUxLTEzkiSeeMFi/r68vpqamaLVaXF1d1UHGIiMjadasGU2aNFEfN2rUiObNm0PJzv3jjz+Sk5OjLkNJDyUvL4+2bduWW1fpQ8iIiIgyg5mtXbsWf39/LC0tDYKl9DL6nVmn05GdnW3Qwzl48CDFxcU8+uijHD58mPr16+Ph4aE+P2fOHG7cuMGLL77I999/b7AswM6dOxkwYABZWVlYWlqqPQa97OxsLl68aBCUJ06cUAPn9vC5vbcTGRlp8Li8bRkZGYmdnR3Ozs5Q6jBMv/1CQ0Px8PAw6DHXJXWqB5OTk0N8fDzW1tacPn2azz//nAEDBtCuXTtCQkKws7PD1tZWvXz8+eefExYWpo5ZlJ6eDiWXbOPj41EUBScnJ4qKikhISKCwsFA9bLCxsSE5OZmrV6+WuyNHRUWVGdbDyclJXf/JkyfZsGGDuv577QSlH3fu3JnQ0FD279+PTqfj8OHD6qX4sLAwrKysDHotpZUOmMLCQjIzMykuLgbAwsICPz8/Vq5cSWZmJunp6Zw8ebJMG/qTtpSMzeTj48O2bdvIzc0lOjqad955h3/96184ODgQHh6Oi4sLsbGx/Pzzz4wZM4YNGzawevVq3N3dad++PefOnePUqVPk5eWxePFiLl26xD/+8Q86derEzZs3WbJkCenp6Rw+fJikpCQKCgpQFMVgqNpt27apoRIZGWmw7W/flqW3wd225e3tREREAKjBdvXqVTIyMkhMTFQHuqtL6lTAREZGotPpePvttxk+fDg7d+5k1qxZbNu2DSsrK0xMTFi6dCkff/wxXbp04cCBAwQGBhITEwMlA4oFBgYyevRotZehvxI1dOhQevXqhZWVFU2aNCEmJkb9sLVp06ZMLVevXiU9Pd3gG5aS8xtarRZ/f39mzJjBoEGD1KsltwfM7QFV+vnBgwfz6quv8n//93/4+voSEhKiDgEbHh6Op6dnmW99gLy8PBISEtR2zc3NeeONN5gxYwYpKSlQcvk5IyODwMBABg0adMcdp/QJXoB///vfJCYm4uHhwQsvvMDkyZPVk7Th4eHExsbSu3dvpk2bhoODA0eOHCEoKEh9LVOmTOG5557D29ubM2fOsHv3bpo2bYqPjw+LFi1i48aNtG/fntmzZ2NmZoa9vT0vvvgir732GgEBAURGRmJubo6Pjw9FRUWcO3dO3VY6nY6YmBj1saIoREdHV2hb3ilgnJyc1PNAwcHBWFpa0rVrV/W8Xl0i4yIJISqlMuMi1akejBDiwZKAEUIYjQSMEMJoJGCEEEYjASOEMBoJGCGE0UjACCGMRgJGCGE0EjBCCKORgBFCGI0EjBDCaCRghBBGIwEjhDAaCRghhNFIwAghjEYCRghhNBIwQgijkYARQhiNBIwQwmgkYIQQRiMBI4QwGgkYIYTRSMAIIYxGAkYIYTQSMEIIo5GAEUIYjQSMEMJoJGCEEEYjASOEMBoJGCGE0UjACCGMRgJGCGE0EjBCCKORgBFCGI0EjBDCaCRghBBGIwEjhDAaCRhRKdevX8fBwYHQ0NDqLkXUABIw1aCgoIAlS5bQpUsXWrRoQadOnVi4cCGFhYUVbmPGjBns3LnTYFpQUBAzZ840QsX/39mzZzEzM8PHx6fceR5EHQB//vknEyZMMPp6xP0zq+4C6pri4mLGjBlDREQEH3zwAW3btuWvv/7i3XffBeCtt966ZxspKSmsW7eOl156yWB6v379cHV1NVrtAGFhYbRt2xZLS8ty53kQdQB89913mJnJR/hhJj2YB2z16tX8+uuvbNu2jWeeeYb27dszfvx4Ro8ezY4dOwB48803mThxIs8++yxubm506dKF3bt3A5CcnExAQAAmJib07duXvn37AhAQEMDcuXOxtrZW17V792569eqFo6MjgYGB7Nu3D4Cff/6Z3r17s3z5cnx8fHBxcTHocSQmJjJmzBhcXV1xc3NjwIABxMbGAnDmzBn8/f3LfX231xESEsLUqVN57bXXcHNzw9PTky1btgCwefNm+vfvz7Rp0/D09MTb25uFCxeqbT3yyCN8+umn6uNvv/0Wd3d3AKZNm8Y333zD7t27cXFxUbfdhg0b6NChA87OzowYMYKioiIKCgro2rUrU6ZMqYJ3UFSGBMwDpCgKK1asYPTo0bRp08bgOUdHR65evQpAeno68fHxzJ49m9OnTxMYGMjUqVPJz8/HycmJV155hf79+5OUlMQvv/wCwPfffw9Au3btANixYwevvfYaH374IefPn+fZZ5/l5ZdfJicnB51Oh1arRVEUjh07xqxZs1i1ahUZGRmkp6czdOhQWrduTVRUFAcOHCA0NFTtKYSFhdGhQ4dyX+PtdeTk5HDw4EGGDBlCREQEjz76KEuWLAEgMzOT8+fPM3DgQEJDQ3nrrbdYuHAhx48fJz8/n/j4eHx9fdW2IyMj1UOzDz/8EFNTU3bt2kVSUhJBQUFERUUxbdo0PvnkE06fPs2kSZMwMzPDxMSEBg0aYGdnV4XvpqgICZgHKC4ujsuXL/PEE0+Uee7ixYs0a9YMSnop48ePx9fXFwcHB1588UVu3bpFcnIyAKdOnSIgIMBg+ejoaGxtbXFxcaG4uJiZM2fy1ltv0adPHywsLBgxYgRZWVlcvHiR+Ph4/Pz8eO2117Czs6Njx44AmJiYEBISgrOzM3PmzMHa2pqLFy9ib29Pq1at1EC4W8CUrgMgISGBZ599lsGDB2Nra4uPjw8mJibq6+zbty+DBg3Czs6O8ePHY2trS2xsLDExMRQXFxuc69FqtWrgnDlzBlNTU4MAKioqAuDChQs0btyYIUOGAGBmZsbPP//Mxx9/fN/vnbg/EjAPkL6H4uzsbDC9sLCQX375hR49eqAoCjExMQY71vXr1wGwt7dHp9Nx5syZMgGj1Wrx9vZW/52SkkLv3r3LrLtRo0ZotVqD9uPi4mjSpAkNGzZk27ZtvPDCC+pzp06dUgMoPDwcExOTu57gLV0Ht/U6AOLj4/Hy8gIgKirK4LmCggKys7NxcHBAq9XSuHFjmjdvbtC2fv7Q0FD8/PyoV6+e+ryfnx/r1q1jyZIl9OvXj6SkpHLrFA+GBMwDpO+hxMfHG0xfv349aWlpTJw4kcTERG7dukXbtm3V5/fu3Uvnzp2xt7cnJiaGrKysMr0IrVarHpZkZmYCGOycO3fupGPHjjRp0oTIyMgyIdCuXTtu3rzJzZs31fMc+nXrAyYiIoJWrVpRv379cl9j6TqysrJISkpSH+vb0D8uPa9+XZaWlvTo0YPo6GiDGhMSEkhPT1d7LKGhoXfsST311FMcP36c3Nxc/v3vf5dbp3gwJGAeIHd3dzp37sz777/PL7/8QmhoKHPmzGH27NnMnz8fT09PtFot1tbWpKWlcfnyZT777DO+/fZb5s2bByXnZyi5XBwfH4+iKHDbzurl5YWFhQWbN2+msLCQ/fv3s2HDBt577z2Ki4vL9JD0O72dnR22trbExcUB8PnnnxMWFoaNjQ0A165dg5IrYeUpXYdWq8XExEQ931RUVERMTAzt2rUjMzOT5ORkcnJyuH79Ojt37uTNN9/knXfeoWHDhuTl5XHt2jUKCwu5cuUKb7zxBqampmrwpqenc/78eVJTU0lJSYGSoI6NjeXKlSvk5OSoQbl+/XoeeeQRdT7x4EjAPEAajYYNGzbg6enJhAkTGDFiBGfPnmXz5s1MnDgRSnbKJk2aMHLkSAICAti7dy8//PADXbt2BaBLly4EBgYyevRo9VxOXl4eCQkJ6o7dpEkTVqxYwerVq3F3d2fx4sVs3LiR3r17ExcXR15enkHPQd+jMTExYenSpXz88cd06dKFAwcOEBgYSExMDAAjRozA3Ny83Evpt9cRGRmJh4cHFhYWAMTGxlJQUEC7du3QarXY2Njw2Wef0a5dO+bNm8e7777Lq6++CsCYMWPUntz48eNp2rQprVu3VntPEydO5OTJkwQEBLBz505ycnLYvn07/fr144knniA4OFi9jH/r1i0yMjIq9TsjUTU0FZlp1apV0728vOb7+/ubG7+kum3SpEm4ubnx3nvvVXcpRrVhwwa+++479u/fX92liEqKjY0lMjLys0mTJr1xr3mlB/OQ0Wq1eHp6VncZRqfVatWTvaL2koB5iBQUFBAfH19nAqYuvM66Tn5n/RCpV68eV65cqe4yHgj9L5NF7SY9GCGE0UjACCGMRgJGCGE0EjBCCKORgBFCGI0EjBDCaCRghBBGIwEjhDAaCRghhNFIwAghjEYCRghhNBIwQgijkYARQhiNBIwQwmgkYIQQRiMBI4QwGgkYIYTRSMAIIYxGAkYIYTQSMEIIo5GAEUIYjQSMEMJoJGCEEEYjASOEMBoJGCGE0UjACCGMRgKmFouNjUVRlOouw8CSJUtwcHDAwcGBadOmPZB1jhgxQl3ntm3bHsg6xf9IwDxkUlNTGTNmDMnJyX+rnd9//50FCxag0Wg4duwYDg4OXLp0yWAeHx8fVqxYcc+2rl69iq+vb5XsnFFRUfTp04eoqCjmzp2LoigEBATQokULvL29CQoKKjNu9cKFC+nTp0+F2p8xYwY7d+4ss84ZM2YQFRXFsGHDKr28uH91OmASExMZOXIkrq6uBAUFERERUd0lcfToUf766y+cnJzuu420tDT++c9/smDBAgDCw8NxcHDA2dlZnSc9PZ3Lly/j5+d3z/ZsbW0ZNGgQrVu3vu+a9KKiovDz86NZs2ZYW1sTHx/P+fPnWbduHRs2bMDb25sXXniBZcuWqct06NCBgQMH3rPtlJQU1q1bh7e3tzrtxo0bpKamEhAQQLNmzTA3N6/U8rcrLi6u1Out6+p0wPzjH//g0KFD3Lp1i99++42XXnrpvtt55ZVXCAoKwtXVlfbt2/P1118zatQoHB0d6dq1K1qtVp3/P//5Dz169KBFixb4+/uzfft2ALZs2cKrr75KRkYGLi4uzJw5E4Ds7GxmzpxJmzZtcHJyokuXLnf9lp0/fz4DBgygadOmUBIwHTp0MJgnLCwMAD8/P7WH8uWXX9KtWzccHR0JCgoiOzublJQUWrRowYYNG2jZsiWU7LSvv/46rVu3xt3dncmTJ3Pr1i0AQkJCmDp1Kq+99hpubm54enqyZcsWAIqKioiLi6Nt27ZqHREREVhZWTF48GAeeeQRFi5cyMSJE1m2bBnFxcW8/vrrPPfcc+Tm5qrLJCQk8Pzzz+Pq6oqbmxubN28mOTmZgIAATExM6Nu3L3379oWSQAMM1vnHH3/Qt29fnJyc6NmzJ+fOnSt3+f379+Pq6srixYvp0qULr7/++n19RuqqOh0w4eHhBo+jo6O5efNmpdvJzs7myJEjzJo1i9OnT+Pg4MCCBQt46623CA0NJTc3l02bNgGwYsUK3nrrLd555x1iY2MZP348H3zwAQAjR46kY8eOzJo1i6SkJObPn4+iKIwbN47o6GgOHz5MfHw8hYWF3Lhx4461pKWlsXnzZl544QV1WlhYGO3btzeYLywsDGdnZ+zt7bG0tCQlJYVjx46xdetW9u7dy++//86BAwdwdHTk008/xdXVFTs7OwoKChgxYgT5+fmcOnWKo0ePcuLECVauXAlATk4OBw8eZMiQIURERPDoo4+yZMkSAOLj4ykoKDDY2cPCwmjXrh0mJv//o9inTx8yMzO5fv06y5Ytw8nJiXbt2kHJIeSQIUNo2LAhR44cITQ0lF69euHk5MQrr7xC//79SUpK4pdffoGSgLGxsVF7b4qiMH78eAYNGkR4eDjTp0+nVatWd10+JycHV1dXTp48yccff1zpz0ddZlbdBVQnb29vjh8/rj52d3enQYMGlW4nOTmZCRMm0LVrVwCsra0ZMGAAHTt2BMDV1ZX69etz8+ZNQkJCePPNNxk2bBiZmZlERESoXfLCwkLCw8N5//331ba3bdvGiRMnOHv2LPb29mRnZ5OcnEynTp3uWMv27duxsbFR2ywoKCAmJoZ//etfBvOVDp2EhAQ0Gg2LFi3C1taW5s2bY25uru70UVFR6g7+7bffkpyczN69e6lXrx729vb06NFD7aElJCTw7LPPMnjwYCg5zxMTE6O2o9FoaNOmjVpHeHg4vr6+BrVdv34djUaDra0tWVlZJCcnq+tftmwZjo6OLF++HI1GY7DcqVOn6NGjh8G0qKgo2rZtq86rKArFxcUkJiZiaWnJ8OHD77q8Vqtl8ODBPPPMM+p7KyquTvdgPvjgA7p3746NjQ1du3blww8/rHQbiqIQHR2Nj4+POk2r1Ro8jo2Nxdvbm9DQUHJycli5ciXu7u54e3uj0+n497//DSU7W2FhocHhzNatWxk2bBj29vYAnD59GktLS4NeQGm//fabGmyU9MoKCgru2IPRn3+JjIzE1dUVW1tbAJKSkigoKMDLy0t9Pfod/MCBA3Tr1o169eqpbV29epXGjRurbZV+7fHx8Wo7MTExuLm5YWVlpT4fERFRJmB+/vln/Pz8sLCwQKvVYmpqqobSkSNHeOKJJ8qEi06n48yZMwQEBBhMj46ONthWJiYmbN26lZiYGAICAtSeSnnLR0VF8eijj95xW4t7q9MBExgYyK5du7h48SL79u1jyJAhlW4jMTGRW7duqTvJpUuXuHnzprqTXb58mYyMDPWxRqMhLCyM06dPk5SUxPr169WdMzQ0FA8PD4NvycTERPXcB8DevXvx9fXF1NT0jvXEx8fj6upq8BhQ10FJ4J0/f17dcSIjI9UAoWSnt7CwUE/qlg6YzMxMmjdvrs577do1fv/9dwYMGEBWVhZJSUll2tI/1vcm9K5cuUJaWprBiebdu3eze/duXn31VbW2Vq1aYWlpCUBWVhb169cv87pjYmLIysoqc67p9oCh5KTxgQMH6N27N++++265yxcVFXHu3DmD1yMqp04HTFWIjIzEzs4OFxcX9XH9+vUNdk4LCws8PDzUb+WlS5ei0+mIjo4mISFBbevq1atkZGSQmJjIhQsXAHByciIuLg6AkydPsmHDBmxsbMqtp7i42OAwz83NDfhfby0sLIw9e/Ywbtw4evXqxSOPPKLWWLrXERERgZeXF2ZmZqSlpXH16lV1J/Pz8+PQoUOkpqaSkZHB66+/TufOnRkwYABarRYTExO1t1FUVERMTEy5AVP6HNixY8d49913mThxIq+++ipPP/20WlvpHTwgIID169cTFRVFYmIie/fuhZKrYgBnz54lPj4eRVFIS0sjIyPD4JBMf34pIyODGzdu0KpVq3KXj4+PJz8/3+Cq0gcffEBgYCCFhYX3+GQIJGD+vtt3AK1WS9u2bdXzF1qtVt1ZmzRpwueff84PP/yAj48PEydOpKCgQF02ODgYS0tLunbtypw5cwCYPXs2Wq0Wf39/ZsyYwaBBg4iNjS23nhYtWhh8w3fq1IlPPvmEI0eO8PjjjzNr1iwGDx7Mpk2b1MOM23swpR9rtVrq1auHh4cHANOnT8fLy4vAwEB69uyJs7Oz2lZkZCQeHh5YWFhASU+poKCAdu3akZ+fT0JCgsHOqg+YwYMHM27cOM6fP89//vMf5s6da7A9Sy8zf/58XF1dGThwII899hjR0dEAdOnShcDAQEaPHs0TTzwBJb0XSs616f3666+MGTOGLl26YGlpySeffFLu8lFRUTRv3hwHBwd1eX9/fxISEuRydQVpKjAPq1atmu7l5TXf39+//B8RiIfC0qVLuXLlCiEhIdVdigH9FaUjR45U6Lc3VWHVqlUsWLBA7Q1Whd27d/PTTz/x2WefVVmbNU1sbCyRkZGfTZo06Y17zSs9mFpm7NixBr+5eVjoryA1aNCAzMzMB7bOVq1akZaWZvA7mvsVFxfH0aNHWbRoUZXUVxdIwNQyjRo14o037vnF8sBFR0ejKAr+/v4sXLjwga3zzJkzeHt7q1eL/g4PDw8WLFigHgKKe6vTv4Oprfr161fdJZQxe/ZsZs+e/UDX+dNPPz3Q9YmypAcjhDAaCRghhNFIwAghjEYCRghhNBIwQgijkYARQhiNBIwQwmgkYIQQRiMBI4QwGgkYIYTRSMBUsfz8fMaOHVtl7Xl4eKhj+uhvPVlddDodjo6Oaj1Xrlwx+jpPnDihrk9//5oHJTo6mvfee69K2oqKilJfh/7+NGvXrlWnTZw4sdxpVfm/wR80CZgqtnr1avbv319lNyT6/fffmTdvHvXq1avwsCHJyck4ODhw6tSpMs8dPHgQX19fzp8/f9c2/vzzTyZMmGAwLSEhgby8PH744Qeio6Np2rQp69evx93dHWdnZx555BFmzpxJWlqaukxlxlS60zqjoqKwtrYmKiqKffv23XX5r776Sh2qpSq89957nDlzpkraat26NVFRUQQFBak33Ro9erR6z5m7TQsNDTUYxqUmkYCpQhkZGSxZskS90XZVaNasGSkpKXh4eGBmVrH/mxoeHo6Jickdb/Xo5ubGwIEDDW6idCffffddmfXpb7kQGBioDoly7NgxAgIC2LNnD9OnT2f//v3069ePa9euQSXHVCpvnW3atKFZs2Y0bNjwrst/9tlntGjR4q7z6HS6e9YB8Msvv3DgwIEqGyurXr16NGvWjMTERPUGWDY2NlhZWZGamnrXaY899hiffvppjbyLngRMFVq4cCEeHh64ubkZ3A7y559/pnfv3ixfvhwfHx+DMY86d+6sDutByT1n3dzc2LhxozotOjr6roOB3S48PBxPT0+Dm2sDfP/993Tt2pUzZ86ot9XcsGEDHTp0wNnZmREjRlBUVMS0adP45ptv2L17Ny4uLuzYsUOtw8XFxeCWnREREfTs2ZP27dszcuRIfvzxR65du8a33357xzGVbt26xYwZM2jTpg2Ojo7q8LF3W2fp22wWFhYyY8YMPDw8cHd3V+9+161bNxISEpg1axaurq7qLTD79+/PG2+8QXBwMG5ubiQlJd1z++l0Ot577z2Cg4O5ceOGwTLBwcEsXLiQ4cOHG4x59cMPP9CiRQuDAFu4cCG+vr4UFRWp7epvAF/6veW2u+7dPq1BgwYUFhYSGhp6z9ofNhIwVeTcuXNs3LiR999/n9atWxt88+l0OrRaLYqicOzYMWbNmsWqVavIyMjA29tbvecuwKZNmzA1NVWHyaCcG1ffzZ3u1A/w9NNP88wzzxjcI3fatGl88sknnD59mkmTJmFmZsaHH36Iqakpu3btIikpiaCgIHX+0nXk5uYSFxdncIc6JycnPDw8SExMLDOmkk6n47nnniM0NJTNmzcTHR3NK6+8AlDhdW7cuJE9e/awd+9efv75Z3XEx48++ggbGxsSExO5ePEiTZo0QafTERMTQ3R0NF988QVarVa9R/HdbNq0iUuXLrFo0SLq169v8GVx8+ZN9u3bx7x58zhz5gw6nY6NGzfi7e1Nfn6+GkYFBQVs2LCBF198Ue2VXbhwgdzcXIPXEx0djaWlJe7u7nedph9qpaaRgKki77//Pr1796ZXr154eHgYfCjj4+Px8/Pjtddew87OTh1WxMTEhLZt26oBo9PpWLt2LWPHjlV7H5mZmaSkpFS6B3P7MCWUjGhQ+h7C+m/WCxcu0LhxY3VUhTNnzmBqalompKKiogzqiIyMpLi4+I7jGtnZ2anL6Ne3e/duTp8+zbfffou/vz92dnbqodOd1pmRkUF6errBOouLi8nKyuL69et4eHioJ35PnTpFx44dDQZwu3DhAjk5OXzyySc4ODhUaEyj7OxsQkJC+Oc//4mDgwPu7u4G72VCQgJvv/02vr6+NG3aFDc3N0xMTPD09MTU1FR9L7du3UpWVpbBCf/yeiv6ZcublpmZSX5+vsH9m2sKCZgq8Ntvv7F//351wLTbezC337U/Li6OJk2aYG9vT9u2bTl37hyUDFN68eJFJk+erM57pw/l3WRlZZGYmHjHHox+GA59W35+fqxbt44lS5bQr18/9ds3NDQUPz8/g7GPCgsLSUhIKDPsa5MmTWjWrJk6LTIykpSUFDp37qy+dn3AHDlyhMDAQIP59e60zjsN+zp58mRefvllgoODmTx5Mnl5eerydxrTyMHB4Y7bojzLly/HxMREHUbYw8NDfS+TkpLIzMws8156eXlhYWGBu7u7+l6uXr2aZ555xuBcV3R0NI6Ojmr4Us7h7+3T9FeR7nV+6WEkAfM3KYrC7NmzMTU1ZdiwYbRs2ZK5c+caHLtHRkaW+ebX73Rt27bl5s2bpKens3r1aoYNG2Yw8H10dDRWVlYV6tpTstMrinLHG0Vb7ZIAACAASURBVGvHxcWRn59vcPL3qaee4vjx4+Tm5qoDwIWGhpYZXyguLo7CwsIyw76WXk9ubi7Tpk3Dy8uLAQMGwG0Bc+vWrTuOaVTeOqOiorC1tTXYHqampsycOZNDhw7x448/qnet++uvv8r02m4f8eFeLl++zPLly7l+/Tre3t60bNmSn376Se3B6Ieo0Q9Dm5mZaTAOlL43+ueff3L27Flefvllg/bvdKhbkWnHjx/H1NS03NE8H2YSMH/T5s2biYmJ4a+//uLChQtcuHCB33//HUoOVYqLi4mJiSkz7pD+Q+nl5UX9+vVZtmwZR44cYcqUKQbtR0VF4eXlpXb99+zZg6enp/rtfrvw8HAaNWrEtWvXiI2NVf8KCwvRarU0atRI7UGsX7+e2NhYrly5Qk5OjnrMn56ezvnz50lNTSUlJUWtw8TERB2lUf86GjduTGRkJD/88AMDBgwgOTmZr7/++o5jKnXq1ImDBw9y4MABUlNT2bJli9pWeessPaZRfn4+K1as4PLly6SkpKDT6XB3d6eoqIjr16+j1Wq5fPmyOr546cMzgJSUFNq0acM333xzx203f/58WrZsSVJSkvpefvLJJ1y8eJGbN2/ecYA6SvUu27dvz9GjR1myZAl9+vQpExy3H2LeuHGDy5cv33PasWPH6NOnzz2v/D2MJGD+htzcXObNm8fUqVMNvmWdnZ2xsLAgPDycuLg48vLyyow7pP8AmZubM378eFauXIm/vz+BgYEG67i9u+zr60tubq7Bb01Ki4iIICMjg0ceeUT96969O0VFRQbf6Dk5OWzfvp1+/frxxBNPEBwcrB4WTJw4kZMnTxIQEMDOnTvVOlq2bKn2QPQnrr///nsGDBjAsmXLGDJkCEePHsXT0xPuMKbShAkTGDVqFFOmTKFr165s375drbu8dZZ+7efOneObb74hICCAt956i+XLl9O+fXvMzMyYPHkyn332Gd26dVNHs7x9h27UqBEtWrS448nSyMhIvvvuO+bMmWNwHkd/jigiIuKOAePk5KRekRs5ciQZGRkcOHCgzBdFcXExcXFxlb6ClJuby8GDB3n99dfv+H4/7GRcpIdARkYGvr6+fPrppwZXjyj5oL3yyiu89tprUPItPGbMGH766SfMzR/c2/HCCy+gKEq53/7G0KpVK958802mTp1aJe0VFhYSHBzM0qVL1RCsarNmzWLfvn2cPHnSYPzsc+fOERgYyIEDB9RDnY0bN/Lee++RmJioznv7tC+++ILff/+d9evXG6Xe+1GZcZFkVIFqdO3aNX799Vc+//xzWrZsSXBwsPpcRkYG169fJy0tTf02Ky4u5qOPPuLrr79+oOFCSW/gscceIy0tjcaNG5c7NnZVSUlJ4caNGzRr1oyrV68ajK19vxYvXszMmTONEi4nT57kyJEjrFmzhhUrVqiBodPpSE9PVwOnTZs25OXl8f/Yu/O4KOr/D+CvXW65CZBLuQXkUEAl8wq80vIKyyxIRNPMPPri8TVTM2+/qWUeaeaVmoileX9VMrUyDxRxdznkEBAFQYhDbvb9++MH82VFDY9hBd/Px8NH7czszHtmdl77mZllPoWFhYiNjYWbmxskEskDh6Wnp+PAgQP44Ycfnnm9TYUDRo0uXbqEf/3rXwgICMCmTZuE0CAi+Pn5obi4GKjXXNbQ0MDatWubvM7y8nLcuHEDGzduxMaNG5GWlqbS/7UY6k4Vxo4dC29vb5w+ffqp5zlr1qxnUNmDzZ07F3l5eViwYIHQrzZq+7ru3bs3AMDBwQH6+vpYs2aN8DdOISEhAIBNmzY1GCaTybBt2zaVu07NDZ8iMcYeC3cdyxh7LnDAMMZEwwHDGBMNBwxjTDQcMIwx0XDAMMZEwwHDGBMNBwxjTDQcMIwx0XDAMMZEwwHDGBMNBwxjTDQcMIwx0XDAMMZEwwHDGBMNBwxjTDQcMIwx0XDAMMZEwwHDGBMNBwxjTDQcMIwx0XDAMMZEwwHDGBMNBwxjTDQcMIwx0XDAMMZEwwHDGBMNBwxjTDQcMIwx0XDAMMZEwwHDGBMNBwxjTDQcMIwx0Wg2dsLs7GzlX3/9VSJuOew5J6n9L6m5DqZGxcXF2kTUqM9AowJGKpUeLSwszCssLHzq4ljzlZeX91pNTY1J69atd6u7FqZeEolEpu4aWAszbty4iHHjxn2p7jpY88HXYBhjouGAYYyJhgOGMSYaDhjGmGg4YBhjouGAYYyJhgOGNZpSqawhomp118Gaj0b/kpcxqVSqwZ8Z9ji4BcMYEw1/G7FGI6IiItJSdx2s+eCAYY0mkUiMJBKJubrrYM0HnyIxxkTDAcMYEw2fIrFGI6IiiUTC12BYo3ELhjWaRCIxAsDXYFijccAwxkTDAcMYEw0HDGNMNBww7HEoiahG3UWw5oMDhj0OqUQi0VB3Eaz54IBhjImGA4YxJhoOGNZoNTU1FQBK1V0Haz4kjZimWSCipQBmqrsOxp6BjhKJ5Kq6i3gWuAXDGBMNBwxjTDQcMIwx0XDAMMZE0+IDZtOmTZBIJMjOzlZ3KU0mPT0dN27cUHcZjxQSEgJ3d3d1l9GkLly4gPLycnWX0aRafMC8aFJSUuDk5IRLly6puxRWz9atWxEQEICSkhJ1l9KkOGBamKqqKiiVSnWXwe7zorVcWhwiWkpEdOXKFerRowfp6uqSo6MjvfbaawSAbt++TXW2b99OHh4epK2tTfb29rRw4UKqqakRxpeWltKsWbPI0dGRtLW1ydXVlRYsWEDV1dV04sQJAkDnzp2j+vT19WnmzJlERLRq1Srq3r07bdy4kWxtbUlHR4e6dOlCJ06coLCwMDIxMSFLS0uKiIig6upqYR737t2jqVOnkqWlJRkaGlKnTp1o9+7dwvhVq1ZRQEAARUZGkrOzM7Vq1Yq6d+9O8fHxRESUlpZGAFT+jRo1ioiIEhMTKSgoiPT19cnOzo7Gjx+vss6NUVRURJs3b6aqqqrHeh8R0e7du8nd3Z309PSoU6dO5OvrS25ubsL4yspKmjVrFtnY2JC2tjZ16NCB9u3bpzKP9PR0CgkJIQsLC2GbRkZGEhHR7NmzSUdHR2X6ixcvEgA6evQoERENGTKEZs6cSZMmTSIjIyMyNDSkoUOH0m+//UZ9+vQhPT09cnR0pB9++EFlPmlpaTRs2DAyMDAgCwsL6t+/P128eFEYP2TIEJoxYwbNnj2bLC0tycTEhN59910qLCwkIqItW7Y02C9btmwhIqLDhw+Tl5cX6enpUfv27embb74hIuqg7uOJ3YeIliYkJJCRkRE5OTnRqlWraM2aNWRpaakSMFu3biUA9N5779FPP/1E06dPJ4lEQgsXLiQiourqagoKCiItLS2aNm0a/fDDDzRr1iwKCQkhImp0wACgwMBA+vPPP2n37t1kaGhIAGj8+PF06dIlmj9/PgGgTZs2ERFRTU0N9e7dW6h9586dNHbsWAJA33//vcp8O3fuTL/++isdP36cHBwcqEuXLkREVF5eTjt37iQAtGDBAjp79iwlJSUREVH37t3JwsKCNm7cSCtWrKA33njjsUOiqKiIrK2tydnZmbZu3aoSjo+ya9cuYXts2rSJZs+eTRoaGioBExYWRpqamjRv3jyKioqiIUOGkEQioTNnzhAR0a1bt8ja2posLS1p+fLltHXrVhozZgytXLmS6DECBgBNmjSJYmJiaMGCBQSAtLS0aM2aNXThwgV68803SUNDgxISEoiI6Pbt22RtbU09e/ak77//njZv3kyvvvoq6erqkkwmE+aroaFB7777Ll24cIG2bt1K2traNH36dCIiysnJoWnTphEAOnToEJ09e5ZycnKouLhYCNwdO3bQzJkzadasWcQB8xwioqWDBg0iU1NTunPnjvAhW79+vRAwSqWSbG1tqUePHiofxDFjxpChoSEVFxfT7t27VQ7q+z1OwNSvY9SoUWRhYUFKpVIY5uTkRO+88w4REe3Zs4e0tbUpKytLZb7vvPMO+fj4qMw3OztbGL9y5UoCQHfv3iUiovj4eAJAUVFRKvOxs7Ojfv36PXCdHkd5eTlt2LCBXF1dycXFhbZt2/bIoCkrKyMLCwvq2bOnynQjRowQAiYhIYEA0Jw5c4TxSqWSXFxcKDAwkIiIPvzwQ9LU1BQO/Ps1NmDat2+vMo29vT0NHz5ceJ2RkUEA6NtvvyUioo8++og6duyo0mqrrKyktm3b0uTJk4X5enh4qOzbwYMHk7e3t/C67nOYm5srDEtOTiYAtHjx4vtXp8UETIu5BnPv3j3psWPHEBoaCgsLC2G4pub/nmt+/fp1ZGVl4c0331R5b//+/VFcXIykpCQcPXoUenp6GDVq1FPXpKenJ/y/rq4utLW1IZH8768z7OzskJeXBwA4cuQIqqqq4OTkBF1dXeFfVFQUbt68qTJffX194f/t7e0BALdu3XpkLaGhoTh+/DgmTZqEO3fu/GPt2dnZuHHjBm7cuIG7d+8Kw3V0dDBu3DgkJCRg4cKF+OSTT/D+++8/dD6///47cnNzMXXqVGho/O9JD/X3y+nTpwFAZb9IJBL069dPuFh99OhRBAUFwc3N7R9rf5T6+wS1+0VHR0d4bWdnBwAq++XatWswMDAQ9omhoSEyMzNV9kurVq1U9q29vf0/7hMnJye88sorWLRoEb755htUVFQ81bo9j1pMwGRlZelWVVXB0dHxodMUFhYCACwtLVWGm5mZ1c0DOTk5sLGxUTkYxCKRSEBEQO0BbWVlhdjYWJV/MpkMFy9efOg8tLW1AQDV1Y/uk37RokVYtWoVIiMj4eTkhLVr1z5y+nfeeQeOjo5wdHTE7NmzVcbV1NQgMjISixYtgrm5Od55552HzicjIwMAnni/FBcXo7i4GDk5OWjTps0ja34W6kKi/n554403GuwXhULxyG2ora39j/tEIpHg6NGjGDVqFKZNm4Z27drhzJkzz3iN1KvFdFvSunXrCgCP/Hau+3aq/40MADk5OQAAU1NTmJiYCK8fpP631LNkamqK3NxcODg4QFdX96nnV3eA1JFIJJg6dSrGjBmD8ePH4+OPP0bHjh3RrVu3B75/4cKFwrd4XTgolUps2rQJy5cvBxFhzpw5CA0NfWQY17UmG7tfbGxshOE5OTnQ1tZGq1atYGJi8sjfMom5X/Ly8p7Zb3bu3y9GRkZYu3Ytpk2bhiFDhmDIkCG4fv16i3moV4tpwRgbG9e4uroiKioKlZWVD5zG2toaDg4OOHLkiMrwvXv3Ql9fH76+vggKCkJJSQl2796tMk3dt1Hdt2z95m92dvZTN2/79OmD6upqrF+/XmX4vXv3Hms+dadP9zfP626TGhoa4osvvgAAXL58+aHz6d69O4YOHYqhQ4eiQ4f/vySQn5+PRYsWYcaMGUhISEBYWNg/tvQ6duwIDQ0N7Ny586HTdOnSBVKpVGW/VFRU4NChQ+jatSs0NDQQFBSE6OjoBj8grL9fKioqkJ+fL4x7Fj827NOnD/7880/ExMSoDH/W+8XR0RGTJ0/G33//jdjY2FZPXfhzosW0YABg3rx5CAkJwSuvvILRo0dDQ0MDX331lco08+fPx6hRozB27Fj0798f0dHR2LdvHz7//HPo6+sjNDQUa9euxahRo3DhwgV06NAB165dw8mTJ3H58mW4u7vD3t4eCxcuROvWrVFcXIxPP/30qX97EhISgg0bNmD69OlIS0uDn58frl69in379iE+Pr7BtYOHsbOzg5OTE1asWAF9fX3k5+dj8uTJeOutt2BkZIR+/foJB7K/v/9j1Whqaorr168Lp2WN0aZNG4wePRqbNm1CWVkZBgwYgNu3b+PQoUOwsrICADg7O2PUqFGYN28eqqur4ezsjI0bNyInJwc7duwAAMydOxcHDx5E165dMXnyZFhZWeH48eMwMDDAd999hz59+kAqlWLKlCmYOnUqZDIZZs58+qd3fP755zh8+DD69euHiIgIWFpa4tixY6iursb+/fsbPZ9XXnkFmpqamDJlCsLDw1FWVobRo0fDzc0NI0aMgKenJ9atWwcTExN07NiRn7nzvKn7HcyaNWvIwcFBuP03atSoBr+DWb9+Pbm6upKWlhY5ODjQsmXLVO4A3L17l8aOHUvm5ubUqlUr8vT0pAULFlB5eblwd6JLly6kp6dHPj4+tH///gfeRSouLhbmOX78eLK1tVW5VdCrVy/q3bu38LqwsJA+/PBDMjc3Jx0dHfLy8qIlS5ZQZWXlQ+d78OBBAkBXrlwRhl24cIE8PT1JX1+f3N3d6caNG/TFF1+Qi4sL6erqkouLC23cuPH+OxeiKSsro48//pjMzMzI2NiY3njjDerVq5fKbeqysjKaOnUqtW7dmrS1tcnX15cOHTqkMp+4uDh67bXXSF9fn0xNTalr1660a9cuYfz27dvJ2dmZ9PT0qF+/fsJPEurfRfL391eZp5ubG7333nsqw+pu89eJj4+n119/nVq1akUGBgbUs2dPlbt0D5pvREQEGRsbqwzbvHkzWVlZkYmJCQUFBVF+fj6NHj2arK2tqVWrVhQQEEBnz56llnQXiR84xdjzhx84xRhj/4QDhjEmGg4YxphoOGAYY6LhgGGMiYYDhjEmGg4YxphoOGAYY6LhgGGMiYYDhjEmGg4YxphoOGAYY6LhgGGMiYYDpoXq168fJBIJJBIJIiMj1V2OWpibmwvbID4+Xt3lvJA4YNRszJgxGDBgwAPHBQYGYurUqf84j48//hg///yzyjCZTIa5c+fi9u3bePPNN0FEcHFxga6uLqytrREYGPhYD0x6nt2+fRtDhw5FZmamynCZTIYVK1ZAW1sbrq6uaqvvRcYBo2axsbHo2LHjA8f1798fL7/88iPfn5WVhbVr18LLy0sYVlBQgNu3byMgIABWVlbQ0tLC9evXkZKSgt27dyMqKgpeXl4YNmwYli5d+szXqan9+uuvuHDhQoOHgltZWSErKwtubm4qvRgw9tjqnmjXnFRVVZGOjo7QO2F9zs7OBIAOHjxIRESnT58mPz8/0tPTI29vb0pISKCMjAzS0dEhqVRK+vr65OfnR0REZ86cIQB048YNYX6RkZHUqlUrld4cJ0yYQMbGxkJ/Rfv27SMfHx/S1dUlNzc3OnDggDDt8ePHqXPnzqSjo0NWVlakUCgoJyeHANBff/0lTDdq1CgaMmQIKZVKcnV1pWnTppG3tzfp6upSp06daPv27eTr60u6uroUGBhIJSUlwnu3bdtGXl5epKOjQw4ODsJ2mTNnDoWGhtLo0aPJyMiIzM3NaefOnUREtHPnTtLU1CQtLS3S19enKVOmqGzH/v3708iRI5/B3mpSLeaJdi1GcwyYuLg4AkCJiYkNxiUlJQkhoVQqycLCgubOnUt5eXkUGRkphMKsWbNo4MCBKu9dt24dGRgYqDwG9N///jcFBASoTPfzzz8LHcTt3buXTE1N6cSJE1ReXk6LFi0iQ0NDunfvHv33v/8lXV1dWrFiBeXk5FBSUhKVlpbSiRMnSCqVqoSEr68vzZ07l4iILCwsqEePHnTjxg1SKBSko6NDvXr1ops3b9Lvv/9OAGj//v1ERPTll1+SoaEh/fzzz1RUVERLliwhBwcHotrHT1pYWNAvv/xCRUVFNGLECPL09BSW2bVrV1q+fPkDt7Gtra3Qa2czwgHzvGmOAbN9+3YyMDB4YB/R+/btIyMjI6LabmXNzMwoNDSU7t27pzJdYGAgzZ8/X2XYRx991CBMXnvtNRo/frzKsE2bNpFEIqF79+6RnZ0dff3118K41NRUAkAymYy8vb3pX//6V4MaV65cSe3atRNeV1dXk66uLv30009UWlpKUqm07hmzVFFRQZqamnTixAmi2tabVCql48ePU0FBAbVq1YqWLv3/Xfj333/TO++8I3RvO3jwYIqIiBCWs2jRIqHXxMrKStLV1aXTp083qO/vv/9WCbFmpMUEDF+DUaPY2Fj4+PhAKm24G65duyZcV5FKpThx4gQUCgWcnZ1x4sQJoLafokuXLiEgIEDlvXK5HJ6eng2WVdf9SJ3Dhw+jY8eOSEpKws2bN9GnTx9hXG5urkotw4cPf2CN9eeZmJiI8vJydOjQATKZDEqlEt7e3gAAhUKB6upq+Pj4AAASEhKgVCrh5eWFCxcuoLS0FKtWrYKpqSmsra2hVCqxefNmAEBcXJzwPgBISkqCh4eHsF5VVVUP7CFBLpcDgMr1Kda0OGDUKDY2Fr6+vg8cFxcXJxycAODn54cLFy6gT58++OSTT4Dag7a4uBh+fn4q770/YHJycpCdna1yMXn//v3Yv38/IiIihJ4Vra2thfE//fQTOnXqJHQC16pVw6565HK5ysH7559/wsDAAE5OTrh27RocHBxgbGwsrI+VlZXQr1RcXBzMzc2FZUokEmRkZCA1NRUlJSWIjIyEhYUFiouLkZ6errItrl69Krw+f/483NzcVLrTrV9fq1atHtmrJBMXB4waXb16FRYWFkhISBD+3b59G6htHdQdRAcOHMDp06eRl5eHgoICuLi4APV6S4yJicH169dBRMjOzkZeXp5KwFy5cgWoPYjPnj2Lf/3rX3j77bcRERGB9957Dx4eHtDR0cEPP/yAqqoqHD58GOvXr8fSpUthb28PS0tLzJ8/H1lZWYiNjcWff/4JACgrKxPqvXz5MubNmwcfHx9IJJIGrY5Hve7YsSN0dHSwePFiKJVKyOVyJCcnC9NJpVK0b98eqO1oTaFQCNvmzp07yM3NRVpaGlJTU1W2r0wmg4eHxwNbiIw9luZ2DSYzM5MANPg3a9YsKisrIw0NDeG6wuTJk8nExISMjY0pODhY6OOptLSUunXrRpqammRtbU1KpZJOnjxJAOjmzZvCspYsWUIASCqVkoWFBQ0aNIiOHz+uUs+PP/5ITk5OQv889cefPn2aOnToQDo6OmRvb0/Hjh0jqr2GZGJiQra2tjR8+HDq0qULffjhh0REFBQURJ999pkwj379+tG0adOE1wMHDqSpU6cKryMjI8nFxYV0dHTI3d2dZDIZUe0Faw8PD2G6ugvjKSkpREQkk8mobdu2pKWlRW+99ZbKOvXu3ZtGjRr1lHtKLVrMNZgWo7kFjFi++uqrBh1+vaisrKweenfpOddiAoZ/fdTCyGQyuLi4IDs7G8bGxo3ucrYlycvLQ35+PrKzs/kCr5rxyWkLI5fLERMTA2traxw/flzd5TQ5IoKTkxPc3NwAvoOkdtyCaWHqLsC+qCQSCYqKitRdBqvFLRjGmGg4YBhjomlJp0h/AyhQdxEtWVVVlQ4AiZaWVrm6a2nhqtRdAGNNbty4cRHjxo37Ut11sOaDT5EYY6JpSadITGTV1dVFUqlUS911sOaDA4Y1mqamphEAc3XXwZoPPkVijImGA4Y9jnIiuqfuIljzwQHDHoeuRCJp+OAVxh6CA4Y9jhKlUsm/NWKNxgHDHoeBVCo1VXcRrPnggGGMiYYDhjWaRCKpJKIyddfBmg8OGNZoRKQtkUhevCdYsSfGP7RjjVZTU1MikUjy1V0Haz44YFijaWhoGAAwU3cdrPnggGGNVtuC4dvUrNE4YFij1bZg+DY1azS+yMseh5KIatRdBGs+JOougD3fxo4dmymVSu1Q+8R+1D5Yu/Z17nfffWep3grZ84xbMOyRpFLpdiJSojZY6sKl1lG1FcaaBQ4Y9khKpXI9gNT7hxNRhlKp/I96qmLNBQcMe6RNmzbdJKJDdadHdYgo+vvvv5eprTDWLHDAsH9UVla2AkBK3WulUnmzqqpqhXqrYs0BBwz7Rzt37rypVCoP1LvIe2rbtm1yddfFnn8cMKxRtLS0VhBRKhHlVFZWLlN3Pax5eORt6g0bNgzV1tYOabpy2POsuLjYj4j0jIyM/lB3Lez5UFVVdWjcuHFbHzb+n37J29Hc3DzY1tb22VfGmrNgdRfA1C83NxdZWVn3ADxxwMDU1BQODg7PvDjGWPOmVCqRlZX1yGn4GgxjTDQcMIwx0XDAMMZEwwHDGBMNBwxjTDQcMIwx0XDAMMZEwwHDGBMNBwxjTDQcMIwx0XDAMMZEwwHDGBMNBwxjTDQcMIwx0XDAMMZEwwHDGBMNBwxjTDQcMIwx0XDAsH+Uk5MDHx8fjBkzRmX4+fPnkZycrLa62POPA4b9o6ysLNy8eRMXL14Uhk2bNg0DBgxAYmKiWmtjz7d/fOg3Y35+foiKikKbNm2EYcXFxWqtiTUP3IJpYSoqKuDg4AALCwvcvXtXGD579mzMmzdPeK1UKuHp6QlbW1uUlJQgJCQEZmZmmDlzJrp27QorKysEBwfjr7/+gpmZGd566y2MHTsWADB58mRERUUBAEJDQ2FmZobJkycL846Li8Pw4cPRpk0b2Nvb46233kJcXJwwfv369TAzM8P777+PwYMHw87ODu3atUNRUVETbSXWVDhgWhgdHR0MGjQINTU1OHLkCACgsrISe/bswY8//ojKykoAwJ9//onbt2+jf//+MDAwEN7/3XffwcbGBgMGDEBYWBhMTU3Rs2dPlWX4+/sLrZmXX34Zw4YNg7+/PwDg4sWLGDBgAH799Ve4ubnByckJ0dHRGDhwIGQymcp8Dh06hLy8PAwbNgzvv/8+jIyMRN8+rGnxKVILFBwcjJ07d+LQoUMIDQ3FkSNHhNbMkSNHMHToUOzbtw8AMHz4cJX3Dhs2DN9//73KsCVLlqBbt27C61GjRuHPP/9EZmYmJk6ciNdff10YN23aNJSVlWHTpk148803AQDbtm3DJ598gqVLl2LHjh3CtPb29oiOjoaenp5IW4KpGwdMC9SzZ0+0bt0aZ86cQVFREXbs2AFdXV1oaGhg27ZteOONN3Dw4EGYmJigT58+Ku8dNmzYEy/35s2buHbtGrS0tHDlyhVcuXIFAFBeXg4AClnZCQAAIABJREFUuHz5ssr0r732GodLC8cB0wJJpVIMGzYM3377LbZs2YLffvsNI0aMgI6ODrZt24Zt27YhLy8PoaGh0NbWVnlv/dOlx5WdnQ38f3/FWLt2bYPxurq6z2xZrHngazAtVN3pyZIlS6BUKjF27Fh88MEHICLMmTMHqD2VelpKpVL4/7prKFZWVsjPz2/w7/4WDGv5OGBaqE6dOsHR0RGVlZXw9/eHr68vPDw80L17d5SXl8Pa2hrdu3d/4vkbGhoCgPBDu8rKSri4uKB169bIzs7Gpk2bhGlzc3ORkpLyDNaKNTccMC1YXQul7vZy/f8fOnQopNIn3/1dunQBaltIffr0QWBgIKRSqdA6mjFjBrp06YI+ffrAz89P5RY5e3FwwLRgwcHBMDc3x9ChQ4Vhr7/+OmxtbRvcPXpcw4cPx/jx42FkZAS5XA5TU1MAwLvvvoutW7fCz88PmZmZUCgUcHJyQu/evZ96fVjzI3nUyA0bNnzu6ek5r3379k1XEWOsWUhNTUVcXNz28PDwUQ+bhlswjDHRcMAwxkTDAcMYEw0HDGNMNBwwjDHRcMAwxkTDAcMYEw0HDGNMNBwwjDHRcMAwxkTDAcMYEw0HDGNMNBwwjDHRcMAwxkTDAdPENm/eDEdHR9jZ2eHll1/Gp59+ipycHHWX9Uxs374dS5cuFV5fuHABZmZmyMjIEIZdvXoVU6dOVXnfgAED8NFHHz10vsuWLcOrr76qMmzGjBk4ePDgM62fPXscME3s3Llz8Pf3x5EjRzBt2jQcP34cQUFByM/PV3dpT2316tWwtrYWXjs7OwMA0tPThWHz589X6W721KlTuHz5MmbMmNFgfjU1NQCADh06oF+/fsLwW7duYdOmTfDw8HiiOuvmy8THAdPEZDIZunfvDh8fHwwfPhy//PIL8vPzsWvXLgBAQkICgoODYWdnBw8PD3z55ZfCe0tKSjBjxgy4ubnBxsYGERERQG3nZ19//bUw3a5du+Do6AjU9kk0ZMgQjB8/Hq6urnBycsLy5csxY8YMODo6ol27dti7d6/w3osXL2Lo0KGwtbWFi4sLFixYAADIy8uDl5cXtm3bhq5du8LGxgaDBw/GvXv3AABdu3ZFamoqPvvsM7Rt2xa5ubl46aWXYGxsLATMqVOn8NtvvyEpKUlY3rJlyxASEgIHBwfMnTsXgwYNwoQJE+Dh4YHdu3dj8uTJePfdd1FWVgbU9pPt7+8PqVSKwMBABAYGCvPavXs3unXrBmtra3Ts2BH79+8XtttLL70ktITqHvfJxMcB04TKysqQnJwMb29vYVjdgZyeno4bN25g4MCB6NOnD65fv46NGzdi6dKlOHfuHJRKJd59913ExMQgMjISCQkJ+Oijj1BRUYGUlBR4eXkJ85TL5fD09AQAVFdX4/z58+jZsycuX76MIUOGYMWKFejcuTOuXr2Kzp0745tvvgFqT2kGDx6M7t27QyaTYefOnVi1ahUyMzOhq6uLW7du4dy5c/j5559x9OhR/PHHHzh58iQAYNGiRTAwMEB6ejoyMjJgYWEB1LZi0tPTQUT44osvMGLECBQUFCA3NxfR0dGIi4sTgjI+Ph4JCQmYOHEi5HI5goOD8dVXX8HW1hZ1T1W0tbXFRx99hN69eyMzMxOnTp0CAKxduxYzZ87ErFmzkJSUhLCwMHz++edAbWgTEXJzc3Hy5EmcOXOmifY4436RmpBcLkdNTY1KGABAQUEBjIyMsHDhQnTv3h0TJkwAAPTo0QM2NjaQy+XIy8vDlStXcOnSJbRu3Rqo7SYkLi4ONTU1QqAAgEKhEJaRlZWFrl274r333gMAWFpaws/PD2+99RYAwMvLS+j1cc6cOejRowemTZuGqqoqxMTEwNTUFK1bt0ZCQgIkEgn+85//wNDQEFZWVtDS0hIeHH7p0iX4+vo2eJB4XcD89NNP+Pvvv7F8+XL89NNPSEpKwvLlyxEeHg4bGxugNmAiIiKE2nV1dVFYWIisrCzUf2zrpUuXVHqaLCwsxJIlSzB9+nS88cYbKCoqgkwmE06h4uPjYW5ujiVLlkBTUxOamvyxbyrcgmlCMpkMFhYWQkCgNnRu3bqFTp064eTJk+jVq5cwjoiQn58Pc3NznD59GgEBASrvRW2YmJubw8rKSmVYXeDI5XKVg1Mul6sEXGJiIjw8PFBRUYFLly7h6tWrsLe3h729PQ4cOIC9e/dCW1sbcrkcbdu2FboryczMRGVlJdq1awcAiImJEfqnrs/JyQnJyclYvHgx5s6dC0NDQ7i4uGD9+vWIj48XLvgWFhbi1q1bKutfty4aGhpwc3MDavthio2NVVlWTEwMSktLsX79ejg6OsLDwwNKpVJomcXHx+OVV16BlpbWY+8z9nQ4yptQXFycyulRWVkZIiIi0K5dO/Tu3Rv37t1TCZDo6GjU1NSgZ8+eOHbs2AO7WU1ISFC52Jmamorc3FwhRBQKhUqvAnK5XKW7WLlcjvHjxwuvv/vuO3Ts2BG6uroqvT7eH1QymQw6OjrChdzLly9j5MiRDepzdnZGbGws/Pz8hDq8vLywd+9eTJs2Debm5kKdWlpacHFxUXm/XC6Hk5OT0CtkYmIiiouL0aFDB5XpJBIJ4uLiUFpaCiMjI5WWlEKhQNeuXR+wR5jYuAXThGQyGczNzSGXyxEVFYW+ffsiKysLP/zwA7S1teHp6Yl9+/ahrKwMCQkJmDVrFj755BOYmZnBz88P0dHROHnyJLKzs4ULs+Xl5cjPz0dVVRXu3LmDKVOmQENDA+7u7sLpRV1rpri4GBkZGcLr0tJSpKWlwdPTEzo6OvD29sb69etRVFSE3NxcXLx4Uai9fquobl3atWsHTU1NVFdXo6CgAAqFArdv30ZhYaEwXV0AzZ8/HxLJ/3di4eXlBRMTE0ycOFGYLj4+Hq6urg1aGQqFQiXYcnNzgdrb3SkpKSAieHt7Q0dHB6tWrYJSqURCQgJSU1NV5s09Y6gHB0wTUSqVUCgU2LNnD/r27YuvvvoKAwcOxNmzZ+Hq6goA+Oabb5Ceng4XFxeEhobigw8+EG7fjh49Gu+88w4+/PBDdOnSRbhDEhISgpKSEri7uyMsLAyWlpZwdnaGnp4eFAoFNDU1hdOLhIQEABBaPAkJCVAqlUJwrF27Fnfv3kVAQAD69++PGzduCPU/6FSr7rWmpiY++OADrF69Gl27dlXpxdHZ2RkDBgxQuWbi6emJjz/+GMbGxsIwhULxwNvO9w/v3LkzAgICMHLkSAwaNAgAYGFhgXXr1iEqKgqenp4IDw9HZWUlUHv3Kzc394lvabOnw/0iMcaeCPeLxBhTKw4YxphoOGAYY6LhgGGMiYYDhjEmGg4YxphoOGAYY6LhgGGMiYYDhjEmGg4YxphoOGAYY6LhgGGMiYYDhjEmGg4YxphoOGAYY6LhgGGMiYYDhjEmGg4YxphoOGAYY6LhgGGMiYYDhjEmGg4YxphoOGAYY6LhgGGMiYYDhjEmGg4YxphoOGAYY6LhgGGMiYYDhjEmGg4YxphoOGAYY6LhgGGMiYYDhjEmGg4YxphoOGAYY6LhgGGMiYYDhjEmGg4Y9lgKCgpgZmaGmJgYdZfCmgEOGDWorKzEypUr0blzZ1hbW8PPzw/Lli1DVVVVo+cxY8YMHDx4UGXY4MGD8emnn4pQ8f9cvXoVmpqa8PT0fOg0TVEHAPz1118YPXq06MthT05T3QW8aGpqahASEgKZTIbPP/8c7u7uuHz5MmbPng0AmDlz5j/O49atW9i0aRPGjRunMjwoKAht27YVrXYAiIuLg7u7O3R1dR86TVPUAQA//vgjNDX5I/w84xZME9uwYQPOnDmDffv24e2334aPjw/CwsIwcuRIHDhwAAAwffp0hIeHY8SIEbC3t0fnzp1x+PBhAEBWVhb8/f0hlUoRGBiIwMBAAIC/vz+++OIL6OvrC8s6fPgwevToARsbGwQEBODYsWMAgBMnTqBXr15Ys2YNPD090aZNG5UWR3p6OkJCQtC2bVvY29ujb9++SEpKAgDExsaiY8eOD12/++tYsmQJJkyYgEmTJsHe3h6urq7Yu3cvACAyMhK9e/dGREQEXF1d4eHhgWXLlgnzevnll/H1118Lr3ft2gVHR0cAQEREBHbs2IHDhw+jTZs2wrbbsmULOnToADs7OwQHB6O6uhqVlZXo0qULPvzww2ewB9nj4IBpQkSEtWvXYuTIkXBzc1MZZ2Njg7y8PABAbm4uUlJSMGfOHFy5cgUBAQGYMGECKioqYGtri48++gi9e/dGZmYmTp06BQDYs2cPAKB9+/YAgAMHDmDSpElYsGAB0tLSMGLECIwfPx6lpaVQKpVQKBQgIpw7dw6fffYZvv32W9y9exe5ubl4/fXX4ezsjPj4eJw8eRIxMTFCSyEuLg4dOnR46DreX0dpaSmio6MxcOBAyGQy9OzZEytXrgQAFBUVIS0tDf369UNMTAxmzpyJZcuW4fz586ioqEBKSgq8vLyEecvlcuHUbMGCBdDQ0MChQ4eQmZmJwYMHIz4+HhEREVixYgWuXLmCMWPGQFNTE1KpFMbGxjAyMnqGe5M1BgdME0pOTsbt27cxaNCgBuMyMjLQunVroLaVEhYWBi8vL5iZmWHs2LEoKSlBVlYWAODSpUvw9/dXeX9CQgIMDQ3Rpk0b1NTU4NNPP8XMmTPx6quvQkdHB8HBwSguLkZGRgZSUlLg7e2NSZMmwcjICL6+vgAAqVSKJUuWwM7ODvPnz4e+vj4yMjJgamoKJycnIRAeFTD16wCA1NRUjBgxAgMGDIChoSE8PT0hlUqF9QwMDET//v1hZGSEsLAwGBoaIikpCYmJiaipqVG51qNQKITAiY2NhYaGhkoAVVdXAwBu3LgBc3NzDBw4EACgqamJEydOYPny5U+879iT4YBpQnUtFDs7O5XhVVVVOHXqFLp16wYiQmJiosqBVVBQAAAwNTWFUqlEbGxsg4BRKBTw8PAQ/v/WrVvo1atXg2W/9NJLUCgUKvNPTk6GhYUFTExMsG/fPoSGhgrjLl26JATQtWvXIJVKH3mBt34duK/VAQApKSlo164dACA+Pl5lXGVlJe7duwczMzMoFAqYm5vDyspKZd5108fExMDb2xva2trCeG9vb2zatAkrV65EUFAQMjMzH1onaxocME2oroWSkpKiMnzz5s3IyclBeHg40tPTUVJSAnd3d2H80aNH0alTJ5iamiIxMRHFxcUNWhEKhUI4LSkqKgIAlYPz4MGD8PX1hYWFBeRyeYMQaN++PQoLC1FYWChc56hbdl3AyGQyODk5QU9P76HrWL+O4uJiZGZmCq/r5lH3uv60dcvS1dVFt27dkJCQoFJjamoqcnNzhRZLTEzMA1tSb775Js6fP4+ysjJ88803D62TNQ0OmCbk6OiITp06Yd68eTh16hRiYmIwf/58zJkzB4sXL4arqysUCgX09fWRk5OD27dvY/Xq1di1axcWLlwI1F6fQe3t4pSUFBARcN/B2q5dO+jo6CAyMhJVVVU4fvw4tmzZgrlz56KmpqZBC6nuoDcyMoKhoSGSk5MBAOvWrUNcXBwMDAwAAPn5+UDtnbCHqV+HQqGAVCoVrjdVV1cjMTER7du3R1FREbKyslBaWoqCggIcPHgQ06dPx6xZs2BiYoLy8nLk5+ejqqoKd+7cwZQpU6ChoSEEb25uLtLS0pCdnY1bt24BtUGdlJSEO3fuoLS0VAjKzZs34+WXXxamY02HA6YJSSQSbNmyBa6urhg9ejSCg4Nx9epVREZGIjw8HKg9KC0sLDB8+HD4+/vj6NGjiIqKQpcuXQAAnTt3RkBAAEaOHClcyykvL0dqaqpwYFtYWGDt2rXYsGEDHB0d8eWXX2Lr1q3o1asXkpOTUV5ertJyqGvRSKVSrFq1CsuXL0fnzp1x8uRJBAQEIDExEQAQHBwMLS2th95Kv78OuVwOFxcX6OjoAACSkpJQWVmJ9u3bQ6FQwMDAAKtXr0b79u2xcOFCzJ49GxMnTgQAhISECC25sLAwWFpawtnZWWg9hYeH4+LFi/D398fBgwdRWlqK/fv3IygoCIMGDcKwYcOE2/glJSW4e/fuY/3OiD0bkkeN3LBhw+eenp7z6n8YmbjGjBkDe3t7zJ07V92liGrLli348ccfcfz4cXWXwp5Qamoq4uLitoeHh4962DTcgnnOKBQKuLq6qrsM0SkUCuFiL2u5OGCeI5WVlUhJSXlhAuZFWM8XHf/O+jmira2NO3fuqLuMJlH3y2TWsnELhjEmGg4YxphoOGAYY6LhgGGMiYYDhjEmGg4YxphoOGAYY6LhgGGMiYYDhjEmGg4YxphoOGAYY6LhgGGMiYYDhjEmGg4YxphoOGAYY6LhgGGMiYYDhjEmGg4YxphoOGAYY6LhgGGMiYYDhjEmGg4YxphoOGAYY6LhgGGMiYYDhjEmGg4YxphoOGDUpKCgAGZmZoiJiVF3KYyJhgNGTa5evQpNTU14enoCAPLy8uDl5YV9+/apuzTGnpkXMmCuX7+O4cOHw97eHoMHD8b169ebvIa4uDi4u7tDV1cXAGBoaIj+/fvD2dm5yWthTCwvZMBMnjwZv/76K4qLi/H7779j4sSJTzSf6dOnIzw8HCNGjIC9vT06d+6Mw4cPC+MTEhIQHBwMOzs7eHh44MsvvxTGxcbGomPHjgCAW7duwdraGlu2bIGDg4Mwzbfffgt/f39YWVmhR48e2LNnD6ytraFUKoVpli1bBi8vL1RXVz/h1mBMPC9kwMTGxqq8vnbt2hPNJzc3FykpKZgzZw6uXLmCgIAATJgwARUVFbhx4wYGDhyIPn364Pr169i4cSOWLl2Kc+fOAbUtmA4dOgAAbGxs8PXXX6Nt27YwMjICACxevBirVq3CwoULkZiYiO+++w7t27dHRUUFMjMzAQCVlZXYsmULxo4dC01NzafcKow9ey9kwHh4eKi8dnR0fKL5ZGVlISwsDF5eXjAzM8PYsWNRUlKCrKwsLFy4EN27d8eECROgp6eHHj16wMbGBnK5HEVFRUhLSxMCBgDi4+PRvn17AEBOTg6+/vprfP311xgwYACMjY3h7u4OV1dXaGhoIDk5GQDw888/o7i4GO+///5TbQ/GxPJCBsycOXPwyiuvwNDQEF26dMGiRYseex5EhMTEROEiLWrvDAGAqakpTp48iV69eqlMn5+fD3Nzc1y7dg1SqVTlvQqFQgiYc+fOQUNDA/369VNZpo6ODhwdHYVrRhs2bMDbb78NMzOzJ9gKjInvhWxXBwYGIjAw8KnmkZ6ejpKSEri7uwvDjh49ik6dOsHY2Bj37t1D69athXHR0dGoqalBz549ERUVBScnJ+jp6QnjFQqF0BIpLi6Grq4upNKG+e/u7o7k5GT89ddfuHr1KtavX/9U68GYmF7IgHkWFAoF9PX1kZOTg3v37iEqKgq7du3Czz//LLRO9u3bh969eyM9PR2zZs3CJ598AjMzM+Tn5wMAampqoKGhgZycHOTl5QktGD8/PxQWFmLlypUIDQ2FXC6Hs7Mz2rRpAx8fH0RFRSEjIwOvvvqqSsAx9rx5IU+RngWFQgELCwsMHz4c/v7+OHr0KKKiotClSxcAwDfffIP09HS4uLggNDQUH3zwAWbMmAEACA4OhpaWFmbOnCnMS1tbGy4uLgAAT09P/Oc//8HWrVvh4+ODOXPmCBdxhw8fjrt37+LkyZP48MMP1bb+jDUGt2CeUHx8PIYOHYq5c+c+cLy3tzdOnjz5wHHt2rXD77//LrwODAxEdna2yjTh4eEIDw9v8F4HBwe88847OHbsGPr27fvU68GYmDhgnpBCoUCfPn2adJkXL17E6dOnsXHjRqxduxYSiaRJl8/Y4+KAeQKVlZVISUmBq6trky537ty5yMvLw4IFC/DWW2816bIZexIcME9AW1sbd+7cafLlHj16tMmXydjT4Iu8jDHRcMAwxkTDAcMYEw0HDGNMNBwwjDHRcMAwxkTDAcMYEw0HDGNMNBwwjDHRcMAwxkTDAfOMJCUlgYjUXcZTu7+/JqVSCRsbG5iZmcHMzEz0P5FoCf1FHTx4EG3atFF5OPvTCgsLE/bB5s2bAQA3btx4ZvMXCwfMM/DHH39g6dKlwl83V1ZWYuXKlejcuTOsra3h5+eHZcuWoaqqqlHzy87ORkhICLKyslSGDx48GJ9++qko61Dn/v6aUlNTUV5ejqioKCQkJMDS0hKbN2+Go6Mj7Ozs8PLLL+PTTz9FTk6OKMt/Vv1F/fXXXxg9enSjp4+JiYGTkxMsLCzg7OyMAQMG4NixY416b1xcHLy9vYUnEkZHR8PLywtpaWlPXP+qVavw3//+F6h9qmFdjV999dUTz7Mp8B87PqWcnBxMnTpV6K6kpqYGISEhkMlk+Pzzz+Hu7o7Lly9j9uzZACA8ZOpRzp49i8uXL8PW1lZleFBQENq2bSvSmvy/+/trio+Ph0QiQUBAAAwMDIDaZwb7+/tj7ty5SEpKwtKlS/HLL7/g7NmzT/18YLH6i/rxxx8fq+eFS5cuQV9fH/v374e2tja2bduG999/H+fPn//Hh8RfvXpV6JIGAOzt7dGvX7+n2jampqa4e/cuUO+h9X369EHHjh0xceJEaGlpPfG8xcQtmKe0ePFi9O3bF5aWlkDtg7jPnDmDffv24e2334aPjw/CwsIwcuRIHDhwAADQo0cPfPrppwgKCkKbNm3w2muvITExEQCwd+9eTJw4EXfv3kWbNm2EFou/vz+++OIL6OvrA7WnLitXroSPjw/s7OwwcOBAJCUlAQBOnDiBXr16Yc2aNfD09FSZD2qfJxwSEoK2bdvC3t4effv2Fd5bv78m1Pbt1KZNGyFcAEAmk6F79+7w8fHB8OHD8csvvyA/Px+7du0CALz88sv4+uuvhel37dolHJSRkZHo3bs3IiIi4OrqCg8PDyxbtkyY9kn6i6o7Nf3uu+/QqVMnWFtbw9PTE4sXLwYAREREYMeOHTh8+DDatGkj7IfKykosXLgQ3t7esLa2RlBQEBQKBVAbdK+88gp8fHzg7u6OSZMmobq6GgkJCY/c9rivS5o9e/agS5cuiI2NhbGxMVDbKnzvvfeE7R8ZGYkpU6YgODhY5bM1aNAgjB07VmVfWFlZwdTUFABgbGyMqqqq5/p0kgPmKeTk5CAyMhKhoaFAbc8Ba9euxciRI+Hm5qYyrY2NDfLy8gAAt2/fRl5eHnbs2IEzZ86goqICs2bNAmofienr64vPPvsMmZmZwkGyZ88eABCe2zt79mz88ssv2Lt3LxISEmBiYiI8klOpVEKhUICIcO7cOXz22Wf49ttvcffuXeTm5uL111+Hs7Mz4uPjcfLkScTExAjf7vUPDtS2YOo/97esrAzJycnw9vYWhtna2sLFxQXp6emoqKhASkoKvLy8hPFyuVw45anrsqVfv36IiYnBzJkzsWzZMpw/f77B8hvbX5REIsGKFSuwZs0afPfdd7h58ya6d++O9PR0AMCCBQugoaGBQ4cOITMzE4MHDwZqr2scOXIE27dvR1JSEqysrLBu3TqgNuj8/f1BRMjMzMQXX3wBQ0NDdO7c+ZHb/tatW7hz546wDm+99RbefvttYb9lZ2dj4MCBMDExwenTpxETE4MePXrAw8NDpYdRmUyGP/74A+PHj3/ovkBti7luPZ9HHDBPYf/+/TAwMBCarMnJybh9+zYGDRrUYNqMjAy0bt0apaWlKCgowLRp02BjYwNHR0cMHz5c+AasqqrCtWvX4O/vr/L+hIQEGBoaok2bNkhKSsLGjRuxbt06tGvXDgYGBnj99dchl8sBACkpKfD29sakSZNgZGQEX19fAIBUKsWSJUtgZ2eH+fPnQ19fHxkZGTA1NYWTk9ND+2uq34+UXC5HTU2NSoCg9uKskZEREhMTUVNT06BLlrrps7KyEBgYiP79+8PIyAhhYWEwNDREUlLSE/cXdfPmTSxfvhyrVq2Cr68vNDQ0kJaWBj8/P6A2LDQ0NFRq/vXXX3Hs2DGsXr0avr6+yM/PR1paGjw8PFBWVobr16/j3//+NywtLdGxY0fk5ORg//79yM/Pf+S2v3r1KvT19dGuXTsAgEQiUemS5quvvoKNjQ3WrFkDR0dHmJubw8bGBu7u7sjKykJpaSlQ2xL29/dH586dVT4D9fdFUVERKioqUFlZ+Y+fVXXhgHkKv//+u3DwovaCJADY2dmpTFdVVYVTp06hW7duSEhIgI6Ojso1hbo7J6jtZbKqqkrlIEPtQVr34YqOjsZLL72kchDfvXsX5ubmwrT1xyUnJ8PCwgImJibYt2+f0OJC7bWGunW4v7+mqqoqpKamqnxrymQyWFhYqHTJIpfLcevWLXTq1AkKhQLm5uawsrJSqb1unvHx8Sq1VVZW4t69ezAzM3vi/qIOHjyI1q1bIygoCABQUVGBa9euCQETExMDb29vaGtrC+/5448/YGBggODgYDg4OCAoKAgDBgzAhAkTIJPJoFQqkZSUhJiYGNy6dQv79u2Dr6/vP277uLg4eHp6Chd4q6urcf36dWHfnT59GoMGDWrwuFN3d3cQEVJTU5GXl4effvpJ5aHuSqUS169fV9kXdXeRrK2t8bzigHkKKSkpKhdd6w66lJQUlek2b96MnJwchIeHQ6FQwM3NDRoaGkDtB+fYsWPC831jYmLg4uIiXGupU/9AKyoqUjnAUXuQ1T0EXC6XN2h1tG/fHoWFhSgsLFS5SHn06FEhYGQymUp/TcnJyaiqqlL5UNfdIalTVlaGiIgItGvXDn379m3wLZuamorc3Fyh9VB/PeqWr6uri27dujVY/v3TP6y/qIyMDJV1io6ORlVVFXx8fIRten9go/bB7CkpKYiLi0NKSgrmzp0LqVSKuLg42Nvbw9zcHG3btoUutbFpAAAXNElEQVSOjo7wnsZs+/rrl5ycjIqKCpV1qL9+daysrGBiYoLk5GRs3boVpqamwqkcAKSlpaG8vFxlX5w/fx4aGhpCkD6POGCeQk1NjXDhDrVd0Hbq1Anz5s3DqVOnEBMTg/nz52POnDlYvHgxXF1dER8fDy0tLeTl5SE5ORkfffQRioqKMGnSJKC2FXT37l2kp6er/M6h/oHm4+OD69ev49KlSygvL8eXX36Jmzdv4uOPP0ZNTU2DHidlMhnat28PIyMjGBoaCl3Prlu3DnFxccIF3Pr9NaG2tSGVSoXmft28zM3NIZfLERUVhb59+yIrKws//PADNDU1UV5ejvz8fFRVVeHOnTuYMmUKNDQ04O7ujqKiIuE0oKCgAAcPHsT06dMxa9YsmJiYNFj+o/qLys3NxW+//YbMzEzY2toiMzMT5eXluHPnDj7//HNoaWkJwZCbm4u0tDRkZ2fj1q1bAIBOnTohJiYGx48fh1KpxG+//Yby8nKg9jTn/mtodR617VHbmqmrv26/vfTSS0Io+fv7Y/PmzYiPj0d6errKY1A7dOiAAwcOYMuWLQgPD1e5M1R3N69+wJw7dw6vvvrqc92zJwfMU7C2tlb5NpJIJNiyZQtcXV0xevRoBAcH4+rVq4iMjBS6IFEoFKioqEBAQAB69+6NiooKHDlyRLgzMGzYMOjq6qJLly6YP38+AKC8vBypqanCgTZgwAB8+OGHePfdd+Hh4YHY2FgcPnwYlpaWSE5ORnl5ucq3aF2LRiqVYtWqVVi+fDk6d+6MkydPIiAgQLiDdX9/TQkJCXBwcBDWse7i8Z49e9C3b1989dVXGDhwIM6ePSs8AD0kJETo8TIsLAyWlpZwdnaGnp4eFAoFDAwMsHr1arRv3x4LFy7E7NmzMXHixAcuv7H9Rb3//vtwdnaGv78/hgwZgp49e6KiokK4+BkeHo6LFy/C398fBw8eFLbhxIkT8a9//QteXl5YsmSJEEiPCphHbXsAGDduHKKjo/HTTz8J61B/XyxevBht27ZFv3790KdPHyQkJAjjwsLCcPjwYRQUFCAsLExluQkJCbCzs4OhoSFQ23KMjo7G5MmTH+sz29Qe2e/Fhg0bPvf09JxXfwOx/1m1ahXu3LmDJUuWNPo97u7uWLt2LXr37i1qbc9CaGgoiAg7dux4JvPbsmULfvzxRxw/fvyZzK8lCgoKgpeXF1avXq0yfOzYsSguLkZkZCQA4Pvvv8cff/wh/KpXHVJTUxEXF7c9PDx81MOm4RbMU3j//feF3000Rl5eHu7cuaNyyvE8i4+Ph52dHXJyclSa/U9KoVA0m3VvSkqlEidOnMCECRMQHx+PTz75RBhXVFSEnJwclYv86enpOHDgwHP/K15wwDydl156CVOmTGn09AqFAnp6eg3uMj2PysvLcePGDWzcuBEeHh4oKSl56nkqFIom70uqOSgrK0N4eDiSk5Px448/qlywHjduHDw8PFQunstkMmzbtk34bdDzjP9U4CnV3RptjJ49ezb4+6Lnla6urnDb/Vmp+3MKpkpfXx+ZmZkPHLd79+4Gw15//fUmqOrZ4BYMY0w0HDCMMdFwwDDGRMMBwxgTDQcMY0w0HDCMMdFwwDDGRMMBwxgTDQcMY0w0HDCMMdFwwDDGRMMBwxgTDQcMY0w0HDCMMdFwwDDGRMMBwxgTDQcMY0w0HDCMMdFwwDDGRMMBwxgTDQcMY0w0HDCMMdFwwDDGRMMBwxgTDQcMY0w0HDCMMdFwwDDGRMMBwxgTDQcMY0w0HDCMMdFwwDDGRMMBwxgTDQcMY0w0mv80wc2bNyuLiooqm6Yc9pyT1P6X1FwHew6UlJRo/dNn4ZEBI5FI9hcUFNwoKCh45sWx5icvL++1mpoak9atW+9Wdy3s+SCRSBLVXQNrIcaNGxcxbty4L9VdB2s++BoMY0w0HDCMMdFwwDDGRMMBwxgTDQcMY0w0HDCMMdFwwLBGUyqVNURUre46WPPxj7/kZayOVCrV4M8MexzcgmGMiYa/jVijEVEREWmpuw7WfHDAsEaTSCRGEonEXN11sOaDT5EYY6LhgGGMiYZPkVijEVGRRCLhazCs0bgFwxpNIpEYAeBrMKzROGAYY6LhgGGMiYYDhjEmGg4Y9jiURFSj7iJY88EBwx6HVCKRaKi7CNZ8cMAwxkTDAcMYEw0HDGs0iURSSURl6q6DNR+SRkzzQiKivgCOq7sO1ixMlkgk36i7iOcRt2AYY6LhgGGMiYYDhjEmGg4YxphoOGAeU3JyMiQSCXbv3q3uUppMUVERrly5ou4yHmjz5s2wtLRERkaGMOxh9dbU1OCPP/5o4gpfbBww7B/5+Pjg+++/V3cZD6SnpwdjY2NoaPzvB8YPq3fMmDEYP358E1f4Yvu/9u49KKrC7+P457BcFgRFyEsBig+b2mTyU/xZFuqjmKI2MDKjjXnDy6aU1tTUlJcRZSZlukxjPmMpqJgUVmIa4DMNk2niL/FaYWlexktggLbGjx+53Pb7/KF7HlZYWJLDsruf1wwzevZw9su2vjtndzmHgaE2mc1mZ4/QjIgAAGbOnIkLFy4gLCxMvc3evF3x5yAPJSJPi4hUVlbKc889J4GBgdKrVy+ZMWOGAJCcnByxKi4ultGjR4ter5fQ0FCZP3++mEwmaWrr1q0SHR0tfn5+0qdPHzEajVJRUSH19fUCQNavX2+z/tSpU+Xxxx8XEZHTp09LUFCQ5Ofny9ChQ8XHx0eioqJk27Ztsm7dOomIiJCgoCBJTEyUyspKm+1s2rRJDAaD6PV6GTx4sKSlpcnt27fV7fr7+8vBgwfliSeeEL1eL4MGDZJ9+/ap39+/f38BoH71799fRERqampk3rx5EhISIiEhIZKYmChXrlyRttTW1kpwcLAsWrTIZvkzzzwjN27cUP9+/fp18fLyku3bt8sXX3whAGTv3r3y1FNPiY+Pj6xatUrmzZunzlVfX9/qvE3XtX5dvny5ox6nZc5+vpKLEZGnzWazREdHS0BAgKxevVq2bdsmI0aMsAnMzz//LAEBATJy5EjJycmRDRs2SHBwsMTFxalP3tTUVAEg06dPl6ysLHn77bclLi5Obt265XBgAEi/fv0kPz9fvvnmGxkyZIgAkNjYWDly5Ijk5ORIYGCgzJ49W93GmjVrJCgoSFauXCmffvqprF69WoKCgmTOnDk22+3du7fs2rVLjh49KpMmTZKAgAD1H/vx48clJCREkpKS5PDhw3L8+HEREVm1apUoiiJpaWmSmZkpsbGxcvPmzTYDIyIyd+5c6dOnjzQ2NoqIyLVr18TLy0veeecddZ2NGzeKj4+PmEwmNTBhYWGSlZUlBw4ckNLSUjl58qTMmTPHJjD25j1//rzExcXJgAED5PDhw3L48GExm80d9TgxMNQ+IvL0e++9JwCksLBQfeKfPXvWJjAzZ86UoKAguXXrlrrOzp07BYAcOnRISktLxdvbW32y3qs9gfn888/V27dv3y4A5MyZM+qyBQsWSN++fUVEpKysTHx8fGT37t022/3oo48EgJhMJnW7u3btUm8/deqUAJDc3Fx1WZ8+feTFF1+02c7s2bMlMDBQ6urq7HXErry8PAEgR44cERGRtWvXCgAZOHCgus6YMWNk8uTJIiJqYNLT05tta/369TaBsTeviMizzz4rjz76qM2yDnqcGBg7+BpMK/bs2YOhQ4diwoQJ6jJvb9vzpB88eBDjx49HcHCwumzSpEkAgBMnTqCwsBANDQ1ISUm573n8/f3VP+v1egCAn5+fuiw8PBw3b94EABQWFqK+vh6zZs2CXq9Xv5Ytu/NvobS0VP2+bt26qX/u378/AOD69eutzjJr1iz89ddfmDx5MkpKStr1c0ycOBE9evTA3r17ISLIysqC0WjEpUuXcOjQIZSXl6OoqAgzZsyw+b64uLh23Y8jtH6cPB2vKtCKa9euYfjw4a2uU1VVhd69e9ssCwkJAQCUlZWhvr4eABAREaHhpHcoiqK++FleXg4AyM/PR3h4eLN1DQYDzpw502y5r68vAKChofVr3MfHx6OgoACvvfYaoqOjsXDhQmzatAk+Pm1fdMDX1xcJCQnYt28f4uPj8dtvvyE1NRU3btzAli1b8OSTT0Kn0yExMdHm+wIDA9vcdntp/Th5OgamFb169UJlZWWr64SHh+OPP/6wWVZRUQEA6Nmzp7pnU15e3uITWFG0+X3Tnj17qn8ePHjwfW/PGq6m4uPjMXHiRGzYsAGvvvoqIiMjsXLlSoe2N336dOzcuRPLly9HQkICwsLCsGTJEiQmJuLs2bOYMGGCzc/QEfO2tLyjHyeyxUOkVsTExOD48eM4f/683XVGjRqFgwcP4vbt/z+Lwe7duwEAsbGxGDduHBRFQWZmps33Wf/Pp9PpEBISYrOrLSI2Hxz7O8aPHw8vLy9s3Gj7S741NTXt3la3bt2aHQrU1tYCALy8vPDKK68gLCwMp06dcnibEydORPfu3XHs2DEsWbJEXRYeHo7Tp083Ozy633mty8vLy2GxWNRlHfk4UXMMTCvefPNN6HQ6jBkzBunp6dixYweWLl1qs87KlStRU1OD+Ph45OTkID09HW+88QbGjRuHsWPHYuDAgTAajdi8eTOmT5+OrVu3Ij09HQ8//DCuXLkC3H3NZufOndi7dy+OHj2KGTNm4Ny5c/c1u8FgwEsvvYSvvvoKCQkJ2LZtG9566y0YDIZ2fyp3zJgx2L9/P9LT07FlyxaUlJTggw8+QGxsLDZv3ozU1FSUlZVhxIgRDm/Tz88PCQkJMBgM6mtciqJg8eLF8PHxaXZ4dL/zWpebTCYsWbIEH3/8MfLy8jr0cSJymPVzMN9++60MHz5c/Pz8JCoqSl5//fVmn4Oxfj7Cz89PQkNDxWg0SlVVlXp7Y2OjrFu3TiIjI8XX11cGDBggixcvltLSUhERqaiokGnTpkn37t0lIiJC1q1bJ1OmTGn2LlJeXp66zZycHAEgFy5cUJelpqaKTqdT/26xWOTdd9+VyMhI8fHxkX79+skLL7wgFRUVdrdbXV0tAOT9999Xl1VUVKhvy0ZGRsqePXskNzdXRowYIQEBAfLggw/Kyy+/3O53lPbt22fz1rSIyI0bN2TatGk2y6zvIp09e7bZNlp6F6mlea3/HZYuXSrdu3eXvn37yvLlyzvqceK7SHbwhFN28IRT1A484ZQdPESiDjV69GgEBwfb/Zo3b56zR6ROxHeRqEN99tlnqKurs3t708+SkPtjYKhDPfTQQ84egboQHiIRkWYYGCLSDANDRJphYIhIMwwMEWmGgSEizTAwRKQZBoaINMPAEJFmGBgi0gwDQ0SaYWComa+//hoRERG4dOmSQ+u3dIJuIjAw1JIBAwZg6tSpCA0NdWj9HTt2NLvaAhEYGLpXdnY2Bg0ahBMnTqgnLJ8wYQLWrFmDuLg4+Pv7Y9CgQeppKFNSUrB161Z8+eWXCAwMRG5urpN/AiIXYD1lpqexWCwye/ZsSU5OVpfFxMTI8OHD5YcffpDy8nIxGAzqhc1qamrE29tbjh075sSpnY6nzLSD+7VkQ1EUlJSUYO7cueqyixcvIjs7G9HR0cDdQygvrzs7vydOnIBOp1NvI2qKh0hko6GhAefOncNjjz0GALh69SqqqqowdOhQdZ3z58/jkUceAQAUFxdj2LBh6oXIiJpiYMjGr7/+itraWjUwP/30E3r06IF+/foBd69kefXqVfX24uLiNq9+SZ6Lh0hko6SkBA888AD69u0L3A2MNSYA8OOPPwIAhgwZAgCorKxEdXU1fv/9d1gsFoSFhTlpcuqKuAdDNkpKSmyC0lJgIiIi1HeYUlJS8P333yMqKorvIFEzvC6SHbwuErUDr4tkB/dgiEgzDAwRaYaBISLNMDBEpBkGhog0w8/B2GcGcMvZQ3QlFotFV1dXF6DX66udPUsX8x9nD0Dk8oxG4z+NRmOxs+cg18FDJHKYoiiNAHTOnoNcBwND7VEP4JqzhyDXwcCQwxobG0VRFIOz5yDXwcCQw3iIRO3FwJDDRKReRM44ew5yHQwMOczrzmns/uHsOch1MDDkMIvFYuFzhtqDTxZyWGNjY4OiKGZnz0Gug4Ehh/n6+lpEJMjZc5DrYGDIYXV1dXUATjt7DnIdDAw5zNfXVwEw0tlzkOtgYMhh9fX1DQB+dvYc5DoYGHJYY2OjRVGUYc6eg1wHA0MO8/b2ruMH7ag9GBhqF0VReJU1chgDQw6zWCz1InLU2XOQ6+B1kahVCxcu/F9FUcJFBF5eXjoABhH5DXf2ZnwzMjIinD0jdV08ZSa1SlGU3YqibPTy8vJvsuy/cGeP5qZTh6Muj4dI1KrMzMytInKxpdsURSnp/InIlTAw1CYReQ/A7abLLBbLDbPZ/D/Om4pcAV+DIYcsWrTolKIowxRFgYgAwL8yMjKecvZc1LVxD4YcoihKOoB/3/1rFYANTh6JXAD3YMhhRqPxe0VRnhCR4xkZGfydJGoT92DIYQEBAdu8vb2rGhsbufdCDuEeDDkkMzMzBMA1X19fs9lsHvv888/zlx6pTQwMtckal6SkpG4AkJ+f/wcjQ45gYKhVd+PyW1JSUkDT5YwMOYKBIbvsxcWKkaG2MDDUorbiYsXIUGsYGGrG0bhYMTJkDwNDNtobFytGhlrCwJDq78bFipGhezEwBHRAXKwKCgpu3b59ezQjQ2BgCB0YFytGhqwYGA/X0XGxYmQIDIxn0youVowMMTAeSuu4WDEyno2B8UCZmZkhiqKUTps2zd+B1e8bI+O5GBgP09lxsWJkPBMD40GcFRcrRsbzMDAewtlxsWJkPAsD4wG6SlysCgoKbpnN5v82Go0/OXsW0hYD4+a6WlysCgoK/m02m0czMu6NgXFjXTUuVoyM+2Ng3FRXj4sVI+PeGBg35CpxsWJk3BcD42ZcLS5WjIx74nWR3Mjdj/9f1youY8eORVpamhabxtSpU7vr9frDGRkZQzW5A3IK7sG4CWtckpKS/Dp62xcuXEBubi7Ky8sxatQo/PLLL1ixYgX8/Dr8rrgn42YYGDegZVwAoLCwEGlpaaioqEBoaCh0Oh0yMzMxePBgLe6OkXEjOmcPQPdH67gAQFRUFP788080NDSguroaa9euxciR2l2aeuDAgX6XL1+eNWXKlP15eXkVmt0RaY6vwbiwzogLANTW1qKoqAgrVqzA/PnzceDAAS3vDmjymswnn3zyD83vjDTDQyQX1VlxuZeIQFE672mzf//+/yiKMnrWrFk/dNqdUofhHowL2rhx42MBAQE/dXZcAHRqXABgypQpgXq9vuDDDz8c36l3TB2CgXFBy5YtK6mrq7tUVlZW7exZtGYymWqrqqpqUlJStD8uow7HQyQXlpWVdSgmJmZYWFhYkLNn0YLJZKr97rvvri1YsGCgs2ehv4eBcXHuGhnGxT0wMG7A3SLDuLgPBsZNuEtkGBf3wsC4EVePDOPifhgYN+OqkWFc3BPfpnYzycnJY0+ePHm6rKzstrNncRTj4r64B+OmsrKyimJiYoaHhYV16fPCMC7ujYFxY109MoyL+2Ng3FxXjQzj4hkYGA/Q1SLDuHgOBsZDdJXIMC6ehYHxIM6ODOPieRgYD+OsyDAunomB8UCdHRnGxXMxMB6qsyLDuHg2BsaDaR0ZxoUYGA+nVWQYFwIDQ9AgMowLWTEwBHRgZBgXaoqBIdX9RoZxoXvxdA2kSk5Ojj158uSpv3OqB5PJ1MC40L24B0PNtHdPxmQyNRQVFV1NTk42aD8duRIGhlrkaGQYF2oNA0N2tRUZxoWI7ktWVlZRSUnJXyaTSZp+Xbx4sT4rK+uis+cjIhd3b2QYFyLqUNbIMC5EpIns7Oy92dnZxc6eg4iIiIiIiIiIiIiIiIiIiIiIqDP9H4eoTTTtUgWtAAAAAElFTkSuQmCC", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "document_store = InMemoryDocumentStore()\n", + "\n", + "indexing = Pipeline()\n", + "indexing.add_component(\n", + " \"file_converter\",\n", + " DataFrameFileToDocument(\n", + " content_column=\"summary\",\n", + " meta_columns=[\"title\", \"authors\", \"published\", \"primary_category\", \"categories\", \"pdf_url\"],\n", + " index_column=\"entry_id\",\n", + " file_format=\"parquet\",\n", + " backend=\"polars\",\n", + " ),\n", + ")\n", + "indexing.add_component(\"writer\", DocumentWriter(document_store))\n", + "indexing.connect(\"file_converter\", \"writer\")\n", + "indexing.show()" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'writer': {'documents_written': 10}}" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "indexing.run({\"file_converter\": {\"file_paths\": [temp_file_parquet.name]}})" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Saving the data in a temporary file." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [], + "source": [ + "# Cleanup\n", + "# delete the temporary file\n", + "from pathlib import Path\n", + "\n", + "Path(temp_file_parquet.name).unlink()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "dataframes-haystack", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.19" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/pyproject.toml b/pyproject.toml index 9b6780b..f87ea03 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -9,7 +9,16 @@ description = "Haystack custom components for your favourite dataframe library." readme = "README.md" requires-python = ">=3.8" license = { file = "LICENSE" } -keywords = ["nlp", "machine-learning", "ai", "haystack", "pandas", "dataframe", "polars", "llm"] +keywords = [ + "nlp", + "machine-learning", + "ai", + "haystack", + "pandas", + "dataframe", + "polars", + "llm", +] authors = [{ name = "Edoardo Abati" }] classifiers = [ "License :: OSI Approved :: MIT License", @@ -26,6 +35,7 @@ classifiers = [ dependencies = [ "haystack-ai>=2.0.0", + "narwhals>=1.1.0", "typing_extensions", ] [project.optional-dependencies] @@ -42,7 +52,7 @@ path = "src/dataframes_haystack/__about__.py" # Default environment [tool.hatch.envs.default] -installer="uv" +installer = "uv" dependencies = [ "coverage[toml]>=6.5", "pytest", @@ -88,8 +98,17 @@ check = "mypy --install-types --non-interactive {args:src/dataframes_haystack te detached = true dependencies = ["black>=24.3.0", "nbqa>=1.8.5", "ruff>=0.3.4"] [tool.hatch.envs.lint.scripts] -style = ["ruff check {args:.}", "black --check --diff {args:.}", "nbqa black --check --diff notebooks/*"] -fmt = ["black {args:.}", "ruff check --fix {args:.}", "nbqa black notebooks/*", "style"] +style = [ + "ruff check {args:.}", + "black --check --diff {args:.}", + "nbqa black --check --diff notebooks/*", +] +fmt = [ + "black {args:.}", + "ruff check --fix {args:.}", + "nbqa black notebooks/*", + "style", +] [tool.black] target-version = ["py38"] @@ -102,50 +121,28 @@ line-length = 120 extend-include = ["*.ipynb"] [tool.ruff.lint] -select = [ - "A", - "ARG", - "B", - "C", - "DTZ", - "E", - "EM", - "F", - "I", - "ICN", - "ISC", - "N", - "PLC", - "PLE", - "PLR", - "PLW", - "Q", - "RUF", - "S", - "T", - "TID", - "UP", - "W", - "YTT", -] +select = ["ALL"] ignore = [ # Allow non-abstract empty methods in abstract base classes "B027", + # No required doctstring for modules, packages + "D100", + "D104", + # No future annotations + "FA100", # Ignore checks for possible passwords "S105", "S106", "S107", # Ignore complexity "C901", + # Generic veriable name df is ok + "PD901", "PLR0911", "PLR0912", "PLR0913", "PLR0915", ] -unfixable = [ - # Don't touch unused imports - "F401", -] [tool.ruff.lint.isort] known-first-party = ["dataframes_haystack"] @@ -153,9 +150,16 @@ known-first-party = ["dataframes_haystack"] [tool.ruff.lint.flake8-tidy-imports] ban-relative-imports = "all" +[tool.ruff.lint.pydocstyle] +convention = "google" + +[tool.ruff.format] +docstring-code-format = true + [tool.ruff.lint.per-file-ignores] # Tests can use magic values, assertions, and relative imports -"tests/**/*" = ["PLR2004", "S101", "TID252"] +"tests/*" = ["PLR2004", "S101", "TID252", "D100", "D103"] +"notebooks/*" = ["PTH123", "SIM115"] # Test coverage @@ -168,7 +172,10 @@ omit = [ ] [tool.coverage.paths] -dataframes_haystack = ["src/dataframes_haystack", "*/dataframes-haystack/src/dataframes_haystack"] +dataframes_haystack = [ + "src/dataframes_haystack", + "*/dataframes-haystack/src/dataframes_haystack", +] tests = ["tests", "*/dataframes-haystack/tests"] [tool.coverage.report] diff --git a/src/dataframes_haystack/components/converters/__init__.py b/src/dataframes_haystack/components/converters/__init__.py index e69de29..0b45612 100644 --- a/src/dataframes_haystack/components/converters/__init__.py +++ b/src/dataframes_haystack/components/converters/__init__.py @@ -0,0 +1,5 @@ +from dataframes_haystack.components.converters._common import DataFrameFileToDocument + +__all__ = [ + "DataFrameFileToDocument", +] diff --git a/src/dataframes_haystack/components/converters/_common.py b/src/dataframes_haystack/components/converters/_common.py new file mode 100644 index 0000000..3a4fcf6 --- /dev/null +++ b/src/dataframes_haystack/components/converters/_common.py @@ -0,0 +1,112 @@ +import logging +from functools import partial +from typing import Any, Dict, List, Literal, Optional, Union + +import narwhals.stable.v1 as nw +from haystack import Document, component + +from dataframes_haystack.components.converters._utils import ( + FileFormat, + ReaderFunc, + frame_to_documents, + get_pandas_readers_map, + get_polars_readers_map, + read_with_select, +) + +logger = logging.getLogger(__name__) + +Backends = Literal["pandas", "polars"] + + +@component +class DataFrameFileToDocument: + """Reads files and converts their data in Documents. + + Usage example: + ```python + from dataframes_haystack.components.converters import DataFrameFileToDocument + + converter = DataFrameFileToDocument(content_column="text_str") + results = converter.run(files=["file1.csv", "file2.csv"]) + documents = results["documents"] + print(documents[0].content) + ``` + """ + + def __init__( + self, + content_column: str, + meta_columns: Union[List[str], None] = None, + index_column: Union[str, None] = None, + file_format: FileFormat = "csv", + read_kwargs: Optional[Dict[str, Any]] = None, + backend: Backends = "polars", + ) -> None: + """Create a DataFrameFileToDocument component. + + Args: + content_column: The name of the DataFrame column that contains the text content. + meta_columns: Optional list of names of the DataFrame columns that contain metadata. + index_column: The name of the DataFrame column that contains the index. + file_format: The format of the files to read. + read_kwargs: Optional keyword arguments to pass to the file reader function. + backend: The backend to use for reading the files. + """ + self.content_column = content_column + self.meta_columns = meta_columns or [] + self.index_column = index_column + self.file_format = file_format + self.read_kwargs = read_kwargs or {} + self.backend = backend + if self.backend not in ["pandas", "polars"]: + msg = f"Unsupported backend: {self.backend}" + raise ValueError(msg) + self._reader_function = self._get_reader_function() + + def _get_reader_function(self) -> ReaderFunc: + file_format_mapping = get_pandas_readers_map() if self.backend == "pandas" else get_polars_readers_map() + reader_function = file_format_mapping.get(self.file_format) + if reader_function: + return reader_function + msg = f"Unsupported file format for {self.backend} backend: {self.file_format}" + raise ValueError(msg) + + def _run_read(self, file_paths: List[str]) -> nw.DataFrame: + selected_columns = [self.index_column, self.content_column, *self.meta_columns] + selected_columns = [col for col in selected_columns if col is not None] + read_func = partial(self._reader_function, **self.read_kwargs) + df_list = [read_with_select(read_func, file_path=path, columns_subset=selected_columns) for path in file_paths] + return nw.concat(df_list, how="vertical") + + @component.output_types(documents=List[Document]) + def run( + self, + file_paths: List[str], + meta: Union[Dict[str, Any], List[Dict[str, Any]], None] = None, + ) -> Dict[str, List[Document]]: + """Reads files and converts their data in Documents. + + Args: + file_paths: List of file paths to read. + meta: + Optional metadata to attach to the Documents. + This value can be either a dictionary or a list of dictionaries. + If it's a dictionary, its content is added to the metadata of all produced Documents. + If it's a list, the length of the list must match the number of rows in the DataFrame, + because the two lists will be zipped. + + Returns: + A dictionary with the following keys: + - `documents`: Created Documents + """ + df = self._run_read(file_paths) + documents = frame_to_documents( + df, + content_column=self.content_column, + meta_columns=self.meta_columns, + index_column=self.index_column, + extra_metadata=meta, + ) + + return {"documents": documents} diff --git a/src/dataframes_haystack/components/converters/_utils.py b/src/dataframes_haystack/components/converters/_utils.py new file mode 100644 index 0000000..391790f --- /dev/null +++ b/src/dataframes_haystack/components/converters/_utils.py @@ -0,0 +1,81 @@ +from typing import Any, Callable, Dict, List, Literal, Union + +import narwhals.stable.v1 as nw +from haystack import Document +from haystack.components.converters.utils import normalize_metadata +from narwhals.typing import IntoDataFrame + +ReaderFunc = Callable[..., IntoDataFrame] +PandasFileFormat = Literal["csv", "fwf", "json", "html", "xml", "excel", "feather", "parquet", "orc", "pickle"] +PolarsFileFormat = Literal["avro", "csv", "delta", "excel", "ipc", "json", "parquet"] + +FileFormat = Union[PandasFileFormat, PolarsFileFormat] + + +def read_with_select( + reader_function: ReaderFunc, + file_path: str, + columns_subset: Union[List[str], None] = None, +) -> nw.DataFrame: + df = reader_function(file_path) + df = nw.from_native(df, eager_only=True) + if columns_subset: + df = df.select(columns_subset) + return df + + +def frame_to_documents( + df: nw.DataFrame, + *, + content_column: str, + meta_columns: Union[List[str], None] = None, + index_column: Union[str, None] = None, + extra_metadata: Union[Dict[str, Any], List[Dict[str, Any]], None] = None, +) -> List[Document]: + meta_list = normalize_metadata(extra_metadata, sources_count=df.shape[0]) + documents = [] + for i, row in enumerate(df.iter_rows(named=True)): + doc_id = str(row.pop(index_column)) if index_column else None + content = row.pop(content_column) + meta_row = {k: v for k, v in row.items() if k in meta_columns} if meta_columns else {} + metadata = {**meta_row, **meta_list[i]} if meta_list else meta_row + doc = Document(id=doc_id, content=content, meta=metadata) + documents.append(doc) + return documents + + +def get_polars_readers_map() -> Dict[str, ReaderFunc]: # pragma: no cover + try: + import polars as pl + except ImportError as e: + msg = "`polars` is not installed. Please run 'pip install \"dataframes-haystack[polars]\"'" + raise ImportError(msg) from e + + return { + "avro": pl.read_avro, + "csv": pl.read_csv, + "delta": pl.read_delta, + "excel": pl.read_excel, + "ipc": pl.read_ipc, + "json": pl.read_json, + "parquet": pl.read_parquet, + } + + +def get_pandas_readers_map() -> Dict[str, ReaderFunc]: # pragma: no cover + import pandas as pd + + return { + "csv": pd.read_csv, + "fwf": pd.read_fwf, + "json": pd.read_json, + "html": pd.read_html, + "xml": pd.read_xml, + "excel": pd.read_excel, + "feather": pd.read_feather, + "parquet": pd.read_parquet, + "orc": pd.read_orc, + "pickle": pd.read_pickle, + "sql": pd.read_sql, + "gbq": pd.read_gbq, + } diff --git a/src/dataframes_haystack/components/converters/pandas.py b/src/dataframes_haystack/components/converters/pandas.py index 9f50d14..06e26c2 100644 --- a/src/dataframes_haystack/components/converters/pandas.py +++ b/src/dataframes_haystack/components/converters/pandas.py @@ -1,18 +1,23 @@ -from typing import Any, Dict, List, Literal, Optional, Union +from functools import partial +from typing import Any, Callable, Dict, List, Optional, Union +import narwhals.stable.v1 as nw import pandas as pd from haystack import Document, component, logging -from haystack.components.converters.utils import normalize_metadata -logger = logging.getLogger(__name__) +from dataframes_haystack.components.converters._utils import PandasFileFormat as FileFormat +from dataframes_haystack.components.converters._utils import ( + frame_to_documents, + get_pandas_readers_map, + read_with_select, +) -FileFormat = Literal["csv", "fwf", "json", "html", "xml", "excel", "feather", "parquet", "orc", "pickle"] +logger = logging.getLogger(__name__) @component class FileToPandasDataFrame: - """ - Converts files to a pandas.DataFrame. + """Converts files to a pandas.DataFrame. Usage example: ```python @@ -30,9 +35,8 @@ def __init__( file_format: FileFormat = "csv", read_kwargs: Union[Dict[str, Any], None] = None, columns_subset: Union[List[str], None] = None, - ): - """ - Create a FileToPandasDataFrame component. + ) -> None: + """Create a FileToPandasDataFrame component. Please refer to the pandas documentation for more information on the supported readers and their parameters: https://pandas.pydata.org/docs/user_guide/io.html @@ -47,40 +51,23 @@ def __init__( self.read_kwargs = read_kwargs or {} self.columns_subset = columns_subset - def _get_read_function(self): + def _get_read_function(self) -> Callable[..., pd.DataFrame]: """Returns the function to read files based on the file format.""" - - file_format_mapping = { - "csv": pd.read_csv, - "fwf": pd.read_fwf, - "json": pd.read_json, - "html": pd.read_html, - "xml": pd.read_xml, - "excel": pd.read_excel, - "feather": pd.read_feather, - "parquet": pd.read_parquet, - "orc": pd.read_orc, - "pickle": pd.read_pickle, - "sql": pd.read_sql, - "gbq": pd.read_gbq, - } + file_format_mapping = get_pandas_readers_map() reader_function = file_format_mapping.get(self.file_format) if reader_function: return reader_function msg = f"Unsupported file format: {self.file_format}" raise ValueError(msg) - def _read_with_select(self, file_path: str) -> pd.DataFrame: + def _read_with_select(self, file_path: str) -> nw.DataFrame: """Reads a file and selects a subset of columns, if provided.""" - df = self._reader_function(file_path, **self.read_kwargs) - if self.columns_subset: - return df[self.columns_subset] - return df + read_func = partial(self._reader_function, **self.read_kwargs) + return read_with_select(read_func, file_path, self.columns_subset) @component.output_types(dataframe=pd.DataFrame) def run(self, file_paths: List[str]) -> Dict[str, pd.DataFrame]: - """ - Converts files to a pandas.DataFrame. + """Converts files to a pandas.DataFrame. Args: file_paths: List of file paths. @@ -90,14 +77,14 @@ def run(self, file_paths: List[str]) -> Dict[str, pd.DataFrame]: - `dataframe`: pandas.DataFrame containing the content of the files. """ df_list = [self._read_with_select(path) for path in file_paths] - df = pd.concat(df_list, ignore_index=True) - return {"dataframe": df} + df = nw.concat(df_list, how="vertical") + pandas_df = nw.to_native(df) + return {"dataframe": pandas_df} @component class PandasDataFrameConverter: - """ - Converts data in a pandas.DataFrame to Documents. + """Converts data in a pandas.DataFrame to Documents. Usage example: ```python @@ -114,10 +101,9 @@ def __init__( self, content_column: str, meta_columns: Optional[List[str]] = None, - use_index_as_id: bool = False, - ): - """ - Create a PandasDataFrameConverter component. + use_index_as_id: bool = False, # noqa: FBT001, FBT002 + ) -> None: + """Create a PandasDataFrameConverter component. Args: content_column: The name of the column in the DataFrame that contains the text content. @@ -130,19 +116,15 @@ def __init__( def _is_compatible_index(self, dataframe: pd.DataFrame) -> bool: """Returns True if the index of the DataFrame can be used as the ID of the Documents.""" - if self.use_index_as_id: - if isinstance(dataframe.index, pd.MultiIndex): - return False - return True + return not (self.use_index_as_id and isinstance(dataframe.index, pd.MultiIndex)) @component.output_types(documents=List[Document]) def run( self, dataframe: pd.DataFrame, meta: Optional[Union[Dict[str, Any], List[Dict[str, Any]]]] = None, - ): - """ - Converts text files to Documents. + ) -> Dict[str, List[Document]]: + """Converts text files to Documents. Args: dataframe: @@ -164,24 +146,22 @@ def run( "Please make sure that the index is not a MultiIndex or set `use_index_as_id` to False." ) raise ValueError(msg) - index_col_name = "index" - meta_list = normalize_metadata(meta, sources_count=dataframe.shape[0]) - - selected_columns = [self.content_column, *self.meta_columns] - - data_rows = dataframe[selected_columns].to_dict(orient="records") if self.use_index_as_id: - indexes = dataframe.index.to_list() - data_rows = [{index_col_name: str(idx), **row} for idx, row in zip(indexes, data_rows)] - - documents = [] - for i, row in enumerate(data_rows): - doc_id = row.pop(index_col_name) if self.use_index_as_id else None - content = row.pop(self.content_column) - meta_row = {k: v for k, v in row.items() if k in self.meta_columns} if self.meta_columns else {} - metadata = {**meta_row, **meta_list[i]} if meta_list else meta_row - doc = Document(id=doc_id, content=content, meta=metadata) - documents.append(doc) - + index_col_name = "__temp_index_col__" + dataframe = dataframe.assign(**{index_col_name: dataframe.index.astype(str)}) + selected_columns = [index_col_name, self.content_column, *self.meta_columns] + else: + index_col_name = None + selected_columns = [self.content_column, *self.meta_columns] + + df = nw.from_native(dataframe, eager_only=True) + df = df.select(selected_columns) + documents = frame_to_documents( + df, + content_column=self.content_column, + meta_columns=self.meta_columns, + index_column=index_col_name, + extra_metadata=meta, + ) return {"documents": documents} diff --git a/src/dataframes_haystack/components/converters/polars.py b/src/dataframes_haystack/components/converters/polars.py index 8b6da14..f62be7e 100644 --- a/src/dataframes_haystack/components/converters/polars.py +++ b/src/dataframes_haystack/components/converters/polars.py @@ -1,7 +1,15 @@ -from typing import Any, Dict, List, Literal, Optional, Union +from functools import partial +from typing import Any, Callable, Dict, List, Optional, Union +import narwhals.stable.v1 as nw from haystack import Document, component, logging -from haystack.components.converters.utils import normalize_metadata + +from dataframes_haystack.components.converters._utils import PolarsFileFormat as FileFormat +from dataframes_haystack.components.converters._utils import ( + frame_to_documents, + get_polars_readers_map, + read_with_select, +) try: import polars as pl @@ -12,13 +20,9 @@ logger = logging.getLogger(__name__) -FileFormat = Literal["avro", "csv", "delta", "excel", "ipc", "json", "parquet"] - - @component class FileToPolarsDataFrame: - """ - Converts files to a polars.DataFrame. + """Converts files to a polars.DataFrame. Usage example: ```python @@ -36,9 +40,8 @@ def __init__( file_format: FileFormat = "csv", read_kwargs: Optional[Dict[str, Any]] = None, columns_subset: Union[List[str], None] = None, - ): - """ - Create a FileToPolarsDataFrame component. + ) -> None: + """Create a FileToPolarsDataFrame component. Please refer to the polars documentation for more information on the supported readers and their parameters: https://docs.pola.rs/api/python/stable/reference/io.html @@ -53,35 +56,23 @@ def __init__( self.read_kwargs = read_kwargs or {} self.columns_subset = columns_subset - def _get_read_function(self): + def _get_read_function(self) -> Callable[..., pl.DataFrame]: """Returns the function to read files based on the file format.""" - - file_format_mapping = { - "avro": pl.read_avro, - "csv": pl.read_csv, - "delta": pl.read_delta, - "excel": pl.read_excel, - "ipc": pl.read_ipc, - "json": pl.read_json, - "parquet": pl.read_parquet, - } + file_format_mapping = get_polars_readers_map() reader_function = file_format_mapping.get(self.file_format) if reader_function: return reader_function msg = f"Unsupported file format: {self.file_format}" raise ValueError(msg) - def _read_with_select(self, file_path: str) -> pl.DataFrame: + def _read_with_select(self, file_path: str) -> nw.DataFrame: """Reads a file and selects only the specified columns.""" - df = self._reader_function(file_path, **self.read_kwargs) - if self.columns_subset: - return df.select(self.columns_subset) - return df + read_func = partial(self._reader_function, **self.read_kwargs) + return read_with_select(read_func, file_path, self.columns_subset) @component.output_types(dataframe=pl.DataFrame) - def run(self, file_paths: List[str]): - """ - Converts files to a polars.DataFrame. + def run(self, file_paths: List[str]) -> Dict[str, pl.DataFrame]: + """Converts files to a polars.DataFrame. Args: file_paths: List of file paths to read. @@ -91,14 +82,14 @@ def run(self, file_paths: List[str]): - `dataframe`: The polars.DataFrame created from the files. """ df_list = [self._read_with_select(path) for path in file_paths] - df = pl.concat(df_list, how="vertical") - return {"dataframe": df} + df = nw.concat(df_list, how="vertical") + polars_df = nw.to_native(df) + return {"dataframe": polars_df} @component class PolarsDataFrameConverter: - """ - Converts data in a polars.DataFrame to Documents. + """Converts data in a polars.DataFrame to Documents. Usage example: ```python @@ -116,9 +107,8 @@ def __init__( content_column: str, meta_columns: Union[List[str], None] = None, index_column: Union[str, None] = None, - ): - """ - Create a PolarsDataFrameConverter component. + ) -> None: + """Create a PolarsDataFrameConverter component. Args: content_column: The name of the column in the DataFrame that contains the text content. @@ -134,9 +124,8 @@ def run( self, dataframe: pl.DataFrame, meta: Optional[Union[Dict[str, Any], List[Dict[str, Any]]]] = None, - ): - """ - Converts data in a polars.DataFrame to Documents. + ) -> Dict[str, List[Document]]: + """Converts data in a polars.DataFrame to Documents. Args: dataframe: @@ -152,18 +141,15 @@ def run( A dictionary with the following keys: - `documents`: Created Documents """ - meta_list = normalize_metadata(meta, sources_count=dataframe.shape[0]) - + df = nw.from_native(dataframe) selected_columns = [self.index_column, self.content_column, *self.meta_columns] - data_rows = dataframe.select(selected_columns).to_dicts() - - documents = [] - for i, row in enumerate(data_rows): - doc_id = str(row.pop(self.index_column)) if self.index_column else None - content = row.pop(self.content_column) - meta_row = {k: v for k, v in row.items() if k in self.meta_columns} if self.meta_columns else {} - metadata = {**meta_row, **meta_list[i]} if meta_list else meta_row - doc = Document(id=doc_id, content=content, meta=metadata) - documents.append(doc) - + selected_columns = [col for col in selected_columns if col is not None] + df = df.select(selected_columns) + documents = frame_to_documents( + df, + content_column=self.content_column, + meta_columns=self.meta_columns, + index_column=self.index_column, + extra_metadata=meta, + ) return {"documents": documents} diff --git a/tests/conftest.py b/tests/conftest.py index 139cddf..d8a40c4 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -3,6 +3,7 @@ import pandas as pd import polars as pl import pytest +from narwhals.typing import IntoDataFrame DATA = { "content": ["content1", "content2"], @@ -16,16 +17,29 @@ """ -@pytest.fixture(scope="function") -def pandas_dataframe(): +def _pandas_df() -> pd.DataFrame: return pd.DataFrame(data=DATA, index=[0, 1]) -@pytest.fixture(scope="function") -def polars_dataframe(): +def _polars_df() -> pl.DataFrame: return pl.DataFrame(data=DATA) +@pytest.fixture +def pandas_dataframe() -> pd.DataFrame: + return _pandas_df() + + +@pytest.fixture +def polars_dataframe() -> pl.DataFrame: + return _polars_df() + + +@pytest.fixture(params=[_pandas_df, _polars_df]) +def dataframe(request: pytest.FixtureRequest) -> IntoDataFrame: + return request.param() + + @pytest.fixture(scope="session") def csv_file_path(tmp_path_factory: pytest.TempPathFactory) -> Path: tmp_path = tmp_path_factory.mktemp("data") / "data.csv" diff --git a/tests/test_converters_common.py b/tests/test_converters_common.py new file mode 100644 index 0000000..443d379 --- /dev/null +++ b/tests/test_converters_common.py @@ -0,0 +1,120 @@ +from pathlib import Path +from typing import Any, Dict, List, Union + +import pytest +from haystack import Document + +from dataframes_haystack.components.converters import DataFrameFileToDocument +from tests.utils import assert_pipeline_yaml_equal + + +@pytest.mark.parametrize("backend", ["pandas", "polars"]) +@pytest.mark.parametrize( + ("meta_columns", "index_column", "read_kwargs", "expected_docs"), + [ + ( + None, + None, + None, + [ + Document(content="content1", meta={}), + Document(content="content2", meta={}), + ], + ), + ( + ["meta2"], + None, + None, + [ + Document(content="content1", meta={"meta2": "meta2_1"}), + Document(content="content2", meta={"meta2": "meta2_2"}), + ], + ), + ( + ["meta2", "meta1"], + None, + None, + [ + Document(content="content1", meta={"meta2": "meta2_1", "meta1": "meta1_1"}), + Document(content="content2", meta={"meta2": "meta2_2", "meta1": "meta1_2"}), + ], + ), + ( + ["meta2"], + "meta1", + None, + [ + Document(id="meta1_1", content="content1", meta={"meta2": "meta2_1"}), + Document(id="meta1_2", content="content2", meta={"meta2": "meta2_2"}), + ], + ), + (["meta2"], None, {"n_rows": 1}, [Document(content="content1", meta={"meta2": "meta2_1"})]), + ], +) +def test_dataframe_file_to_document( + csv_file_path: Path, + meta_columns: Union[List[str], None], + index_column: Union[str, None], + read_kwargs: Union[Dict[str, Any], None], + expected_docs: List[Document], + backend: str, +) -> None: + if backend == "pandas" and read_kwargs and read_kwargs.get("n_rows"): + read_kwargs = {"nrows": read_kwargs["n_rows"]} + converter = DataFrameFileToDocument( + content_column="content", + meta_columns=meta_columns, + index_column=index_column, + read_kwargs=read_kwargs, + backend=backend, + ) + results = converter.run(file_paths=[str(csv_file_path)]) + documents = results["documents"] + for output_doc, expected_doc in zip(documents, expected_docs): + assert output_doc.content == expected_doc.content + assert output_doc.meta == expected_doc.meta + if index_column: + assert output_doc.id == expected_doc.id + + +@pytest.mark.parametrize( + ("kwargs", "error_msg"), + [ + ({"backend": "foo"}, "Unsupported backend: foo"), + ({"backend": "pandas", "file_format": "foo"}, "Unsupported file format for pandas backend: foo"), + ({"backend": "polars", "file_format": "foo"}, "Unsupported file format for polars backend: foo"), + ], +) +def test_dataframe_file_to_document_valueerror(kwargs: Dict[str, Any], error_msg: str) -> None: + with pytest.raises(ValueError, match=error_msg): + DataFrameFileToDocument(content_column="a", **kwargs) + + +def test_converter_in_pipeline() -> None: + from textwrap import dedent + + from haystack.components.preprocessors import DocumentCleaner + from haystack.core.pipeline import Pipeline + + pipeline = Pipeline() + pipeline.add_component("converter", DataFrameFileToDocument(content_column="content")) + pipeline.add_component("cleaner", DocumentCleaner()) + pipeline.connect("converter", "cleaner") + + yaml_pipeline = pipeline.dumps() + + converter_expected_yaml = """\ + converter: + init_parameters: + backend: polars + content_column: content + file_format: csv + index_column: null + meta_columns: [] + read_kwargs: {} + type: dataframes_haystack.components.converters._common.DataFrameFileToDocument + """ + assert dedent(converter_expected_yaml) in yaml_pipeline + + new_pipeline = Pipeline.loads(yaml_pipeline) + assert_pipeline_yaml_equal(yaml_pipeline, new_pipeline.dumps()) diff --git a/tests/test_converters_pandas.py b/tests/test_converters_pandas.py index 42dbfed..30f6ff4 100644 --- a/tests/test_converters_pandas.py +++ b/tests/test_converters_pandas.py @@ -9,7 +9,7 @@ from tests.utils import assert_pipeline_yaml_equal -def test_pandas_dataframe_default_converter(pandas_dataframe: pd.DataFrame): +def test_pandas_dataframe_default_converter(pandas_dataframe: pd.DataFrame) -> None: converter = PandasDataFrameConverter(content_column="content") results = converter.run(dataframe=pandas_dataframe) documents = results["documents"] @@ -21,7 +21,10 @@ def test_pandas_dataframe_default_converter(pandas_dataframe: pd.DataFrame): @pytest.mark.parametrize("use_index_as_id", [True, False]) -def test_pandas_dataframe_converter_use_index_as_id(pandas_dataframe: pd.DataFrame, use_index_as_id: bool): +def test_pandas_dataframe_converter_use_index_as_id( + pandas_dataframe: pd.DataFrame, + use_index_as_id: bool, # noqa: FBT001 +) -> None: converter = PandasDataFrameConverter(content_column="content", use_index_as_id=use_index_as_id) results = converter.run(dataframe=pandas_dataframe) documents = results["documents"] @@ -35,7 +38,7 @@ def test_pandas_dataframe_converter_use_index_as_id(pandas_dataframe: pd.DataFra @pytest.mark.parametrize( - "meta_columns, expected_meta", + ("meta_columns", "expected_meta"), [ (["meta1"], [{"meta1": "meta1_1"}, {"meta1": "meta1_2"}]), (["meta2"], [{"meta2": "meta2_1"}, {"meta2": "meta2_2"}]), @@ -52,7 +55,7 @@ def test_pandas_dataframe_converter_meta_columns( pandas_dataframe: pd.DataFrame, meta_columns: List[str], expected_meta: List[Dict[str, str]], -): +) -> None: converter = PandasDataFrameConverter(content_column="content", meta_columns=meta_columns) results = converter.run(dataframe=pandas_dataframe) documents = results["documents"] @@ -62,7 +65,7 @@ def test_pandas_dataframe_converter_meta_columns( @pytest.mark.parametrize( - "meta, expected_meta", + ("meta", "expected_meta"), [ ( [{"extra_meta_1": "value1"}, {"extra_meta_2": "value2"}], @@ -84,7 +87,7 @@ def test_pandas_dataframe_converter_all_metadata( pandas_dataframe: pd.DataFrame, meta: Union[Dict[str, Any], List[Dict[str, Any]]], expected_meta: List[Dict[str, str]], -): +) -> None: converter = PandasDataFrameConverter(content_column="content", meta_columns=["meta1"]) results = converter.run(dataframe=pandas_dataframe, meta=meta) documents = results["documents"] @@ -95,17 +98,19 @@ def test_pandas_dataframe_converter_all_metadata( def test_pandas_dataframe_converters_multindex_error( pandas_dataframe: pd.DataFrame, -): +) -> None: pandas_dataframe.index = pd.MultiIndex.from_tuples([("a", 0), ("b", 1)]) converter = PandasDataFrameConverter(content_column="content", use_index_as_id=True) - with pytest.raises(ValueError): + with pytest.raises(ValueError, match="The index of the DataFrame cannot be used"): converter.run(dataframe=pandas_dataframe) @pytest.mark.parametrize("column_subset", [None, ["content"], ["content", "meta1"]]) def test_file_to_pandas_converter( - csv_file_path: Path, pandas_dataframe: pd.DataFrame, column_subset: Union[List[str], None] -): + csv_file_path: Path, + pandas_dataframe: pd.DataFrame, + column_subset: Union[List[str], None], +) -> None: converter = FileToPandasDataFrame(columns_subset=column_subset) results = converter.run(file_paths=[str(csv_file_path)]) if column_subset: @@ -113,19 +118,19 @@ def test_file_to_pandas_converter( assert_frame_equal(results["dataframe"], pandas_dataframe) -def test_file_to_pandas_converter_read_kwargs(csv_file_path: Path, pandas_dataframe: pd.DataFrame): +def test_file_to_pandas_converter_read_kwargs(csv_file_path: Path, pandas_dataframe: pd.DataFrame) -> None: cols_to_select = ["content", "meta2"] converter = FileToPandasDataFrame(read_kwargs={"usecols": cols_to_select}) results = converter.run(file_paths=[str(csv_file_path)]) assert_frame_equal(results["dataframe"], pandas_dataframe[cols_to_select]) -def test_file_to_pandas_converter_valueerror(): - with pytest.raises(ValueError): +def test_file_to_pandas_converter_valueerror() -> None: + with pytest.raises(ValueError, match="Unsupported file format"): FileToPandasDataFrame(file_format="foo") -def test_converter_in_pipeline(): +def test_converter_in_pipeline() -> None: from textwrap import dedent from haystack.components.preprocessors import DocumentCleaner diff --git a/tests/test_converters_polars.py b/tests/test_converters_polars.py index cc8ae0f..c69b8a1 100644 --- a/tests/test_converters_polars.py +++ b/tests/test_converters_polars.py @@ -9,18 +9,7 @@ from tests.utils import assert_pipeline_yaml_equal -@pytest.fixture(scope="function") -def polars_dataframe(): - return pl.DataFrame( - data={ - "content": ["content1", "content2"], - "meta1": ["meta1_1", "meta1_2"], - "meta2": ["meta2_1", "meta2_2"], - }, - ) - - -def test_polars_dataframe_default_converter(polars_dataframe: pl.DataFrame): +def test_polars_dataframe_default_converter(polars_dataframe: pl.DataFrame) -> None: converter = PolarsDataFrameConverter(content_column="content") results = converter.run(dataframe=polars_dataframe) documents = results["documents"] @@ -31,7 +20,7 @@ def test_polars_dataframe_default_converter(polars_dataframe: pl.DataFrame): assert [doc.embedding for doc in documents] == [None, None] -def test_polars_dataframe_converter_index_column(polars_dataframe: pl.DataFrame): +def test_polars_dataframe_converter_index_column(polars_dataframe: pl.DataFrame) -> None: polars_dataframe = polars_dataframe.with_columns(pl.Series("index", [0, 1])) converter = PolarsDataFrameConverter(content_column="content", index_column="index") results = converter.run(dataframe=polars_dataframe) @@ -42,7 +31,7 @@ def test_polars_dataframe_converter_index_column(polars_dataframe: pl.DataFrame) @pytest.mark.parametrize( - "meta_columns, expected_meta", + ("meta_columns", "expected_meta"), [ (["meta1"], [{"meta1": "meta1_1"}, {"meta1": "meta1_2"}]), (["meta2"], [{"meta2": "meta2_1"}, {"meta2": "meta2_2"}]), @@ -59,7 +48,7 @@ def test_polars_dataframe_converter_meta_columns( polars_dataframe: pl.DataFrame, meta_columns: List[str], expected_meta: List[Dict[str, str]], -): +) -> None: converter = PolarsDataFrameConverter(content_column="content", meta_columns=meta_columns) results = converter.run(dataframe=polars_dataframe) documents = results["documents"] @@ -69,7 +58,7 @@ def test_polars_dataframe_converter_meta_columns( @pytest.mark.parametrize( - "meta, expected_meta", + ("meta", "expected_meta"), [ ( [{"extra_meta_1": "value1"}, {"extra_meta_2": "value2"}], @@ -99,7 +88,7 @@ def test_polars_dataframe_converter_all_metadata( polars_dataframe: pl.DataFrame, meta: Union[Dict[str, Any], List[Dict[str, Any]]], expected_meta: List[Dict[str, str]], -): +) -> None: converter = PolarsDataFrameConverter(content_column="content", meta_columns=["meta1"]) results = converter.run(dataframe=polars_dataframe, meta=meta) documents = results["documents"] @@ -110,8 +99,10 @@ def test_polars_dataframe_converter_all_metadata( @pytest.mark.parametrize("column_subset", [None, ["content"], ["content", "meta1"]]) def test_file_to_polars_converter( - csv_file_path: Path, polars_dataframe: pl.DataFrame, column_subset: Union[List[str], None] -): + csv_file_path: Path, + polars_dataframe: pl.DataFrame, + column_subset: Union[List[str], None], +) -> None: converter = FileToPolarsDataFrame(columns_subset=column_subset) results = converter.run(file_paths=[str(csv_file_path)]) if column_subset: @@ -119,19 +110,19 @@ def test_file_to_polars_converter( assert_frame_equal(results["dataframe"], polars_dataframe) -def test_file_to_polars_converter_read_kwargs(csv_file_path: Path, polars_dataframe: pl.DataFrame): +def test_file_to_polars_converter_read_kwargs(csv_file_path: Path, polars_dataframe: pl.DataFrame) -> None: cols_to_select = ["content", "meta2"] converter = FileToPolarsDataFrame(read_kwargs={"columns": cols_to_select}) results = converter.run(file_paths=[str(csv_file_path)]) assert_frame_equal(results["dataframe"], polars_dataframe.select(cols_to_select)) -def test_file_to_polars_converter_valueerror(): - with pytest.raises(ValueError): +def test_file_to_polars_converter_valueerror() -> None: + with pytest.raises(ValueError, match="Unsupported file format"): FileToPolarsDataFrame(file_format="foo") -def test_converter_in_pipeline(): +def test_converter_in_pipeline() -> None: from textwrap import dedent from haystack.components.preprocessors import DocumentCleaner diff --git a/tests/test_converters_utils.py b/tests/test_converters_utils.py new file mode 100644 index 0000000..2e9c661 --- /dev/null +++ b/tests/test_converters_utils.py @@ -0,0 +1,97 @@ +from pathlib import Path +from typing import Any, Dict, List, Union + +import narwhals as nw +import pandas as pd +import polars as pl +import pytest +from haystack import Document +from narwhals.typing import IntoDataFrame + +from dataframes_haystack.components.converters._utils import ReaderFunc, frame_to_documents, read_with_select + + +@pytest.mark.parametrize("reader_func", [pd.read_csv, pl.read_csv]) +def test_read_with_select(reader_func: ReaderFunc, csv_file_path: Path) -> None: + df = read_with_select(reader_func, str(csv_file_path)) + assert df.shape == (2, 3) + output = {col: series.to_list() for col, series in df.to_dict().items()} + expected = { + "content": ["content1", "content2"], + "meta1": ["meta1_1", "meta1_2"], + "meta2": ["meta2_1", "meta2_2"], + } + assert output == expected + + +@pytest.mark.parametrize("reader_func", [pd.read_csv, pl.read_csv]) +def test_read_with_select_subset(reader_func: ReaderFunc, csv_file_path: Path) -> None: + df = read_with_select(reader_func, str(csv_file_path), columns_subset=["content", "meta2"]) + assert df.shape == (2, 2) + output = {col: series.to_list() for col, series in df.to_dict().items()} + expected = { + "content": ["content1", "content2"], + "meta2": ["meta2_1", "meta2_2"], + } + assert output == expected + + +@pytest.mark.parametrize( + ("meta_columns", "index_column", "extra_metadata", "expected_docs"), + [ + ( + ["meta1", "meta2"], + None, + None, + [ + Document(content="content1", meta={"meta1": "meta1_1", "meta2": "meta2_1"}), + Document(content="content2", meta={"meta1": "meta1_2", "meta2": "meta2_2"}), + ], + ), + ( + ["meta2"], + "meta1", + None, + [ + Document(id="meta1_1", content="content1", meta={"meta2": "meta2_1"}), + Document(id="meta1_2", content="content2", meta={"meta2": "meta2_2"}), + ], + ), + ( + ["meta1"], + None, + {"extra": "metadata"}, + [ + Document(content="content1", meta={"meta1": "meta1_1", "extra": "metadata"}), + Document(content="content2", meta={"meta1": "meta1_2", "extra": "metadata"}), + ], + ), + ( + None, + None, + [{"extra": "metadata1"}, {"extra": "metadata2"}], + [ + Document(content="content1", meta={"extra": "metadata1"}), + Document(content="content2", meta={"extra": "metadata2"}), + ], + ), + ], +) +def test_frame_to_documents( + dataframe: IntoDataFrame, + meta_columns: Union[List[str], None], + index_column: Union[str, None], + extra_metadata: Union[Dict[str, Any], List[Dict[str, Any]], None], + expected_docs: List[Document], +) -> None: + documents = frame_to_documents( + nw.from_native(dataframe), + content_column="content", + meta_columns=meta_columns, + index_column=index_column, + extra_metadata=extra_metadata, + ) + assert len(documents) == 2 + for doc, expected_doc in zip(documents, expected_docs): + assert doc.content == expected_doc.content + assert doc.meta == expected_doc.meta diff --git a/tests/utils.py b/tests/utils.py index be19c94..3ca6657 100644 --- a/tests/utils.py +++ b/tests/utils.py @@ -1,7 +1,8 @@ import yaml -def assert_pipeline_yaml_equal(current_pipeline_yaml: str, expected_pipeline_yaml: str): +def assert_pipeline_yaml_equal(current_pipeline_yaml: str, expected_pipeline_yaml: str) -> None: + """Assert that the current Haystack pipeline YAML is equal to the expected one.""" current_pipeline = yaml.safe_load(current_pipeline_yaml) expected_pipeline = yaml.safe_load(expected_pipeline_yaml) current_pipeline["connections"] = sorted(current_pipeline["connections"], key=lambda x: x["receiver"])