Skip to content

Commit 3a62740

Browse files
committed
Update config docs
1 parent 2dbb6eb commit 3a62740

2 files changed

Lines changed: 7 additions & 3 deletions

File tree

datafusion/common/src/config.rs

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -384,8 +384,12 @@ config_namespace! {
384384
/// and `Binary/BinaryLarge` with `BinaryView`.
385385
pub schema_force_view_types: bool, default = false
386386

387-
/// (reading) If true, parquet reader will read columns of `Binary/LargeBinary` with `Utf8`,
388-
/// and `BinaryView` with `Utf8View`.
387+
/// (reading) If true, parquet reader will read columns of
388+
/// `Binary/LargeBinary` with `Utf8`, and `BinaryView` with `Utf8View`.
389+
///
390+
/// Parquet files generated by some legacy writers do not correctly set
391+
/// the UTF8 flag for strings, causing string columns to be loaded as
392+
/// BLOB instead.
389393
pub binary_as_string: bool, default = false
390394

391395
// The following options affect writing to parquet files

docs/source/user-guide/configs.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ Environment variables are read during `SessionConfig` initialisation so they mus
5757
| datafusion.execution.parquet.pushdown_filters | false | (reading) If true, filter expressions are be applied during the parquet decoding operation to reduce the number of rows decoded. This optimization is sometimes called "late materialization". |
5858
| datafusion.execution.parquet.reorder_filters | false | (reading) If true, filter expressions evaluated during the parquet decoding operation will be reordered heuristically to minimize the cost of evaluation. If false, the filters are applied in the same order as written in the query |
5959
| datafusion.execution.parquet.schema_force_view_types | false | (reading) If true, parquet reader will read columns of `Utf8/Utf8Large` with `Utf8View`, and `Binary/BinaryLarge` with `BinaryView`. |
60-
| datafusion.execution.parquet.binary_as_string | false | (reading) If true, parquet reader will read columns of `Binary/LargeBinary` with `Utf8`, and `BinaryView` with `Utf8View`. |
60+
| datafusion.execution.parquet.binary_as_string | false | (reading) If true, parquet reader will read columns of `Binary/LargeBinary` with `Utf8`, and `BinaryView` with `Utf8View`. Parquet files generated by some legacy writers do not correctly set the UTF8 flag for strings, causing string columns to be loaded as BLOB instead. |
6161
| datafusion.execution.parquet.data_pagesize_limit | 1048576 | (writing) Sets best effort maximum size of data page in bytes |
6262
| datafusion.execution.parquet.write_batch_size | 1024 | (writing) Sets write_batch_size in bytes |
6363
| datafusion.execution.parquet.writer_version | 1.0 | (writing) Sets parquet writer version valid values are "1.0" and "2.0" |

0 commit comments

Comments
 (0)