You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge?
DataFrame.writeParquet shipped in #27, but the symmetric CSV writer is
missing. DataFusion exposes DataFrame::write_csv, which is commonly
used for ad-hoc exports.
Describe the solution you'd like
Add a CsvWriteOptions value class covering the knobs that DataFrameWriteOptions + CSV-specific options expose
(single-file output, partition columns, header, delimiter, quote,
escape, null-value string, compression).
Cover with tests in the spirit of DataFrameWriteParquetTest
(single-file, compression, retain-after-write semantics).
Describe alternatives you've considered
COPY (SELECT * FROM df) TO '…' via SQL works but requires a
registered name and a SQL detour.
Additional context
Aligns the writer story: with this and the JSON writer, we cover the
three formats DataFusion's DataFrame API can write today
(write_parquet, write_csv, write_json).
Is your feature request related to a problem or challenge?
DataFrame.writeParquetshipped in #27, but the symmetric CSV writer ismissing. DataFusion exposes
DataFrame::write_csv, which is commonlyused for ad-hoc exports.
Describe the solution you'd like
CsvWriteOptionsvalue class covering the knobs thatDataFrameWriteOptions+ CSV-specific options expose(single-file output, partition columns, header, delimiter, quote,
escape, null-value string, compression).
proto/csv_write_options.proto, following the conventionestablished by
parquet_write_options.protoin feat(write): add DataFrame.writeParquet with ParquetWriteOptions #27.DataFrame.writeCsv(path[, options]).DataFrameWriteParquetTest(single-file, compression, retain-after-write semantics).
Describe alternatives you've considered
COPY (SELECT * FROM df) TO '…'via SQL works but requires aregistered name and a SQL detour.
Additional context
Aligns the writer story: with this and the JSON writer, we cover the
three formats DataFusion's DataFrame API can write today
(
write_parquet,write_csv,write_json).