Skip to content

Commit 3574a5e

Browse files
committed
GH-3509: Optimize BinaryPlainValuesReader by reading directly from ByteBuffer
BinaryPlainValuesReader.readBytes is the hot-path decoder for BINARY (and STRING) columns using PLAIN encoding. The current implementation funnels every length read through BytesUtils.readIntLittleEndian(InputStream), which calls in.read() four times with full IOException plumbing and virtual dispatch on ByteBufferInputStream, and slices every value through a virtual ByteBufferInputStream.slice(int). This change replaces the ByteBufferInputStream field with a single ByteBuffer set up once in initFromPage. The length prefix is then a single ByteBuffer.getInt() (one bounds check, JIT-friendly little-endian intrinsic, no IOException plumbing) and each value slice is a direct ByteBuffer.slice() instead of a virtual ByteBufferInputStream.slice(int). When the input is a MultiBufferInputStream the upfront stream.slice(available) call may consolidate the page into a single fresh ByteBuffer. This is one allocation per page in exchange for inlined per-value reads, which is a clear win whenever the page contains more than a handful of values. Benchmark (BinaryEncodingBenchmark.decodePlain, 100k values per invocation, JDK 18, JMH -wi 5 -i 10 -f 3, 30 samples per row): cardinality stringLen Before (ops/s) After (ops/s) Improvement HIGH 10 23,114,969 27,126,384 +17.4% (1.17x) HIGH 100 20,516,861 22,200,091 +8.2% (1.08x) HIGH 1000 7,069,927 7,679,070 +8.6% (1.09x) LOW 10 22,885,778 26,459,404 +15.6% (1.16x) LOW 100 20,349,900 22,158,675 +8.9% (1.09x) LOW 1000 6,279,616 7,500,811 +19.4% (1.19x) Per-op allocation is unchanged (~88 B/op = the returned Binary + the per-value ByteBuffer slice). The improvement is largest at small string lengths because the per-value fixed cost dominates more there. All 573 parquet-column tests pass.
1 parent d96c669 commit 3574a5e

1 file changed

Lines changed: 23 additions & 16 deletions

File tree

parquet-column/src/main/java/org/apache/parquet/column/values/plain/BinaryPlainValuesReader.java

Lines changed: 23 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -19,36 +19,38 @@
1919
package org.apache.parquet.column.values.plain;
2020

2121
import java.io.IOException;
22+
import java.nio.ByteBuffer;
23+
import java.nio.ByteOrder;
2224
import org.apache.parquet.bytes.ByteBufferInputStream;
23-
import org.apache.parquet.bytes.BytesUtils;
2425
import org.apache.parquet.column.values.ValuesReader;
25-
import org.apache.parquet.io.ParquetDecodingException;
2626
import org.apache.parquet.io.api.Binary;
2727
import org.slf4j.Logger;
2828
import org.slf4j.LoggerFactory;
2929

30+
/**
31+
* Plain encoding reader for BINARY values.
32+
*
33+
* <p>Reads directly from a {@link ByteBuffer} with {@link ByteOrder#LITTLE_ENDIAN} byte order,
34+
* using {@link ByteBuffer#getInt()} for the 4-byte length prefix instead of 4 individual
35+
* {@code InputStream.read()} calls through {@link org.apache.parquet.bytes.BytesUtils#readIntLittleEndian}.
36+
*/
3037
public class BinaryPlainValuesReader extends ValuesReader {
3138
private static final Logger LOG = LoggerFactory.getLogger(BinaryPlainValuesReader.class);
32-
private ByteBufferInputStream in;
39+
private ByteBuffer buffer;
3340

3441
@Override
3542
public Binary readBytes() {
36-
try {
37-
int length = BytesUtils.readIntLittleEndian(in);
38-
return Binary.fromConstantByteBuffer(in.slice(length));
39-
} catch (IOException | RuntimeException e) {
40-
throw new ParquetDecodingException("could not read bytes at offset " + in.position(), e);
41-
}
43+
int length = buffer.getInt();
44+
ByteBuffer valueSlice = buffer.slice();
45+
valueSlice.limit(length);
46+
buffer.position(buffer.position() + length);
47+
return Binary.fromConstantByteBuffer(valueSlice);
4248
}
4349

4450
@Override
4551
public void skip() {
46-
try {
47-
int length = BytesUtils.readIntLittleEndian(in);
48-
in.skipFully(length);
49-
} catch (IOException | RuntimeException e) {
50-
throw new ParquetDecodingException("could not skip bytes at offset " + in.position(), e);
51-
}
52+
int length = buffer.getInt();
53+
buffer.position(buffer.position() + length);
5254
}
5355

5456
@Override
@@ -57,6 +59,11 @@ public void initFromPage(int valueCount, ByteBufferInputStream stream) throws IO
5759
"init from page at offset {} for length {}",
5860
stream.position(),
5961
(stream.available() - stream.position()));
60-
this.in = stream.remainingStream();
62+
int available = stream.available();
63+
if (available > 0) {
64+
this.buffer = stream.slice(available).order(ByteOrder.LITTLE_ENDIAN);
65+
} else {
66+
this.buffer = ByteBuffer.allocate(0).order(ByteOrder.LITTLE_ENDIAN);
67+
}
6168
}
6269
}

0 commit comments

Comments
 (0)