Commit 3574a5e
committed
GH-3509: Optimize BinaryPlainValuesReader by reading directly from ByteBuffer
BinaryPlainValuesReader.readBytes is the hot-path decoder for BINARY (and
STRING) columns using PLAIN encoding. The current implementation funnels
every length read through BytesUtils.readIntLittleEndian(InputStream),
which calls in.read() four times with full IOException plumbing and
virtual dispatch on ByteBufferInputStream, and slices every value through
a virtual ByteBufferInputStream.slice(int).
This change replaces the ByteBufferInputStream field with a single
ByteBuffer set up once in initFromPage. The length prefix is then a
single ByteBuffer.getInt() (one bounds check, JIT-friendly little-endian
intrinsic, no IOException plumbing) and each value slice is a direct
ByteBuffer.slice() instead of a virtual ByteBufferInputStream.slice(int).
When the input is a MultiBufferInputStream the upfront stream.slice(available)
call may consolidate the page into a single fresh ByteBuffer. This is one
allocation per page in exchange for inlined per-value reads, which is a
clear win whenever the page contains more than a handful of values.
Benchmark (BinaryEncodingBenchmark.decodePlain, 100k values per
invocation, JDK 18, JMH -wi 5 -i 10 -f 3, 30 samples per row):
cardinality stringLen Before (ops/s) After (ops/s) Improvement
HIGH 10 23,114,969 27,126,384 +17.4% (1.17x)
HIGH 100 20,516,861 22,200,091 +8.2% (1.08x)
HIGH 1000 7,069,927 7,679,070 +8.6% (1.09x)
LOW 10 22,885,778 26,459,404 +15.6% (1.16x)
LOW 100 20,349,900 22,158,675 +8.9% (1.09x)
LOW 1000 6,279,616 7,500,811 +19.4% (1.19x)
Per-op allocation is unchanged (~88 B/op = the returned Binary + the
per-value ByteBuffer slice). The improvement is largest at small string
lengths because the per-value fixed cost dominates more there.
All 573 parquet-column tests pass.1 parent d96c669 commit 3574a5e
1 file changed
Lines changed: 23 additions & 16 deletions
File tree
- parquet-column/src/main/java/org/apache/parquet/column/values/plain
Lines changed: 23 additions & 16 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| 22 | + | |
| 23 | + | |
22 | 24 | | |
23 | | - | |
24 | 25 | | |
25 | | - | |
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
30 | 37 | | |
31 | 38 | | |
32 | | - | |
| 39 | + | |
33 | 40 | | |
34 | 41 | | |
35 | 42 | | |
36 | | - | |
37 | | - | |
38 | | - | |
39 | | - | |
40 | | - | |
41 | | - | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
42 | 48 | | |
43 | 49 | | |
44 | 50 | | |
45 | 51 | | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
| 52 | + | |
| 53 | + | |
52 | 54 | | |
53 | 55 | | |
54 | 56 | | |
| |||
57 | 59 | | |
58 | 60 | | |
59 | 61 | | |
60 | | - | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
61 | 68 | | |
62 | 69 | | |
0 commit comments