fix(encoding): accept 1-bit booleans in the single-row full-zip fallback - #8934
Merged
zhangyue19921010 merged 1 commit intoSep 2, 2026
Conversation
When a single top-level row carries more rep/def levels than one mini-block chunk can hold, the primitive encoder falls back to full-zip after a pre-check that the value block is something full-zip can serialize. That pre-check rejected 1-bit booleans as non-byte-aligned, although `encode_full_zip` widens them to bytes before compressing (lance-format#6723). A sparse `List<List<Boolean>>` row therefore failed with "Mini-block cannot encode N rep/def levels in one top-level row" even though it encodes fine, and even when the user explicitly requested `structural_encoding=fullzip`. Before lance-format#6787 the same row was written through full-zip. The pre-check now lets 1-bit fixed-width blocks through, matching what `encode_full_zip` accepts. The boolean test that asserted the error now asserts a full-zip round trip alongside the existing string case.
github-actions
Bot
added
A-encoding
![lance-gatekeeper[bot]](https://avatars.githubusercontent.com/in/4433141?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
✅ Gate recommendation: approve.
The capability check now matches the existing full-zip boolean encoder/decoder contract: 1-bit booleans are widened, while other unsupported non-byte-aligned widths remain rejected. The regression covers full-zip selection and round-trip decoding for the unsplittable single-row shape.
lance-gatekeeper
Bot
added
the
K-approved
Collaborator
Author
@Xuanwo Thanks for your review :) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a single top-level row carries more rep/def levels than one mini-block chunk can hold, the primitive encoder falls back to full-zip after a pre-check that the value block is something full-zip can serialize. That pre-check rejected 1-bit booleans as non-byte-aligned, although
encode_full_zipwidens them to bytes before compressing (#6723).A sparse
List<List<Boolean>>row therefore failed with "Mini-block cannot encode N rep/def levels in one top-level row" even though it encodes fine, and even when the user explicitly requestedstructural_encoding=fullzip. Before #6787 the same row was written through full-zip.The pre-check now lets 1-bit fixed-width blocks through, matching what
encode_full_zipaccepts. The boolean test that asserted the error now asserts a full-zip round trip alongside the existing string case.