feat(function): add Function v1 artifact and schema contracts - #9041
Closed
Xuanwo wants to merge 3 commits into
Closed
feat(function): add Function v1 artifact and schema contracts#9041Xuanwo wants to merge 3 commits into
Xuanwo wants to merge 3 commits into
Conversation
Xuanwo
marked this pull request as ready for review
September 7, 2026 14:20
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lance Function artifacts need a shared contract that producers, callers, and workers can validate without importing user code or consulting a catalog. Add the experimental
lance-functioncrate to parse Function v1 descriptors, resolve OCI metadata to an exact platform manifest identity, and validate the Arrow interface.Artifact identity hashes the original manifest bytes. Referenced manifests and configs are checked for size and digest, and selected index platform claims must agree with the image config. Image paths use container-root symlink semantics, including rejection of aliases into the worker-reserved directory.
Descriptor validation rejects duplicate JSON keys, unknown fields, unsupported versions, and unknown required capabilities. Schema validation checks individual V5 IPC messages before converting metadata to maps, preserves nested Arrow types and metadata, and rejects unknown extensions unless the caller supplies a storage/metadata validator. Schema file loading requires a caller-selected per-file byte budget, rejects oversized files before reading, and parses files one at a time; the budget does not limit decoded Arrow allocations. Scalar output requires one field; initialization requires exactly one row, including the zero-field case.
Interoperability coverage uses unchanged metadata from an independently built Docker/Skopeo image and schema bytes produced inside it by PyArrow 21. The crate is independent of catalog and execution services; layer fetching/unpacking, runtime isolation, native adapter preflight, and execution remain caller responsibilities. The format is not yet declared stable.