Skip to content

Commit 9d111ae

Browse files
committed
Add transactional logical metadata editing
Implemented 0.4.92 with a bounded C++ Editing API supporting add, set, remove, and remove-all operations. It preserves provenance, uses dirty tombstones, detects singleton conflicts, shares validation with Creation, and exposes immutable thin Python bindings. See OpenMeta/src/include/openmeta/metadata_editing.h:17, OpenMeta/docs/editing.md:1, and OpenMeta/CHANGES.md:3.
1 parent aa56377 commit 9d111ae

27 files changed

Lines changed: 2076 additions & 304 deletions

CHANGES.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,33 @@
11
# OpenMeta Changes
22

3+
## 0.4.92 - 2026-08-02
4+
5+
Changes compared with `0.4.91`.
6+
7+
### Added
8+
9+
- Added the transactional v1 `edit_metadata(...)` C++ contract for bounded
10+
logical add, set, remove, and remove-all operations over finalized stores.
11+
- Added deterministic repeated-field occurrence handling, explicit singleton
12+
conflicts and repair, request-order semantics, stable statuses, operation
13+
diagnostics, and resource limits shared with the Creation field map.
14+
- Added provenance-preserving value updates, dirty tombstones, deterministic
15+
portable-XMP additions, and support for adding metadata to an empty finalized
16+
store without inventing source-block provenance.
17+
- Added immutable thin Python editing operations and
18+
`Document.edit_metadata(...)`, returning a detached edited document.
19+
- Added C++ creation/editing/transfer tests, a Python editing smoke gate, and
20+
public editing, quick-start, stability, and lifecycle documentation.
21+
22+
### Changed
23+
24+
- Creation and Editing now share one private logical-field descriptor and
25+
validation implementation, preventing mapping and constraint drift.
26+
- `MetaStore` now exposes `is_finalized()` so transactional editing can reject
27+
incomplete source stores explicitly.
28+
- Editing readiness is now tracked at about 75-80%; the active implementation
29+
sequence advances to Transfer, Translation, and Writing.
30+
331
## 0.4.91 - 2026-07-31
432

533
Changes compared with `0.4.90`.

CMakeLists.txt

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -364,10 +364,12 @@ set(OPENMETA_SOURCES
364364
src/openmeta/metadata_capabilities.cc
365365
src/openmeta/metadata_concepts.cc
366366
src/openmeta/metadata_creation.cc
367+
src/openmeta/metadata_editing.cc
367368
src/openmeta/metadata_fuzzy_search.cc
368369
src/openmeta/metadata_interpretation.cc
369370
src/openmeta/metadata_query.cc
370371
src/openmeta/metadata_transfer.cc
372+
src/openmeta/metadata_logical_field_internal.cc
371373
src/openmeta/meta_store.cc
372374
src/openmeta/meta_edit.cc
373375
src/openmeta/meta_value.cc
@@ -614,6 +616,16 @@ if(OPENMETA_BUILD_PYTHON)
614616
COMMENT "Running python metadata creation smoke gate"
615617
VERBATIM
616618
)
619+
620+
add_custom_target(openmeta_gate_python_metadata_editing_smoke
621+
DEPENDS openmeta_python
622+
COMMAND ${CMAKE_COMMAND}
623+
"-DOPENMETA_PYTHON_EXECUTABLE=${Python_EXECUTABLE}"
624+
"-DOPENMETA_PYTHONPATH=${CMAKE_CURRENT_BINARY_DIR}/python"
625+
-P "${CMAKE_CURRENT_SOURCE_DIR}/tests/python_metadata_editing_smoke_test.cmake"
626+
COMMENT "Running python metadata editing smoke gate"
627+
VERBATIM
628+
)
617629
endif()
618630
endif()
619631

@@ -816,6 +828,7 @@ if(OPENMETA_BUILD_TESTS)
816828
tests/metadata_capabilities_test.cc
817829
tests/metadata_concepts_test.cc
818830
tests/metadata_creation_test.cc
831+
tests/metadata_editing_test.cc
819832
tests/metadata_fuzzy_search_test.cc
820833
tests/metadata_interpretation_test.cc
821834
tests/metadata_query_test.cc
@@ -1069,6 +1082,14 @@ if(OPENMETA_BUILD_TESTS)
10691082
"-DOPENMETA_PYTHONPATH=${CMAKE_CURRENT_BINARY_DIR}/python"
10701083
-P "${CMAKE_CURRENT_SOURCE_DIR}/tests/python_metadata_creation_smoke_test.cmake"
10711084
)
1085+
1086+
add_test(
1087+
NAME openmeta_python_metadata_editing_smoke
1088+
COMMAND ${CMAKE_COMMAND}
1089+
"-DOPENMETA_PYTHON_EXECUTABLE=${Python_EXECUTABLE}"
1090+
"-DOPENMETA_PYTHONPATH=${CMAKE_CURRENT_BINARY_DIR}/python"
1091+
-P "${CMAKE_CURRENT_SOURCE_DIR}/tests/python_metadata_editing_smoke_test.cmake"
1092+
)
10721093
endif()
10731094

10741095
if(OPENMETA_TEST_RUNTIME_LIBRARY_PATH)

README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@ stable transfer/edit building blocks for export workflows.
1717
- Reassemble chunked payloads and optionally decompress supported carriers.
1818
- Decode metadata into a normalized `MetaStore`.
1919
- Create a fresh finalized metadata store from bounded logical portable fields.
20+
- Edit logical portable fields transactionally without mutating the source
21+
store.
2022
- Export sidecars and previews.
2123
- Prepare, compile, emit, and edit metadata transfers for bounded target
2224
families.
@@ -69,6 +71,8 @@ If you already own the encoder, SDK objects, or output container, follow
6971
search contract, quality gates, and benchmark
7072
- [docs/creation.md](docs/creation.md): bounded fresh metadata construction,
7173
field mapping, validation, and Python use
74+
- [docs/editing.md](docs/editing.md): transactional logical add, set, remove,
75+
provenance, conflict, and Python behavior
7276
- [docs/doxygen.md](docs/doxygen.md): API reference
7377
- [SECURITY.md](SECURITY.md): security model and reporting
7478
- [NOTICE.md](NOTICE.md): notices and third-party dependency information

VERSION

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
0.4.91
1+
0.4.92

docs/api_stability.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ different status.
2323
| `ExportNameStyle::Canonical` and `ExportNameStyle::XmpPortable` | `openmeta/interop_export.h` | Stable | Stable naming modes for key-space-aware and portable exports. |
2424
| `ExportNameStyle::FlatHost` | `openmeta/interop_export.h` | Stable | Stable v1 flat host naming contract. See [flat_host_mapping.md](flat_host_mapping.md). |
2525
| Fresh metadata creation: `create_metadata(...)` and typed field helpers | `openmeta/metadata_creation.h` | Experimental | Transactional v1 contract for bounded host-provided logical fields. Produces a finalized canonical portable-XMP `MetaStore`, preserves additive creator/keyword order, rejects duplicate singletons, validates UTF-8/XML text and typed numeric constraints, and exposes the same C++ policy through a thin Python `create_metadata(...)` wrapper. Direct EXIF/IPTC projection remains a Translation concern. See [creation.md](creation.md). |
26+
| Logical metadata editing: `edit_metadata(...)` and add/set/remove helpers | `openmeta/metadata_editing.h` | Experimental | Transactional v1 contract over finalized stores and the Creation logical field map. Set/remove preserve provenance through dirty values and tombstones; add handles deterministic repeated fields and explicit singleton conflicts. Python returns a detached edited `Document` without mutating its source. See [editing.md](editing.md). |
2627
| EXIF/TIFF orientation helpers: `interpret_exif_orientation(...)`, `exif_orientation_name(...)`, `exif_orientation_rotation_degrees_cw(...)`, `exif_orientation_rotation_only(...)` | `openmeta/orientation.h` | Stable | Small utility contract for user-facing orientation labels, clockwise rotation degrees, mirrored-state detection, dimension-swap detection, and rotation-only fallbacks. Python exposes the same helpers through thin scalar/dictionary wrappers. |
2728
| EXIF/TIFF/DNG numeric value names and version formatting: `exif_tag_numeric_value_name(...)`, `exif_tag_numeric_value_format(...)`, `exif_tag_byte_value_format(...)`, and focused helpers | `openmeta/exif_value_names.h` | Stable | Small helper contract for common enum-like TIFF/EXIF/DNG numeric values such as compression, photometric interpretation, planar configuration, exposure program/mode, metering mode, light source, flash, color space, white balance, scene capture type, gain control, CFA layout, DNG calibration illuminants, and EXIF 3.1 lens-correction / noise-reduction status values, plus selected bounded Canon/Nikon/Sony/Fujifilm/Pentax/Olympus/Panasonic/Phase One/Kodak/Minolta/Sigma/Samsung/Ricoh/Apple/FLIR/JVC/GE/Reconyx/Microsoft/Motorola/Nintendo/Sanyo MakerNote contexts including NikonSettings On/Off labels, Reconyx scalar labels, Microsoft stitch labels, Motorola `CustomRendered` labels, Nintendo category labels, Sanyo public-context scalar labels, current Canon RF lens-type labels, current Nikon Z `LensData0800` `LensID` labels, and an ambiguous Pentax Sigma/Samsung/Tokina lens-family label where stable. Version/firmware helpers format selected standard EXIF byte-version fields, Nikon version-like payloads, Olympus packed firmware values, and native RAF firmware payloads without treating formatted versions as enum labels. Unknown or ambiguous values return an empty string or `false` and remain lossless metadata. |
2829
| Photoshop IRB decode: `decode_photoshop_irb(...)`, `measure_photoshop_irb(...)` | `openmeta/photoshop_irb_decode.h` | Experimental | Bounded resource traversal with stable raw resource preservation behavior, but the interpreted subset can still grow. Current interpretation includes fixed-layout resource fields, display/grid/thumbnail/color-sampler headers, working-path and numbered clipping-path byte counts / record summaries, descriptor-header summaries plus safe descriptor class-name/class-ID/item-count fields, bounded descriptor item bodies for `bool`, `long`, `comp`, `doub`, `UntF`, `TEXT`, `enum`, `type`, and `GlbC`, opaque `alis` and `tdta` byte counts, ordered `obj ` property/class/enumerated/offset/identifier/index/name reference fields with per-value and aggregate limits, descriptor item/reference type-name/type-code fields, parsed maximum depth, and parsed per-type counters, nested object/list traversal with item path/depth/list-index and parsed-value count fields, `XMLData`, ImageReady ASCII text resources, Lightroom workflow text, MacintoshPrintInfo / Macintosh NSPrintInfo / Windows DEVMODE / AlternateDuotoneColors / AlternateSpotColors / obsolete Photoshop tag byte counts, legacy halftone/transfer/duotone/EPS byte summaries, embedded IPTC/ICC/EXIF/EXIF2/XMP byte-count fields, and optional embedded IPTC-IIM, XMP, and ICC payload decode. |

docs/creation.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,9 @@ Invalid requests raise `ValueError` with the C++ status and rejected field
120120
index. The returned object is a normal `Document`: it can be queried, dumped,
121121
or converted into a transfer snapshot.
122122

123+
Use the matching transactional Editing API to modify these fields in an
124+
existing finalized store. See [editing.md](editing.md).
125+
123126
## Scope And Safety
124127

125128
Creation writes canonical portable-XMP entries because that representation can

docs/development.md

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ model should stay compact:
2121
| Query | Find entries by exact name or semantic group, then expose normalized query candidates, structured interpretation records, bounded cross-family concept resolutions, transfer hints, sensitivity, and conflict flags for crop/border/active-area, exposure/gain, color/WB/profile/source-color-transform, orientation, date/time, GPS, descriptive fields including contact/event/person/organization/product/artwork/rights/license/release, editorial, accessibility, taxonomy, registry, image-region, document-identity, document-lineage, document-history, technical-image, audio, and preview semantics, lens-correction, computational/thermal/stitch, RAW/source-processing fields, and BMFF derived-image construction and tiled-image configuration evidence across standard and vendor metadata. | High, measured about 99.77% for declared query targets. |
2222
| Fuzzy Search | Optionally find misspelled, aliased, or near-match metadata names and property paths with bounded deterministic top-k ranking and explicit exact/alias/fuzzy provenance. | High enough for the current milestone, about 80-85%; the standalone RapidFuzz-backed API, curated positive/adversarial quality gate, bounded ASCII contract, Python wrappers, Release/libc++ CI gate, and opt-in scaling benchmark are implemented. |
2323
| Creation | Build fresh metadata entries from host-provided values through a transactional, bounded logical-field request that produces a finalized canonical portable-XMP store. | Medium-high, about 70-75%; the v1 C++ contract, common descriptive/capture fields, UTF-8/XML and typed-value validation, resource limits, deterministic collection ordering, portable serialization, semantic-query visibility, and thin Python wrapper are implemented. |
24-
| Editing | Modify existing logical metadata entries while preserving valid surrounding structure. | Medium, about 60-70%. |
24+
| Editing | Modify existing logical metadata entries while preserving valid surrounding structure. | Medium-high, about 75-80%; the v1 logical add/set/remove transaction, deterministic occurrence handling, singleton conflict repair, dirty/tombstone behavior, provenance preservation, portable serialization, transfer visibility, and immutable thin Python wrapper are implemented. |
2525
| Transfer | Move metadata between files using explicit compatible-file or rendered-image safety policies. | Medium-high, about 80-85%. |
2626
| Translation | Project metadata between families, mainly bounded EXIF/IPTC/XMP portable mappings. | Medium, about 60-70%. |
2727
| Writing | Serialize metadata and write or rewrite it into target containers. | Medium, about 65-75%. |
@@ -38,12 +38,15 @@ retains a `98-100%` range because not every declared container lane has an
3838
independent conformance sample set, even though tracked inputs have explicit
3939
read outcomes.
4040

41-
The first Creation milestone is implemented in
41+
The first Creation and Editing milestones are implemented in
4242
[`metadata_creation.h`](../src/include/openmeta/metadata_creation.h) and
43-
documented in [`creation.md`](creation.md). The active implementation sequence
44-
now advances to Editing, Transfer, Translation, and Writing. Creation resumes
45-
for arbitrary/custom properties, multilingual alternatives, structured values,
46-
and direct family projection. Adapters and Utilities remain deferred. Fuzzy
43+
[`metadata_editing.h`](../src/include/openmeta/metadata_editing.h), with their
44+
contracts documented in [`creation.md`](creation.md) and
45+
[`editing.md`](editing.md). The active implementation sequence now advances to
46+
Transfer, Translation, and Writing. Creation and Editing resume for
47+
arbitrary/custom properties, multilingual alternatives, structured values,
48+
direct family projection, structural block operations, and broader
49+
cross-family synchronization. Adapters and Utilities remain deferred. Fuzzy
4750
Search resumes before those final two stages for independently sourced quality
4851
expansion, designed Unicode/transliteration behavior, multilingual gates, and
4952
an optional immutable index for repeated searches over large stores.

docs/editing.md

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
# Metadata Editing
2+
3+
`openmeta/metadata_editing.h` provides a bounded transactional editing contract
4+
for the same logical fields accepted by the high-level Creation API. It edits
5+
canonical portable-XMP entries in a finalized `MetaStore` without requiring a
6+
host to work with namespace paths or entry IDs.
7+
8+
The API is experimental and versioned by
9+
`kMetadataEditingContractVersion == 1`.
10+
11+
## C++ Example
12+
13+
```cpp
14+
#include "openmeta/metadata_editing.h"
15+
16+
#include <array>
17+
18+
const std::array operations = {
19+
openmeta::make_metadata_edit_set(
20+
openmeta::make_metadata_creation_text(
21+
openmeta::MetadataCreationFieldKind::Title, "Edited title")),
22+
openmeta::make_metadata_edit_add(
23+
openmeta::make_metadata_creation_text(
24+
openmeta::MetadataCreationFieldKind::Keyword, "approved")),
25+
openmeta::make_metadata_edit_remove(
26+
openmeta::MetadataCreationFieldKind::Creator, 0),
27+
};
28+
29+
openmeta::MetadataEditingRequest request;
30+
request.operations = operations;
31+
32+
openmeta::MetaStore edited;
33+
const openmeta::MetadataEditingResult result =
34+
openmeta::edit_metadata(source, request, &edited);
35+
```
36+
37+
`source` must be finalized. The output is replaced only after every operation
38+
has passed validation and the entire edit has committed. On failure, `edited`
39+
is unchanged and `failed_operation_index` identifies the rejected operation
40+
when available.
41+
42+
## Operation Semantics
43+
44+
| Operation | Behavior |
45+
| --- | --- |
46+
| `Add` | Creates an absent singleton or appends a creator/keyword value. Adding an existing singleton is an explicit conflict. |
47+
| `Set` | Replaces one active value. Existing key, origin, block, wire provenance, and flags are preserved; `Dirty` is added. |
48+
| `Remove` | Marks one active value as `Deleted | Dirty`. The tombstone remains available for writeback and audit behavior. |
49+
| `RemoveAll` | Tombstones every active occurrence. This can repair malformed duplicate singleton fields before a new value is added in the same transaction. |
50+
51+
Operations observe earlier operations in request order. For repeated creators
52+
and keywords, `occurrence` is a zero-based index into the current active
53+
logical values. Removing an occurrence shifts later values for subsequent
54+
operations. New repeated values receive the next unused canonical property
55+
index; existing index gaps are not renumbered.
56+
57+
Singleton fields accept only occurrence zero. If a malformed store contains
58+
multiple active copies, single-value `Set` and `Remove` return
59+
`AmbiguousTarget`; use `RemoveAll` followed by `Add` when that repair is
60+
intended. Missing targets are errors rather than silent no-ops.
61+
62+
## Provenance And Blocks
63+
64+
`Set` changes only the value and dirty flag. `Remove` changes only entry flags.
65+
Both therefore retain the original block and wire provenance.
66+
67+
`Add` emits a new dirty canonical portable-XMP entry. When the store already
68+
contains an XMP entry with a valid block, the new entry uses that block and a
69+
later deterministic order. A finalized empty store can also be edited; its new
70+
entry has no source block because no original carrier exists. Portable XMP and
71+
transfer preparation can serialize that entry normally.
72+
73+
Editing does not compact tombstones automatically. Call the lower-level
74+
`compact(...)` helper only when losing deleted-entry identity is appropriate
75+
for the host workflow.
76+
77+
## Validation And Limits
78+
79+
`Add` and `Set` use exactly the Creation field mapping and value validation.
80+
Text must be non-empty valid UTF-8 and XML 1.0 character data. Orientation,
81+
rating, dimensions, color space, ISO, and rational values use the constraints
82+
documented in [creation.md](creation.md).
83+
84+
The hard maxima are:
85+
86+
- `1024` operations
87+
- `1 MiB` of UTF-8 text per `Add` or `Set` operation
88+
- `8 MiB` of UTF-8 text per request
89+
90+
`MetadataEditingRequest::limits` may lower but not raise these bounds. The
91+
implementation keeps no global state. Concurrent calls are safe when callers
92+
use distinct output stores and do not mutate the finalized source.
93+
94+
## Python
95+
96+
Python operation objects own their text and pass the same request to C++:
97+
98+
```python
99+
import openmeta
100+
101+
K = openmeta.MetadataCreationFieldKind
102+
edited = document.edit_metadata([
103+
openmeta.metadata_edit_set(
104+
openmeta.metadata_creation_text(K.Title, "Edited title")),
105+
openmeta.metadata_edit_add(
106+
openmeta.metadata_creation_text(K.Keyword, "approved")),
107+
openmeta.metadata_edit_remove(K.Creator, 0),
108+
])
109+
```
110+
111+
The method returns a detached edited `Document`; the original document is not
112+
mutated. The result can be queried, dumped as XMP, or converted into a transfer
113+
snapshot. Invalid requests raise `ValueError` with the C++ status and rejected
114+
operation index.
115+
116+
## Current Scope
117+
118+
This milestone edits the 24 logical fields listed in
119+
[creation.md](creation.md). It does not yet provide high-level arbitrary
120+
EXIF/IPTC/XMP/custom-key operations, language-alternative selection beyond
121+
`x-default`, structural block editing, a full EXIF/IPTC/XMP synchronization
122+
engine, or direct in-place file patching. Lower-level `MetaEdit` remains
123+
available for entry-ID-based host code, while transfer and writer APIs handle
124+
container persistence.

docs/interpretation_status.md

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -70,11 +70,12 @@ outputs.
7070

7171
## Active Project Sequence
7272

73-
The first bounded Creation milestone is complete. The current implementation
74-
focus advances to Editing, Transfer, Translation, and Writing. Adapters and
75-
Utilities remain deferred. Creation resumes for custom, multilingual,
76-
structured, and direct-family output; Fuzzy Search resumes before those final
77-
two stages for its Unicode/multilingual and optional-index milestones.
73+
The first bounded Creation and Editing milestones are complete. The current
74+
implementation focus advances to Transfer, Translation, and Writing. Adapters
75+
and Utilities remain deferred. Creation and Editing resume for custom,
76+
multilingual, structured, direct-family output, structural block operations,
77+
and broader synchronization; Fuzzy Search resumes before those final two
78+
stages for its Unicode/multilingual and optional-index milestones.
7879

7980
## Interpretation Maintenance Priorities
8081

0 commit comments

Comments
 (0)