preserve embedded NUL bytes in custom function text values - #1406
Conversation
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1406 +/- ##
==========================================
+ Coverage 47.16% 51.63% +4.47%
==========================================
Files 12 13 +1
Lines 1533 1867 +334
==========================================
+ Hits 723 964 +241
- Misses 669 742 +73
- Partials 141 161 +20 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
LGTM. Correctly preserves embedded NULs on both paths, guard matches the blob path, and the test covers both directions. Thanks! Nit (non-blocking): the TEXT arg case calls |
|
Ref for the ordering note: https://www.sqlite.org/c3ref/value_blob.html |
|
good catch, went ahead and flipped it to text-then-bytes on the TEXT case so it follows the documented order in that ref. left the existing BLOB case as-is since it predates this and isn't really in scope here. |
|
Thank you |
|
Glad to contribute. Thanks |
…4.48) (#140) This PR contains the following updates: | Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) | |---|---|---|---| | [github.com/mattn/go-sqlite3](https://github.com/mattn/go-sqlite3) | `v1.14.47` → `v1.14.48` |  |  | --- ### Release Notes <details> <summary>mattn/go-sqlite3 (github.com/mattn/go-sqlite3)</summary> ### [`v1.14.48`](https://github.com/mattn/go-sqlite3/releases/tag/v1.14.48): 1.14.48 [Compare Source](mattn/go-sqlite3@v1.14.47...v1.14.48) #### What's Changed - Add Serialize and Deserialize support by [@​otoolep](https://github.com/otoolep) in [#​1089](mattn/go-sqlite3#1089) - Replace namedValue with driver.NamedValue to avoid copying exec/query args by [@​charlievieth](https://github.com/charlievieth) in [#​1128](mattn/go-sqlite3#1128) - Add go 1.20 to workflow matrix, remove 1.17 by [@​connyay](https://github.com/connyay) in [#​1136](mattn/go-sqlite3#1136) - Add build tags to support both x86 and ARM compilation on macOS by [@​Spaider](https://github.com/Spaider) in [#​1069](mattn/go-sqlite3#1069) - Fix virtual table example. by [@​andrzh](https://github.com/andrzh) in [#​1149](mattn/go-sqlite3#1149) - Update README.md by [@​parthokr](https://github.com/parthokr) in [#​1163](mattn/go-sqlite3#1163) - Update amalgamation code by [@​mattn](https://github.com/mattn) in [#​1166](mattn/go-sqlite3#1166) - Update amalgamation code by [@​mattn](https://github.com/mattn) in [#​1197](mattn/go-sqlite3#1197) - Fix docker job by [@​itizir](https://github.com/itizir) in [#​1201](mattn/go-sqlite3#1201) - Fix musl build ([#​1164](mattn/go-sqlite3#1164)) by [@​leso-kn](https://github.com/leso-kn) in [#​1177](mattn/go-sqlite3#1177) - update go version to 1.19 by [@​mattn](https://github.com/mattn) in [#​1208](mattn/go-sqlite3#1208) - Update amalgamation code to 3.45.0 by [@​mattn](https://github.com/mattn) in [#​1207](mattn/go-sqlite3#1207) - Update amalgamation code to 3.45.1 by [@​mattn](https://github.com/mattn) in [#​1211](mattn/go-sqlite3#1211) - close channel by [@​mattn](https://github.com/mattn) in [#​1213](mattn/go-sqlite3#1213) - fix: some typos by [@​pomadev](https://github.com/pomadev) in [#​1222](mattn/go-sqlite3#1222) - Add support for libsqlite3 on z/OS by [@​dustin-ward](https://github.com/dustin-ward) in [#​1239](mattn/go-sqlite3#1239) - Update amalgamation code to 3.46.1 by [@​mattn](https://github.com/mattn) in [#​1273](mattn/go-sqlite3#1273) - close statement when missing query arguments by [@​mattn](https://github.com/mattn) in [#​1281](mattn/go-sqlite3#1281) - Upgrade upload-artifact action by [@​jonstacks](https://github.com/jonstacks) in [#​1300](mattn/go-sqlite3#1300) - Remove suggestion that CGO isn't always needed by [@​samjewell](https://github.com/samjewell) in [#​1290](mattn/go-sqlite3#1290) - remove superfluous use of runtime.SetFinalizer on SQLiteRows by [@​charlievieth](https://github.com/charlievieth) in [#​1301](mattn/go-sqlite3#1301) - Fix sqlite3\_opt\_unlock\_notify with USE\_LIBSQLITE3 by [@​q66](https://github.com/q66) in [#​1262](mattn/go-sqlite3#1262) - Fix memory leak in callbackRetText function by [@​hionay](https://github.com/hionay) in [#​1259](mattn/go-sqlite3#1259) - docs: clarify GCP section by [@​justinsb](https://github.com/justinsb) in [#​1305](mattn/go-sqlite3#1305) - Add ability to set an int64 file control by [@​jonstacks](https://github.com/jonstacks) in [#​1298](mattn/go-sqlite3#1298) - Update amalgamation code to 3.49.1 by [@​mattn](https://github.com/mattn) in [#​1335](mattn/go-sqlite3#1335) - Update amalgamation code to 3.50.3 by [@​mattn](https://github.com/mattn) in [#​1343](mattn/go-sqlite3#1343) - Drop userauth implementation by [@​mattn](https://github.com/mattn) in [#​1344](mattn/go-sqlite3#1344) - fix syntax error by [@​eraytufan](https://github.com/eraytufan) in [#​1346](mattn/go-sqlite3#1346) - update amalgamation code by [@​mattn](https://github.com/mattn) in [#​1347](mattn/go-sqlite3#1347) - use quote include instead of angled include for sqlite3-binding.h by [@​nautaa](https://github.com/nautaa) in [#​1362](mattn/go-sqlite3#1362) - Upgrade SQLite to version [`3051001`](mattn/go-sqlite3@3051001) by [@​mattn](https://github.com/mattn) in [#​1366](mattn/go-sqlite3#1366) - Feat: add percentile extension option by [@​dsonck92](https://github.com/dsonck92) in [#​1364](mattn/go-sqlite3#1364) - Upgrade SQLite to version [`3051002`](mattn/go-sqlite3@3051002) by [@​mattn](https://github.com/mattn) in [#​1370](mattn/go-sqlite3#1370) - Use unsafe slice by [@​mattn](https://github.com/mattn) in [#​1373](mattn/go-sqlite3#1373) - Call sqlite3\_clear\_bindings() after sqlite3\_reset() in bind() by [@​mattn](https://github.com/mattn) in [#​1374](mattn/go-sqlite3#1374) - Upgrade SQLite to version [`3051003`](mattn/go-sqlite3@3051003) by [@​mattn](https://github.com/mattn) in [#​1375](mattn/go-sqlite3#1375) - Ensure Close always removes runtime finalizer to prevent memory leak by [@​mattn](https://github.com/mattn) in [#​1376](mattn/go-sqlite3#1376) - Fix json example by [@​Jaculabilis](https://github.com/Jaculabilis) in [#​1313](mattn/go-sqlite3#1313) - Add missing virtual table constraint op constants by [@​theimpostor](https://github.com/theimpostor) in [#​1379](mattn/go-sqlite3#1379) - Eliminate unnecessary bounds checks in hot paths by [@​mattn](https://github.com/mattn) in [#​1381](mattn/go-sqlite3#1381) - \[codex] optimize sqlite bind fast path by [@​mattn](https://github.com/mattn) in [#​1382](mattn/go-sqlite3#1382) - \[codex] batch row column fetches in Next by [@​mattn](https://github.com/mattn) in [#​1383](mattn/go-sqlite3#1383) - Raise minimum Go version to 1.21 by [@​mattn](https://github.com/mattn) in [#​1384](mattn/go-sqlite3#1384) - Reduce sqlite bind overhead by [@​mattn](https://github.com/mattn) in [#​1385](mattn/go-sqlite3#1385) - reduce CGO call overhead for exec and bind paths by [@​mattn](https://github.com/mattn) in [#​1386](mattn/go-sqlite3#1386) - \[codex] add opt-in statement cache by [@​mattn](https://github.com/mattn) in [#​1387](mattn/go-sqlite3#1387) - Fix panic when querying input with no SQL (only comments/whitespace) by [@​mattn](https://github.com/mattn) in [#​1392](mattn/go-sqlite3#1392) - evict least-recently-used stmt when cache is full by [@​mattn](https://github.com/mattn) in [#​1388](mattn/go-sqlite3#1388) - Upgrade SQLite to version [`3053000`](mattn/go-sqlite3@3053000) by [@​mattn](https://github.com/mattn) in [#​1394](mattn/go-sqlite3#1394) - add sqlite\_dbstat tag for the DBSTAT virtual table by [@​calmh](https://github.com/calmh) in [#​1338](mattn/go-sqlite3#1338) - avoid out of bounds write in unlock\_notify\_wait on 64 bit platforms by [@​calmh](https://github.com/calmh) in [#​1399](mattn/go-sqlite3#1399) - modernise reflect.SliceHeader to unsafe.Slice by [@​calmh](https://github.com/calmh) in [#​1400](mattn/go-sqlite3#1400) - guard oversized string length in ResultText by [@​dxbjavid](https://github.com/dxbjavid) in [#​1402](mattn/go-sqlite3#1402) - bind via sqlite3\_bind\_text64/blob64 to avoid 32-bit length truncation by [@​dxbjavid](https://github.com/dxbjavid) in [#​1403](mattn/go-sqlite3#1403) - Upgrade SQLite to version [`3053002`](mattn/go-sqlite3@3053002) by [@​mattn](https://github.com/mattn) in [#​1404](mattn/go-sqlite3#1404) - guard oversized blob length in callbackRetBlob by [@​dxbjavid](https://github.com/dxbjavid) in [#​1405](mattn/go-sqlite3#1405) - preserve embedded NUL bytes in custom function text values by [@​dxbjavid](https://github.com/dxbjavid) in [#​1406](mattn/go-sqlite3#1406) - Follow documented call order for sqlite3\_value\_blob in callbackArgString by [@​mattn](https://github.com/mattn) in [#​1407](mattn/go-sqlite3#1407) - Use atomic.Value for handle table and add concurrent lookup benchmark by [@​mattn](https://github.com/mattn) in [#​1412](mattn/go-sqlite3#1412) - cache column metadata for prepared and cached statements by [@​mattn](https://github.com/mattn) in [#​1413](mattn/go-sqlite3#1413) - free leaked schema string in GetFilename by [@​dxbjavid](https://github.com/dxbjavid) in [#​1408](mattn/go-sqlite3#1408) - Fix race in SQLiteStmt.Close by holding conn lock across cache check by [@​mattn](https://github.com/mattn) in [#​1416](mattn/go-sqlite3#1416) - Add CodeRabbit as a sponsor by [@​mattn](https://github.com/mattn) in [#​1417](mattn/go-sqlite3#1417) - Return error from vtable cursor open instead of ignoring it by [@​mattn](https://github.com/mattn) in [#​1419](mattn/go-sqlite3#1419) - Check sqlite3\_malloc64 result in Deserialize by [@​mattn](https://github.com/mattn) in [#​1420](mattn/go-sqlite3#1420) - Fix panic when registered functions return named types by [@​mattn](https://github.com/mattn) in [#​1421](mattn/go-sqlite3#1421) - Return error instead of silently ignoring unsupported bind types by [@​mattn](https://github.com/mattn) in [#​1422](mattn/go-sqlite3#1422) - Add CodeRabbit configuration by [@​mattn](https://github.com/mattn) in [#​1418](mattn/go-sqlite3#1418) - Close database on all error paths in Open by [@​mattn](https://github.com/mattn) in [#​1423](mattn/go-sqlite3#1423) - Check preupdate value fetch result to avoid NULL dereference by [@​mattn](https://github.com/mattn) in [#​1424](mattn/go-sqlite3#1424) - Use C.int in exported callbacks to match C declarations by [@​mattn](https://github.com/mattn) in [#​1425](mattn/go-sqlite3#1425) - Fix leak of extension load error message by [@​mattn](https://github.com/mattn) in [#​1426](mattn/go-sqlite3#1426) - Upgrade SQLite to version [`3053003`](mattn/go-sqlite3@3053003) by [@​mattn](https://github.com/mattn) in [#​1427](mattn/go-sqlite3#1427) #### New Contributors - [@​charlievieth](https://github.com/charlievieth) made their first contribution in [#​1128](mattn/go-sqlite3#1128) - [@​connyay](https://github.com/connyay) made their first contribution in [#​1136](mattn/go-sqlite3#1136) - [@​Spaider](https://github.com/Spaider) made their first contribution in [#​1069](mattn/go-sqlite3#1069) - [@​andrzh](https://github.com/andrzh) made their first contribution in [#​1149](mattn/go-sqlite3#1149) - [@​parthokr](https://github.com/parthokr) made their first contribution in [#​1163](mattn/go-sqlite3#1163) - [@​leso-kn](https://github.com/leso-kn) made their first contribution in [#​1177](mattn/go-sqlite3#1177) - [@​pomadev](https://github.com/pomadev) made their first contribution in [#​1222](mattn/go-sqlite3#1222) - [@​dustin-ward](https://github.com/dustin-ward) made their first contribution in [#​1239](mattn/go-sqlite3#1239) - [@​jonstacks](https://github.com/jonstacks) made their first contribution in [#​1300](mattn/go-sqlite3#1300) - [@​samjewell](https://github.com/samjewell) made their first contribution in [#​1290](mattn/go-sqlite3#1290) - [@​q66](https://github.com/q66) made their first contribution in [#​1262](mattn/go-sqlite3#1262) - [@​hionay](https://github.com/hionay) made their first contribution in [#​1259](mattn/go-sqlite3#1259) - [@​justinsb](https://github.com/justinsb) made their first contribution in [#​1305](mattn/go-sqlite3#1305) - [@​eraytufan](https://github.com/eraytufan) made their first contribution in [#​1346](mattn/go-sqlite3#1346) - [@​nautaa](https://github.com/nautaa) made their first contribution in [#​1362](mattn/go-sqlite3#1362) - [@​dsonck92](https://github.com/dsonck92) made their first contribution in [#​1364](mattn/go-sqlite3#1364) - [@​Jaculabilis](https://github.com/Jaculabilis) made their first contribution in [#​1313](mattn/go-sqlite3#1313) - [@​theimpostor](https://github.com/theimpostor) made their first contribution in [#​1379](mattn/go-sqlite3#1379) - [@​calmh](https://github.com/calmh) made their first contribution in [#​1338](mattn/go-sqlite3#1338) - [@​dxbjavid](https://github.com/dxbjavid) made their first contribution in [#​1402](mattn/go-sqlite3#1402) **Full Changelog**: <mattn/go-sqlite3@v1.14.16...v1.14.48> </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xMDEuMSIsInVwZGF0ZWRJblZlciI6IjQzLjEwMS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJ0eXBlL3BhdGNoIl19--> Reviewed-on: https://git.erwanleboucher.dev/eleboucher/apoci/pulls/140
callbackArgString hands a TEXT argument to a custom function by converting it with C.GoString, which stops at the first NUL byte, so a function declared with a string parameter silently loses everything after an embedded NUL in the value it was given; callbackRetText has the mirror problem and returns its string through sqlite3_result_text with a -1 length, so a returned string is cut off at its first NUL on the way back into the database. A function that filters or validates its input then only sees the bytes before the NUL, which can let the rest slip through unchecked. I read the real byte count with sqlite3_value_bytes on the way in and pass the explicit length to sqlite3_result_text on the way out, with the same oversize guard the blob path already carries since we now hand it a real length.