Skip to content

preserve embedded NUL bytes in custom function text values - #1406

Merged
mattn merged 2 commits into
mattn:masterfrom
dxbjavid:function-text-embedded-nul
Jun 9, 2026
Merged

preserve embedded NUL bytes in custom function text values#1406
mattn merged 2 commits into
mattn:masterfrom
dxbjavid:function-text-embedded-nul

Conversation

@dxbjavid

@dxbjavid dxbjavid commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

callbackArgString hands a TEXT argument to a custom function by converting it with C.GoString, which stops at the first NUL byte, so a function declared with a string parameter silently loses everything after an embedded NUL in the value it was given; callbackRetText has the mirror problem and returns its string through sqlite3_result_text with a -1 length, so a returned string is cut off at its first NUL on the way back into the database. A function that filters or validates its input then only sees the bytes before the NUL, which can let the rest slip through unchecked. I read the real byte count with sqlite3_value_bytes on the way in and pass the explicit length to sqlite3_result_text on the way out, with the same oversize guard the blob path already carries since we now hand it a real length.

@codecov-commenter

codecov-commenter commented Jun 9, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 62.50000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 51.63%. Comparing base (18cdded) to head (b1e8d68).
⚠️ Report is 95 commits behind head on master.

Files with missing lines Patch % Lines
callback.go 62.50% 2 Missing and 1 partial ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1406      +/-   ##
==========================================
+ Coverage   47.16%   51.63%   +4.47%     
==========================================
  Files          12       13       +1     
  Lines        1533     1867     +334     
==========================================
+ Hits          723      964     +241     
- Misses        669      742      +73     
- Partials      141      161      +20     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mattn

mattn commented Jun 9, 2026

Copy link
Copy Markdown
Owner

LGTM. Correctly preserves embedded NULs on both paths, guard matches the blob path, and the test covers both directions. Thanks!

Nit (non-blocking): the TEXT arg case calls sqlite3_value_bytes before sqlite3_value_text; SQLite's documented order is text-then-bytes. Consistent with the existing BLOB case and harmless on the UTF-8 path, so fine to leave.

@mattn

mattn commented Jun 9, 2026

Copy link
Copy Markdown
Owner

Ref for the ordering note: https://www.sqlite.org/c3ref/value_blob.html

@dxbjavid

dxbjavid commented Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

good catch, went ahead and flipped it to text-then-bytes on the TEXT case so it follows the documented order in that ref. left the existing BLOB case as-is since it predates this and isn't really in scope here.

@mattn
mattn merged commit 518ffdb into mattn:master Jun 9, 2026
11 checks passed
@mattn

mattn commented Jun 9, 2026

Copy link
Copy Markdown
Owner

Thank you

@dxbjavid

dxbjavid commented Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

Glad to contribute. Thanks

eleboucher pushed a commit to eleboucher/apoci that referenced this pull request Jul 14, 2026
…4.48) (#140)

This PR contains the following updates:

| Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|
| [github.com/mattn/go-sqlite3](https://github.com/mattn/go-sqlite3) | `v1.14.47` → `v1.14.48` | ![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fmattn%2fgo-sqlite3/v1.14.48?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fmattn%2fgo-sqlite3/v1.14.47/v1.14.48?slim=true) |

---

### Release Notes

<details>
<summary>mattn/go-sqlite3 (github.com/mattn/go-sqlite3)</summary>

### [`v1.14.48`](https://github.com/mattn/go-sqlite3/releases/tag/v1.14.48): 1.14.48

[Compare Source](mattn/go-sqlite3@v1.14.47...v1.14.48)

#### What's Changed

- Add Serialize and Deserialize support by [@&#8203;otoolep](https://github.com/otoolep) in [#&#8203;1089](mattn/go-sqlite3#1089)
- Replace namedValue with driver.NamedValue to avoid copying exec/query args by [@&#8203;charlievieth](https://github.com/charlievieth) in [#&#8203;1128](mattn/go-sqlite3#1128)
- Add go 1.20 to workflow matrix, remove 1.17 by [@&#8203;connyay](https://github.com/connyay) in [#&#8203;1136](mattn/go-sqlite3#1136)
- Add build tags to support both x86 and ARM compilation on macOS by [@&#8203;Spaider](https://github.com/Spaider) in [#&#8203;1069](mattn/go-sqlite3#1069)
- Fix virtual table example. by [@&#8203;andrzh](https://github.com/andrzh) in [#&#8203;1149](mattn/go-sqlite3#1149)
- Update README.md by [@&#8203;parthokr](https://github.com/parthokr) in [#&#8203;1163](mattn/go-sqlite3#1163)
- Update amalgamation code by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1166](mattn/go-sqlite3#1166)
- Update amalgamation code by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1197](mattn/go-sqlite3#1197)
- Fix docker job by [@&#8203;itizir](https://github.com/itizir) in [#&#8203;1201](mattn/go-sqlite3#1201)
- Fix musl build ([#&#8203;1164](mattn/go-sqlite3#1164)) by [@&#8203;leso-kn](https://github.com/leso-kn) in [#&#8203;1177](mattn/go-sqlite3#1177)
- update go version to 1.19 by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1208](mattn/go-sqlite3#1208)
- Update amalgamation code to 3.45.0 by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1207](mattn/go-sqlite3#1207)
- Update amalgamation code to 3.45.1 by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1211](mattn/go-sqlite3#1211)
- close channel by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1213](mattn/go-sqlite3#1213)
- fix: some typos by [@&#8203;pomadev](https://github.com/pomadev) in [#&#8203;1222](mattn/go-sqlite3#1222)
- Add support for libsqlite3 on z/OS by [@&#8203;dustin-ward](https://github.com/dustin-ward) in [#&#8203;1239](mattn/go-sqlite3#1239)
- Update amalgamation code to 3.46.1 by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1273](mattn/go-sqlite3#1273)
- close statement when missing query arguments by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1281](mattn/go-sqlite3#1281)
- Upgrade upload-artifact action by [@&#8203;jonstacks](https://github.com/jonstacks) in [#&#8203;1300](mattn/go-sqlite3#1300)
- Remove suggestion that CGO isn't always needed by [@&#8203;samjewell](https://github.com/samjewell) in [#&#8203;1290](mattn/go-sqlite3#1290)
- remove superfluous use of runtime.SetFinalizer on SQLiteRows by [@&#8203;charlievieth](https://github.com/charlievieth) in [#&#8203;1301](mattn/go-sqlite3#1301)
- Fix sqlite3\_opt\_unlock\_notify with USE\_LIBSQLITE3 by [@&#8203;q66](https://github.com/q66) in [#&#8203;1262](mattn/go-sqlite3#1262)
- Fix memory leak in callbackRetText function by [@&#8203;hionay](https://github.com/hionay) in [#&#8203;1259](mattn/go-sqlite3#1259)
- docs: clarify GCP section by [@&#8203;justinsb](https://github.com/justinsb) in [#&#8203;1305](mattn/go-sqlite3#1305)
- Add ability to set an int64 file control by [@&#8203;jonstacks](https://github.com/jonstacks) in [#&#8203;1298](mattn/go-sqlite3#1298)
- Update amalgamation code to 3.49.1 by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1335](mattn/go-sqlite3#1335)
- Update amalgamation code to 3.50.3 by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1343](mattn/go-sqlite3#1343)
- Drop userauth implementation by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1344](mattn/go-sqlite3#1344)
- fix syntax error by [@&#8203;eraytufan](https://github.com/eraytufan) in [#&#8203;1346](mattn/go-sqlite3#1346)
- update amalgamation code by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1347](mattn/go-sqlite3#1347)
- use quote include instead of angled include for sqlite3-binding.h by [@&#8203;nautaa](https://github.com/nautaa) in [#&#8203;1362](mattn/go-sqlite3#1362)
- Upgrade SQLite to version [`3051001`](mattn/go-sqlite3@3051001) by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1366](mattn/go-sqlite3#1366)
- Feat: add percentile extension option by [@&#8203;dsonck92](https://github.com/dsonck92) in [#&#8203;1364](mattn/go-sqlite3#1364)
- Upgrade SQLite to version [`3051002`](mattn/go-sqlite3@3051002) by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1370](mattn/go-sqlite3#1370)
- Use unsafe slice by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1373](mattn/go-sqlite3#1373)
- Call sqlite3\_clear\_bindings() after sqlite3\_reset() in bind() by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1374](mattn/go-sqlite3#1374)
- Upgrade SQLite to version [`3051003`](mattn/go-sqlite3@3051003) by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1375](mattn/go-sqlite3#1375)
- Ensure Close always removes runtime finalizer to prevent memory leak by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1376](mattn/go-sqlite3#1376)
- Fix json example by [@&#8203;Jaculabilis](https://github.com/Jaculabilis) in [#&#8203;1313](mattn/go-sqlite3#1313)
- Add missing virtual table constraint op constants by [@&#8203;theimpostor](https://github.com/theimpostor) in [#&#8203;1379](mattn/go-sqlite3#1379)
- Eliminate unnecessary bounds checks in hot paths by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1381](mattn/go-sqlite3#1381)
- \[codex] optimize sqlite bind fast path by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1382](mattn/go-sqlite3#1382)
- \[codex] batch row column fetches in Next by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1383](mattn/go-sqlite3#1383)
- Raise minimum Go version to 1.21 by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1384](mattn/go-sqlite3#1384)
- Reduce sqlite bind overhead by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1385](mattn/go-sqlite3#1385)
- reduce CGO call overhead for exec and bind paths by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1386](mattn/go-sqlite3#1386)
- \[codex] add opt-in statement cache by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1387](mattn/go-sqlite3#1387)
- Fix panic when querying input with no SQL (only comments/whitespace) by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1392](mattn/go-sqlite3#1392)
- evict least-recently-used stmt when cache is full by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1388](mattn/go-sqlite3#1388)
- Upgrade SQLite to version [`3053000`](mattn/go-sqlite3@3053000) by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1394](mattn/go-sqlite3#1394)
- add sqlite\_dbstat tag for the DBSTAT virtual table by [@&#8203;calmh](https://github.com/calmh) in [#&#8203;1338](mattn/go-sqlite3#1338)
- avoid out of bounds write in unlock\_notify\_wait on 64 bit platforms by [@&#8203;calmh](https://github.com/calmh) in [#&#8203;1399](mattn/go-sqlite3#1399)
- modernise reflect.SliceHeader to unsafe.Slice by [@&#8203;calmh](https://github.com/calmh) in [#&#8203;1400](mattn/go-sqlite3#1400)
- guard oversized string length in ResultText by [@&#8203;dxbjavid](https://github.com/dxbjavid) in [#&#8203;1402](mattn/go-sqlite3#1402)
- bind via sqlite3\_bind\_text64/blob64 to avoid 32-bit length truncation by [@&#8203;dxbjavid](https://github.com/dxbjavid) in [#&#8203;1403](mattn/go-sqlite3#1403)
- Upgrade SQLite to version [`3053002`](mattn/go-sqlite3@3053002) by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1404](mattn/go-sqlite3#1404)
- guard oversized blob length in callbackRetBlob by [@&#8203;dxbjavid](https://github.com/dxbjavid) in [#&#8203;1405](mattn/go-sqlite3#1405)
- preserve embedded NUL bytes in custom function text values by [@&#8203;dxbjavid](https://github.com/dxbjavid) in [#&#8203;1406](mattn/go-sqlite3#1406)
- Follow documented call order for sqlite3\_value\_blob in callbackArgString by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1407](mattn/go-sqlite3#1407)
- Use atomic.Value for handle table and add concurrent lookup benchmark by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1412](mattn/go-sqlite3#1412)
- cache column metadata for prepared and cached statements by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1413](mattn/go-sqlite3#1413)
- free leaked schema string in GetFilename by [@&#8203;dxbjavid](https://github.com/dxbjavid) in [#&#8203;1408](mattn/go-sqlite3#1408)
- Fix race in SQLiteStmt.Close by holding conn lock across cache check by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1416](mattn/go-sqlite3#1416)
- Add CodeRabbit as a sponsor by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1417](mattn/go-sqlite3#1417)
- Return error from vtable cursor open instead of ignoring it by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1419](mattn/go-sqlite3#1419)
- Check sqlite3\_malloc64 result in Deserialize by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1420](mattn/go-sqlite3#1420)
- Fix panic when registered functions return named types by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1421](mattn/go-sqlite3#1421)
- Return error instead of silently ignoring unsupported bind types by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1422](mattn/go-sqlite3#1422)
- Add CodeRabbit configuration by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1418](mattn/go-sqlite3#1418)
- Close database on all error paths in Open by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1423](mattn/go-sqlite3#1423)
- Check preupdate value fetch result to avoid NULL dereference by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1424](mattn/go-sqlite3#1424)
- Use C.int in exported callbacks to match C declarations by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1425](mattn/go-sqlite3#1425)
- Fix leak of extension load error message by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1426](mattn/go-sqlite3#1426)
- Upgrade SQLite to version [`3053003`](mattn/go-sqlite3@3053003) by [@&#8203;mattn](https://github.com/mattn) in [#&#8203;1427](mattn/go-sqlite3#1427)

#### New Contributors

- [@&#8203;charlievieth](https://github.com/charlievieth) made their first contribution in [#&#8203;1128](mattn/go-sqlite3#1128)
- [@&#8203;connyay](https://github.com/connyay) made their first contribution in [#&#8203;1136](mattn/go-sqlite3#1136)
- [@&#8203;Spaider](https://github.com/Spaider) made their first contribution in [#&#8203;1069](mattn/go-sqlite3#1069)
- [@&#8203;andrzh](https://github.com/andrzh) made their first contribution in [#&#8203;1149](mattn/go-sqlite3#1149)
- [@&#8203;parthokr](https://github.com/parthokr) made their first contribution in [#&#8203;1163](mattn/go-sqlite3#1163)
- [@&#8203;leso-kn](https://github.com/leso-kn) made their first contribution in [#&#8203;1177](mattn/go-sqlite3#1177)
- [@&#8203;pomadev](https://github.com/pomadev) made their first contribution in [#&#8203;1222](mattn/go-sqlite3#1222)
- [@&#8203;dustin-ward](https://github.com/dustin-ward) made their first contribution in [#&#8203;1239](mattn/go-sqlite3#1239)
- [@&#8203;jonstacks](https://github.com/jonstacks) made their first contribution in [#&#8203;1300](mattn/go-sqlite3#1300)
- [@&#8203;samjewell](https://github.com/samjewell) made their first contribution in [#&#8203;1290](mattn/go-sqlite3#1290)
- [@&#8203;q66](https://github.com/q66) made their first contribution in [#&#8203;1262](mattn/go-sqlite3#1262)
- [@&#8203;hionay](https://github.com/hionay) made their first contribution in [#&#8203;1259](mattn/go-sqlite3#1259)
- [@&#8203;justinsb](https://github.com/justinsb) made their first contribution in [#&#8203;1305](mattn/go-sqlite3#1305)
- [@&#8203;eraytufan](https://github.com/eraytufan) made their first contribution in [#&#8203;1346](mattn/go-sqlite3#1346)
- [@&#8203;nautaa](https://github.com/nautaa) made their first contribution in [#&#8203;1362](mattn/go-sqlite3#1362)
- [@&#8203;dsonck92](https://github.com/dsonck92) made their first contribution in [#&#8203;1364](mattn/go-sqlite3#1364)
- [@&#8203;Jaculabilis](https://github.com/Jaculabilis) made their first contribution in [#&#8203;1313](mattn/go-sqlite3#1313)
- [@&#8203;theimpostor](https://github.com/theimpostor) made their first contribution in [#&#8203;1379](mattn/go-sqlite3#1379)
- [@&#8203;calmh](https://github.com/calmh) made their first contribution in [#&#8203;1338](mattn/go-sqlite3#1338)
- [@&#8203;dxbjavid](https://github.com/dxbjavid) made their first contribution in [#&#8203;1402](mattn/go-sqlite3#1402)

**Full Changelog**: <mattn/go-sqlite3@v1.14.16...v1.14.48>

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xMDEuMSIsInVwZGF0ZWRJblZlciI6IjQzLjEwMS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJ0eXBlL3BhdGNoIl19-->

Reviewed-on: https://git.erwanleboucher.dev/eleboucher/apoci/pulls/140
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants