UTF-8 validation

[piotrrzysko]

The C++ version of simdjson validates character encoding. We would like to have such validation in simdjson-java as well.

Perhaps, as @amCap1712 suggested, we may utilize the great work done by @AugustNagro: https://github.com/AugustNagro/utf8.java

[AugustNagro]

Nice work Piotr! Please feel free to use or copy the utf8 validation code as you wish. While the code is correct you may find new optimization opportunities. The vector api has certainly improved since java 17. I personally have been waiting for the API to graduate from 'incubator' to 'preview' to give the library an overhaul. It's also my (perhaps outdated) understanding that the vector api will only ship after valhalla value types. There was an effort here to improve the library with loop unrolling, etc: https://github.com/merykitty/utf8-validator/tree/master

…

On Wed, Jul 19, 2023, 11:17 PM Piotr Rżysko ***@***.***> wrote: The C++ version of simdjson validates character encoding <https://github.com/simdjson/simdjson/blob/master/src/generic/stage1/utf8_validator.h#L19>. We would like to have such validation in simdjson-java as well. Perhaps, as @amCap1712 <https://github.com/amCap1712> suggested <simdjson/simdjson#121 (comment)>, we may utilize the great work done by @AugustNagro <https://github.com/AugustNagro>: https://github.com/AugustNagro/utf8.java — Reply to this email directly, view it on GitHub <#6>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABQOKNQOYKZG2M6IHQEKTVDXRDEODANCNFSM6AAAAAA2Q5IVCE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[piotrrzysko]

Implemented in #13.