fix(search): tokenize ~= and ~* as single fuzzy-match operators#9508
Open
mrbeandev wants to merge 1 commit intoTriliumNext:mainfrom
Open
fix(search): tokenize ~= and ~* as single fuzzy-match operators#9508mrbeandev wants to merge 1 commit intoTriliumNext:mainfrom
mrbeandev wants to merge 1 commit intoTriliumNext:mainfrom
Conversation
The lexer treats `~` as a relation prefix (as in `~author.title`), so when a user wrote `note.title ~= books` or `note.title ~* books` the `~` was emitted as its own token and the `=`/`*` started a second token. The parser then failed with "Unrecognized expression books". Add a dedicated branch that keeps `~=` and `~*` glued together when the current word is the bare tilde, mirroring how `~!` and `#!` are handled one branch above. Relation prefixes such as `~author` are unaffected because they never have `=` or `*` as their first character. Regression coverage is added in lex.spec.ts for both operators, the quoted variant, and a relation-prefix case to guard against accidental tokenization of `~foo` as an operator in the future. Fixes TriliumNext#9426
dosubot
Bot
added
the
size:XS
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request updates the lexer to correctly tokenize fuzzy-match operators ~= and ~* as single units, preventing them from being misinterpreted as a relation prefix followed by an operator. Corresponding regression tests have been added to verify this behavior and ensure that the standard relation prefix ~ remains functional. I have no feedback to provide.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
~and=/*as two separate tokens whenever a query used the fuzzy-match operators~=or~*, so queries such asnote.title ~= booksfailed withUnrecognized expression bookseven though the parser already knows about those operators (parse.tsOPERATORSset).lex.tsthat keeps~=/~*glued together when the current word is the bare~, following the same pattern used one branch above for~!and#!.~authorkeep working because their first character is never=or*.Fixes #9426
Test plan
pnpm exec vitest run src/services/search/services/lex.spec.ts— 27/27 pass, including four new regression cases fornote.title ~= books,note.title ~* books,#author ~= tolkien,#author ~*'lord of the rings', and a~author.title = Tolkienguard for the relation-prefix path.pnpm exec vitest run src/services/search— full search suite 144 passed / 26 skipped, no regressions.