Motivation
In rslib/src/template.rs (TemplateMode::next_token) and rslib/src/cloze.rs (cloze::normal_text), tokenizers iterate character-by-character and execute nom parser combinators on every character. For templates and cloze notes containing substantial HTML/CSS, this causes thousands of redundant failed parser combinator executions.
Proposed change
Fast-forward through non-delimiter slices by scanning for candidate ASCII delimiter bytes ({, <, }) using str::find before invoking the nom parsers, avoiding unnecessary parser overhead on plain text and HTML chunks.
Motivation
In
rslib/src/template.rs(TemplateMode::next_token) andrslib/src/cloze.rs(cloze::normal_text), tokenizers iterate character-by-character and executenomparser combinators on every character. For templates and cloze notes containing substantial HTML/CSS, this causes thousands of redundant failed parser combinator executions.Proposed change
Fast-forward through non-delimiter slices by scanning for candidate ASCII delimiter bytes (
{,<,}) usingstr::findbefore invoking thenomparsers, avoiding unnecessary parser overhead on plain text and HTML chunks.