autolink returns invalid string when extracting a url followed by a multibyte character

[goulvench]

When a string contains an email immediately followed by a multibyte character, autolink captures the email plus the first byte of the next character, causing invalid strings.

require "redcarpet"
renderer = Redcarpet::Render::HTML.new                                                                            
md = Redcarpet::Markdown.new(renderer, autolink: true)
                                                   
result = md.render("contact@example.com中")
# "<p><a href=\"mailto:contact@example.com%E4\">contact@example.com\xE4</a>\xB8\xAD</p>\n"
result.valid_encoding? # false

# Expected:
# "<p><a href=\"mailto:contact@example.com\">contact@example.com</a>中</p>\n"
result.valid_encoding? # true

The rendered output is invalid UTF-8 because it contains \xE4, the stray leading byte from 中.

Cause

isalnum() returns true for lead bytes of a valid multibyte sequence (on systems using UTF-8 locale at least).

Workaround

If you need to autolink strings which may contain multibyte characters, use scrub to remove invalid characters.

class CustomRenderer < Redcarpet::Render::HTML
  def autolink(url, link_type)
    url.scrub!
    # ...
  end
end