Skip to content

Prefer charset_normalizer for char detection#6714

Closed
nijel wants to merge 1 commit intopsf:mainfrom
nijel:patch-1
Closed

Prefer charset_normalizer for char detection#6714
nijel wants to merge 1 commit intopsf:mainfrom
nijel:patch-1

Conversation

@nijel
Copy link
Copy Markdown

@nijel nijel commented May 22, 2024

charset_normalizer is a dependency for requests, while chardet is not, so it is more likely to be found.

For vendored requests this would still load whatever is available.

The reason I noticed this memory profiling in case both libraries are present and charset_normalizer has way lower memory footprint than chardet.

charset_normalizer is dependency for requests, while chardet is not, so it is more likely to be found.
@sigmavirus24
Copy link
Copy Markdown
Contributor

Just don't install chardet

@nijel
Copy link
Copy Markdown
Author

nijel commented May 22, 2024

I'd love to, but it's a dependency for a third-party library (otsaloma/gaupol#222).

What is actually the reason for trying chardet first when it's not dependency for requests? It's just an extra import that is very likely to fail.

@github-actions github-actions Bot locked as resolved and limited conversation to collaborators May 23, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants