gh-146311: Reject non-canonical padding bits in base32, 64, & 85 decoding (GH-146312)

Add `canonical=False` keyword argument to `a2b_base64`, `a2b_base32`, `a2b_base85`, and `a2b_ascii85` (and their `base64` module wrappers). When `canonical=True`, non-canonical encodings are rejected per [RFC 4648 section 3.5](https://datatracker.ietf.org/doc/html/rfc4648.html#section-3.5).

This is independent of `strict_mode`.

For base85/ascii85, the check also rejects single-character final groups (never produced by a conforming encoder) and verifies partial group padding matches what the encoder would produce.

Co-authored-by: Serhiy Storchaka via lots of great code review!

Author: gpshead

Doc/library/base64.rst

@@ -73,8 +73,8 @@ POST request.
       Added the *padded* and *wrapcol* parameters.
 
 
-.. function:: b64decode(s, altchars=None, validate=False, *, padded=True)
-              b64decode(s, altchars=None, validate=True, *, ignorechars, padded=True)
+.. function:: b64decode(s, altchars=None, validate=False, *, padded=True, canonical=False)
+              b64decode(s, altchars=None, validate=True, *, ignorechars, padded=True, canonical=False)
 
    Decode the Base64 encoded :term:`bytes-like object` or ASCII string
    *s* and return the decoded :class:`bytes`.
@@ -112,10 +112,13 @@ POST request.
    If *validate* is true, these non-alphabet characters in the input
    result in a :exc:`binascii.Error`.
 
+   If *canonical* is true, non-zero padding bits are rejected.
+   See :func:`binascii.a2b_base64` for details.
+
    For more information about the strict base64 check, see :func:`binascii.a2b_base64`
 
    .. versionchanged:: 3.15
-      Added the *ignorechars* and *padded* parameters.
+      Added the *canonical*, *ignorechars*, and *padded* parameters.
 
    .. deprecated:: 3.15
       Accepting the ``+`` and ``/`` characters with an alternative alphabet
@@ -179,7 +182,7 @@ POST request.
       Added the *padded* and *wrapcol* parameters.
 
 
-.. function:: b32decode(s, casefold=False, map01=None, *, padded=True, ignorechars=b'')
+.. function:: b32decode(s, casefold=False, map01=None, *, padded=True, ignorechars=b'', canonical=False)
 
    Decode the Base32 encoded :term:`bytes-like object` or ASCII string *s* and
    return the decoded :class:`bytes`.
@@ -205,12 +208,15 @@ POST request.
    *ignorechars* should be a :term:`bytes-like object` containing characters
    to ignore from the input.
 
+   If *canonical* is true, non-zero padding bits are rejected.
+   See :func:`binascii.a2b_base32` for details.
+
    A :exc:`binascii.Error` is raised if *s* is
    incorrectly padded or if there are non-alphabet characters present in the
    input.
 
    .. versionchanged:: 3.15
-      Added the *ignorechars* and *padded* parameters.
+      Added the *canonical*, *ignorechars*, and *padded* parameters.
 
 
 .. function:: b32hexencode(s, *, padded=True, wrapcol=0)
@@ -224,7 +230,7 @@ POST request.
       Added the *padded* and *wrapcol* parameters.
 
 
-.. function:: b32hexdecode(s, casefold=False, *, padded=True, ignorechars=b'')
+.. function:: b32hexdecode(s, casefold=False, *, padded=True, ignorechars=b'', canonical=False)
 
    Similar to :func:`b32decode` but uses the Extended Hex Alphabet, as defined in
    :rfc:`4648`.
@@ -237,7 +243,7 @@ POST request.
    .. versionadded:: 3.10
 
    .. versionchanged:: 3.15
-      Added the *ignorechars* and *padded* parameters.
+      Added the *canonical*, *ignorechars*, and *padded* parameters.
 
 
 .. function:: b16encode(s, *, wrapcol=0)
@@ -317,7 +323,7 @@ Refer to the documentation of the individual functions for more information.
    .. versionadded:: 3.4
 
 
-.. function:: a85decode(b, *, foldspaces=False, adobe=False, ignorechars=b' \t\n\r\v')
+.. function:: a85decode(b, *, foldspaces=False, adobe=False, ignorechars=b' \t\n\r\v', canonical=False)
 
    Decode the Ascii85 encoded :term:`bytes-like object` or ASCII string *b* and
    return the decoded :class:`bytes`.
@@ -334,8 +340,16 @@ Refer to the documentation of the individual functions for more information.
    This should only contain whitespace characters, and by
    default contains all whitespace characters in ASCII.
 
+   If *canonical* is true, non-canonical encodings are rejected.
+   See :func:`binascii.a2b_ascii85` for details.
+
    .. versionadded:: 3.4
 
+   .. versionchanged:: next
+      Added the *canonical* parameter.
+      Single-character final groups are now always rejected as encoding
+      violations.
+
 
 .. function:: b85encode(b, pad=False, *, wrapcol=0)
 
@@ -355,7 +369,7 @@ Refer to the documentation of the individual functions for more information.
       Added the *wrapcol* parameter.
 
 
-.. function:: b85decode(b, *, ignorechars=b'')
+.. function:: b85decode(b, *, ignorechars=b'', canonical=False)
 
    Decode the base85-encoded :term:`bytes-like object` or ASCII string *b* and
    return the decoded :class:`bytes`.  Padding is implicitly removed, if
@@ -364,10 +378,15 @@ Refer to the documentation of the individual functions for more information.
    *ignorechars* should be a :term:`bytes-like object` containing characters
    to ignore from the input.
 
+   If *canonical* is true, non-canonical encodings are rejected.
+   See :func:`binascii.a2b_base85` for details.
+
    .. versionadded:: 3.4
 
    .. versionchanged:: 3.15
-      Added the *ignorechars* parameter.
+      Added the *canonical* and *ignorechars* parameters.
+      Single-character final groups are now always rejected as encoding
+      violations.
 
 
 .. function:: z85encode(s, pad=False, *, wrapcol=0)
@@ -392,7 +411,7 @@ Refer to the documentation of the individual functions for more information.
       Added the *wrapcol* parameter.
 
 
-.. function:: z85decode(s, *, ignorechars=b'')
+.. function:: z85decode(s, *, ignorechars=b'', canonical=False)
 
    Decode the Z85-encoded :term:`bytes-like object` or ASCII string *s* and
    return the decoded :class:`bytes`.  See `Z85  specification
@@ -401,10 +420,15 @@ Refer to the documentation of the individual functions for more information.
    *ignorechars* should be a :term:`bytes-like object` containing characters
    to ignore from the input.
 
+   If *canonical* is true, non-canonical encodings are rejected.
+   See :func:`binascii.a2b_base85` for details.
+
    .. versionadded:: 3.13
 
    .. versionchanged:: 3.15
-      Added the *ignorechars* parameter.
+      Added the *canonical* and *ignorechars* parameters.
+      Single-character final groups are now always rejected as encoding
+      violations.
 
 
 .. _base64-legacy:

Doc/library/binascii.rst

@@ -48,8 +48,8 @@ The :mod:`!binascii` module defines the following functions:
       Added the *backtick* parameter.
 
 
-.. function:: a2b_base64(string, /, *, padded=True, alphabet=BASE64_ALPHABET, strict_mode=False)
-              a2b_base64(string, /, *, ignorechars, padded=True, alphabet=BASE64_ALPHABET, strict_mode=True)
+.. function:: a2b_base64(string, /, *, padded=True, alphabet=BASE64_ALPHABET, strict_mode=False, canonical=False)
+              a2b_base64(string, /, *, ignorechars, padded=True, alphabet=BASE64_ALPHABET, strict_mode=True, canonical=False)
 
    Convert a block of base64 data back to binary and return the binary data. More
    than one line may be passed at a time.
@@ -83,11 +83,15 @@ The :mod:`!binascii` module defines the following functions:
    * Contains no excess data after padding (including excess padding, newlines, etc.).
    * Does not start with a padding.
 
+   If *canonical* is true, non-zero padding bits in the last group are rejected
+   with :exc:`binascii.Error`, enforcing canonical encoding as defined in
+   :rfc:`4648` section 3.5.  This check is independent of *strict_mode*.
+
    .. versionchanged:: 3.11
       Added the *strict_mode* parameter.
 
    .. versionchanged:: 3.15
-      Added the *alphabet*, *ignorechars* and *padded* parameters.
+      Added the *alphabet*, *canonical*, *ignorechars*, and *padded* parameters.
 
 
 .. function:: b2a_base64(data, *, padded=True, alphabet=BASE64_ALPHABET, wrapcol=0, newline=True)
@@ -113,7 +117,7 @@ The :mod:`!binascii` module defines the following functions:
       Added the *alphabet*, *padded* and *wrapcol* parameters.
 
 
-.. function:: a2b_ascii85(string, /, *, foldspaces=False, adobe=False, ignorechars=b'')
+.. function:: a2b_ascii85(string, /, *, foldspaces=False, adobe=False, ignorechars=b'', canonical=False)
 
    Convert Ascii85 data back to binary and return the binary data.
 
@@ -122,7 +126,8 @@ The :mod:`!binascii` module defines the following functions:
    characters). Each group encodes 32 bits of binary data in the range from
    ``0`` to ``2 ** 32 - 1``, inclusive. The special character ``z`` is
    accepted as a short form of the group ``!!!!!``, which encodes four
-   consecutive null bytes.
+   consecutive null bytes. A single-character final group is always rejected
+   as an encoding violation.
 
    *foldspaces* is a flag that specifies whether the 'y' short sequence
    should be accepted as shorthand for 4 consecutive spaces (ASCII 0x20).
@@ -135,6 +140,12 @@ The :mod:`!binascii` module defines the following functions:
    to ignore from the input.
    This should only contain whitespace characters.
 
+   If *canonical* is true, non-canonical encodings are rejected with
+   :exc:`binascii.Error`.  Here "canonical" means the encoding that
+   :func:`b2a_ascii85` would produce: the ``z`` abbreviation must be used
+   for all-zero groups (rather than ``!!!!!``), and partial final groups
+   must use the same padding digits as the encoder.
+
    Invalid Ascii85 data will raise :exc:`binascii.Error`.
 
    .. versionadded:: 3.15
@@ -163,22 +174,28 @@ The :mod:`!binascii` module defines the following functions:
    .. versionadded:: 3.15
 
 
-.. function:: a2b_base85(string, /, *, alphabet=BASE85_ALPHABET, ignorechars=b'')
+.. function:: a2b_base85(string, /, *, alphabet=BASE85_ALPHABET, ignorechars=b'', canonical=False)
 
    Convert Base85 data back to binary and return the binary data.
    More than one line may be passed at a time.
 
    Valid Base85 data contains characters from the Base85 alphabet in groups
    of five (except for the final group, which may have from two to five
    characters). Each group encodes 32 bits of binary data in the range from
-   ``0`` to ``2 ** 32 - 1``, inclusive.
+   ``0`` to ``2 ** 32 - 1``, inclusive. A single-character final group is
+   always rejected as an encoding violation.
 
    Optional *alphabet* must be a :class:`bytes` object of length 85 which
    specifies an alternative alphabet.
 
    *ignorechars* should be a :term:`bytes-like object` containing characters
    to ignore from the input.
 
+   If *canonical* is true, non-canonical encodings are rejected with
+   :exc:`binascii.Error`.  Here "canonical" means the encoding that
+   :func:`b2a_base85` would produce: partial final groups must use the
+   same padding digits as the encoder.
+
    Invalid Base85 data will raise :exc:`binascii.Error`.
 
    .. versionadded:: 3.15
@@ -202,7 +219,7 @@ The :mod:`!binascii` module defines the following functions:
    .. versionadded:: 3.15
 
 
-.. function:: a2b_base32(string, /, *, padded=True, alphabet=BASE32_ALPHABET, ignorechars=b'')
+.. function:: a2b_base32(string, /, *, padded=True, alphabet=BASE32_ALPHABET, ignorechars=b'', canonical=False)
 
    Convert base32 data back to binary and return the binary data.
 
@@ -231,6 +248,10 @@ The :mod:`!binascii` module defines the following functions:
    presented before the end of the encoded data and the excess pad characters
    will be ignored.
 
+   If *canonical* is true, non-zero padding bits in the last group are rejected
+   with :exc:`binascii.Error`, enforcing canonical encoding as defined in
+   :rfc:`4648` section 3.5.
+
    Invalid base32 data will raise :exc:`binascii.Error`.
 
    .. versionadded:: 3.15

Doc/whatsnew/3.15.rst

@@ -729,6 +729,15 @@ base64
   :func:`~base64.z85decode`.
   (Contributed by Serhiy Storchaka in :gh:`144001` and :gh:`146431`.)
 
+* Added the *canonical* parameter in
+  :func:`~base64.b32decode`, :func:`~base64.b32hexdecode`,
+  :func:`~base64.b64decode`, :func:`~base64.urlsafe_b64decode`,
+  :func:`~base64.a85decode`, :func:`~base64.b85decode`, and
+  :func:`~base64.z85decode`,
+  to reject encodings with non-zero padding bits or other non-canonical
+  forms.
+  (Contributed by Gregory P. Smith in :gh:`146311`.)
+
 
 binascii
 --------
@@ -762,6 +771,10 @@ binascii
   :func:`~binascii.unhexlify`, and :func:`~binascii.a2b_base64`.
   (Contributed by Serhiy Storchaka in :gh:`144001` and :gh:`146431`.)
 
+* Added the *canonical* parameter in :func:`~binascii.a2b_base64`,
+  to reject encodings with non-zero padding bits.
+  (Contributed by Gregory P. Smith in :gh:`146311`.)
+
 
 calendar
 --------

Include/internal/pycore_global_objects_fini_generated.h

@@ -359,6 +359,7 @@ struct _Py_global_strings {
         STRUCT_FOR_ID(callable)
         STRUCT_FOR_ID(callback)
         STRUCT_FOR_ID(cancel)
+        STRUCT_FOR_ID(canonical)
         STRUCT_FOR_ID(capath)
         STRUCT_FOR_ID(capitals)
         STRUCT_FOR_ID(category)