dpkg status files have base64 filenames

[johngmyers]

Our image scanner is having trouble determining the versions of things in distroless-based images.

The files in var/lib/dpkg/status.d now have filenames that appear to be encoded in base64:

drwxr-xr-x  0 0      0           0 Dec 31  1969 var/lib/dpkg/
drwxr-xr-x  0 0      0           0 Dec 31  1969 var/lib/dpkg/status.d/
-rw-r--r--  0 0      0         720 Dec 31  1969 var/lib/dpkg/status.d/YmFzZS1maWxlcw==
-rw-r--r--  0 0      0         859 Dec 31  1969 var/lib/dpkg/status.d/Zm9udGNvbmZpZy1jb25maWc=
-rw-r--r--  0 0      0        1105 Dec 31  1969 var/lib/dpkg/status.d/Zm9udHMtZGVqYXZ1LWNvcmU=
-rw-r--r--  0 0      0        1525 Dec 31  1969 var/lib/dpkg/status.d/b3Blbmpkay0xMS1qcmUtaGVhZGxlc3M=
-rw-r--r--  0 0      0         918 Dec 31  1969 var/lib/dpkg/status.d/b3BlbnNzbA==
-rw-r--r--  0 0      0         846 Dec 31  1969 var/lib/dpkg/status.d/bGliYzY=
-rw-r--r--  0 0      0         511 Dec 31  1969 var/lib/dpkg/status.d/bGliZ29tcDE=
-rw-r--r--  0 0      0         576 Dec 31  1969 var/lib/dpkg/status.d/bGliZ2NjMQ==
-rw-r--r--  0 0      0         594 Dec 31  1969 var/lib/dpkg/status.d/bGliZXhwYXQx
-rw-r--r--  0 0      0         874 Dec 31  1969 var/lib/dpkg/status.d/bGliZm9udGNvbmZpZzE=
-rw-r--r--  0 0      0        1500 Dec 31  1969 var/lib/dpkg/status.d/bGliZnJlZXR5cGU2
-rw-r--r--  0 0      0         794 Dec 31  1969 var/lib/dpkg/status.d/bGlianBlZzYyLXR1cmJv
-rw-r--r--  0 0      0         762 Dec 31  1969 var/lib/dpkg/status.d/bGlibGNtczItMg==
-rw-r--r--  0 0      0         791 Dec 31  1969 var/lib/dpkg/status.d/bGlic3NsMS4x
-rw-r--r--  0 0      0        3859 Dec 31  1969 var/lib/dpkg/status.d/bGlic3RkYysrNg==
-rw-r--r--  0 0      0         582 Dec 31  1969 var/lib/dpkg/status.d/bGlicG5nMTYtMTY=
-rw-r--r--  0 0      0         708 Dec 31  1969 var/lib/dpkg/status.d/bGlidXVpZDE=
-rw-r--r--  0 0      0         414 Dec 31  1969 var/lib/dpkg/status.d/bmV0YmFzZQ==
-rw-r--r--  0 0      0         683 Dec 31  1969 var/lib/dpkg/status.d/dHpkYXRh
-rw-r--r--  0 0      0         528 Dec 31  1969 var/lib/dpkg/status.d/emxpYjFn

Prior to the Python 3 PRs landing, they looked like:

drwxr-xr-x  0 0      0           0 Dec 31  1969 var/lib/dpkg/
drwxr-xr-x  0 0      0           0 Dec 31  1969 var/lib/dpkg/status.d/
-rw-r--r--  0 0      0         720 Dec 31  1969 var/lib/dpkg/status.d/base
-rw-r--r--  0 0      0        1105 Dec 31  1969 var/lib/dpkg/status.d/fonts
-rw-r--r--  0 0      0         846 Dec 31  1969 var/lib/dpkg/status.d/libc6
-rw-r--r--  0 0      0         594 Dec 31  1969 var/lib/dpkg/status.d/libexpat1
-rw-r--r--  0 0      0         874 Dec 31  1969 var/lib/dpkg/status.d/libfontconfig1
-rw-r--r--  0 0      0         576 Dec 31  1969 var/lib/dpkg/status.d/libgcc1
-rw-r--r--  0 0      0         511 Dec 31  1969 var/lib/dpkg/status.d/libgomp1
-rw-r--r--  0 0      0         762 Dec 31  1969 var/lib/dpkg/status.d/liblcms2
-rw-r--r--  0 0      0         582 Dec 31  1969 var/lib/dpkg/status.d/libpng16
-rw-r--r--  0 0      0         791 Dec 31  1969 var/lib/dpkg/status.d/libssl1
-rw-r--r--  0 0      0        3859 Dec 31  1969 var/lib/dpkg/status.d/libstdc
-rw-r--r--  0 0      0         708 Dec 31  1969 var/lib/dpkg/status.d/libuuid1
-rw-r--r--  0 0      0         414 Dec 31  1969 var/lib/dpkg/status.d/netbase
-rw-r--r--  0 0      0        1525 Dec 31  1969 var/lib/dpkg/status.d/openjdk
-rw-r--r--  0 0      0         918 Dec 31  1969 var/lib/dpkg/status.d/openssl
-rw-r--r--  0 0      0         683 Dec 31  1969 var/lib/dpkg/status.d/tzdata
-rw-r--r--  0 0      0         528 Dec 31  1969 var/lib/dpkg/status.d/zlib1g

Was this an intentional change?

[dlorenc]

I noticed this too, and wasn't able to fully understand it but I think it had something to do with python 2 vs 3. We use a special flag in the cloud build releases to force python 2.

[loosebazooka]

@dlorenc that's gone, there's no more building with python2

[dlorenc]

Ah right. @johngmyers what version of python are you using? We force 3 here: https://github.com/GoogleContainerTools/distroless/blob/main/cloudbuild.yaml#L23

[dlorenc]

Ah sorry I misread - you're saying this is in the default images, not ones you're building yourself?

[johngmyers]

I see this with the official distroless java11 image, so I think that eliminates my build chain

[johngmyers]

I see this in both the official images and the ones I'm building myself.

[chanseokoh]

FYI #581 #741
I also noticed back then that the package names are incomplete in that they are truncated before special characters.

[chanseokoh]

they are truncated before special characters.

So for example, I think libstdc should be libstdc++6, openjdk be openjdk-11-jre-headless, etc.

[loosebazooka]

they are truncated before special characters.

So for example, I think libstdc should be libstdc++6, openjdk be openjdk-11-jre-headless, etc.

has this always been an issue?

[chanseokoh]

I don't know. I didn't care. No one really complained though. It's just that it doesn't look correct. Maybe a vulnerability scanner will not be able to recognize installed packages and their versions.

[loosebazooka]

k well I'll try to fix the py2 -> py3 migration issue for now, and then have someone look at the naming issue

[evanj]

IIRC, I think this comes from download_dpkg in the package_manager directory:

distroless/package_manager/dpkg_parser.py

Line 114 in 5ab7f98

out_file = os.path.join("file", util.encode_package_name(pkg_name))

I suspect this is because Bazel has issues with special characters in file names (e.g. +), although I think it may have relaxed the restrictions in the last 2 years, so it could work now? bazelbuild/bazel#374

[loosebazooka]

So this should be fixed with: bazelbuild/rules_docker#1922 (which fixes parsing metadata, not the filenames)

[pombredanne]

@chanseokoh you wrote in #787 (comment)

Maybe a vulnerability scanner will not be able to recognize installed packages and their versions.

That's the point I made in #741 ... basically official distroless images makeup is essentially NOT observable and IMHO this lack of observability makes them poorly suited to be used --more or less-- as base images of the whole K8S ecosystem. What can I do to help?