Skip to content

Troubleshooting

Jump to bottom
Sergio Soto edited this page Apr 8, 2026 · 2 revisions

Troubleshooting

Build Issues

"client version 1.38 is too old"

vastde embeds Docker client API 1.38; Docker Desktop v4.40+ requires 1.40. Fix: Add {"min-api-version": "1.38"} to Docker Engine config (Settings -> Docker Engine).

"run-image does not exist on the daemon"

CNB run image not cached after Docker restart. Fix:

docker pull docker.io/paketobuildpacks/run-jammy-full:0.1.76

"builder image does not exist on the daemon"

Builder image not cached. Fix:

docker pull docker.selab.vastdata.com:5000/vast-builder:latest

Build succeeds but "failed to save image"

Disk full. Check with df -h /. Docker needs ~3-4GB free for the build. Fix: docker system prune -f or free disk space.

"handler file not found" validation error

Running vastde functions build from wrong directory. Fix: Run from the project root (where Dockerfile.fix and functions/ are), not from inside functions/oiio_proxy_generator/.

Deployment Issues

RevisionMissing

Container fails to start on K8s.

  • Missing Dockerfile.fix layer -- apply it after vastde functions build
  • Cached old image -- use unique version tags, not :latest
  • Local revision -- uncheck "Make local revision" in VMS UI

Pipeline shows "InProgress" indefinitely

K8s is pulling the image. Wait 1-2 minutes. If it persists, check that the image tag matches what was pushed to the registry.

401 Unauthorized on vastde commands

Session expired. Fix: Re-run vastde config init with credentials.

Old code still running after update

K8s caches images by tag. Fix: Always use a new version tag (v2.0.1, v2.0.2, etc.) for each build. Never reuse tags.

Runtime Issues

"oiiotool not found in PATH"

Dockerfile.fix PATH doesn't include /layers/apt-buildpack/apt/usr/bin. Verify Dockerfile.fix contains:

ENV PATH="/layers/apt-buildpack/apt/usr/bin:/layers/paketo-buildpacks_cpython/cpython/bin:${PATH}"

"ffmpeg not found in PATH"

Same as above. Both oiiotool and ffmpeg are installed to /layers/apt-buildpack/apt/usr/bin.

"oiiotool colorconvert failed: Could not construct the color transform"

The colorspace name is not recognized by oiiotool's built-in color management. Valid built-in names: linear, sRGB, Rec709 (no dot, no space). If you need ACES names (ACEScg, ARRI LogC), set OCIO_CONFIG_PATH to a valid OCIO config file.

"OpenColorIO::v1::Exception" crash

The OCIO environment variable is set to a non-existent config file. oiiotool reads this env var directly and crashes. Fix: Ensure Dockerfile.fix does NOT set ENV OCIO=.... The v2.0.0 Dockerfile.fix has this removed.

VastDB "connection refused"

  • Check VAST_DB_ENDPOINT is reachable from the pod
  • Verify database-enabled bucket exists with DATABASE protocol
  • Check credentials (VAST_DB_ACCESS_KEY / VAST_DB_SECRET_KEY)

Kafka delivery error

  • Verify KAFKA_BROKER is reachable from the pod
  • Check topic spaceharbor.proxy exists on the broker
  • VAST Event Broker: use VAST local users, not AD/LDAP

Logs

Key log markers to search for:

Marker Meaning
INITIALIZING OIIO-PROXY-GENERATOR 2.0.0 Container started, init() running
Color mode: oiiotool built-in Using built-in transforms (no OCIO config)
OCIO mode: config=... Using OCIO config file
Downloaded ... bytes S3 GET succeeded
Thumbnail generated Step 1 complete
Proxy generated Step 2 complete
Uploading s3://... S3 PUT with ContentType and tags
PROXY GENERATION COMPLETE Full processing succeeded
Persisted proxy_outputs VastDB write successful
Published proxy.generated Kafka event sent
Processing failed Error in oiiotool/ffmpeg/color transform

Clone this wiki locally