Fix libnvdla_compiler.so COPY path for Jetson 6.x runtime - #2306
Closed
alexnorell wants to merge 3 commits into
Closed
Fix libnvdla_compiler.so COPY path for Jetson 6.x runtime#2306alexnorell wants to merge 3 commits into
alexnorell wants to merge 3 commits into
Conversation
The COPY in PR #2201 used /usr/lib/aarch64-linux-gnu/tegra/, but in nvcr.io/nvidia/l4t-jetpack:r36.x the actual path is /usr/lib/aarch64-linux-gnu/nvidia/. The glob silently matched zero files and produced an empty layer (cached forever), so libnvdla_compiler.so never made it into the runtime image. ONNX Runtime's TRT execution provider then failed to load and silently fell back to CPU. Verified on the AI1 device: ONNXRuntime logs: Failed to load library libonnxruntime_providers_tensorrt.so: libnvdla_compiler.so: cannot open shared object file Falling back to ['CPUExecutionProvider'] and retrying. Inspecting nvcr.io/nvidia/l4t-jetpack:r36.4.0 confirms /usr/lib/aarch64-linux-gnu/nvidia/libnvdla_compiler.so is the correct path.
alexnorell
requested review from
PawelPeczek-Roboflow,
dkosowski87,
grzegorz-roboflow,
hansent,
probicheaux,
rafel-roboflow and
yeldarby
as code owners
May 2, 2026 14:59
Jetson 6.0 is no longer maintained. Drops the dedicated build/test workflows, Dockerfile, container_adapter mappings, and docs referencing roboflow-inference-server-jetson-6.0.0. - Delete .github/workflows/docker.jetson.6.0.0.yml - Delete .github/workflows/test.jetson_6.0.0.yml - Delete docker/dockerfiles/Dockerfile.onnx.jetson.6.0.0 - Drop L4T 36.0 / 36.3 → 6.0.0 entries from container_adapter - Drop matching test parametrize cases - Update install guide and example READMEs to point at 6.2.0
alexnorell
marked this pull request as draft
May 2, 2026 15:08
alexnorell
added a commit
that referenced
this pull request
May 11, 2026
The actual location in nvcr.io/nvidia/l4t-jetpack:r36.4.0 is /usr/lib/aarch64-linux-gnu/nvidia/libnvdla_compiler.so, not tegra/. The original COPY in PR #2201 used the wrong path; the glob silently matched zero files, so the libnvdla fix shipped an empty layer and ONNX Runtime kept falling back to CPU on Jetson 6.x. Same one-line correction as PR #2306. Bundling it here so this PR's build can also actually ship libnvdla_compiler.so.
10 tasks
Contributor
Author
|
Superseded by #2321 |
alexnorell
added a commit
that referenced
this pull request
May 12, 2026
The actual location in nvcr.io/nvidia/l4t-jetpack:r36.4.0 is /usr/lib/aarch64-linux-gnu/nvidia/libnvdla_compiler.so, not tegra/. The original COPY in PR #2201 used the wrong path; the glob silently matched zero files, so the libnvdla fix shipped an empty layer and ONNX Runtime kept falling back to CPU on Jetson 6.x. Same one-line correction as PR #2306. Bundling it here so this PR's build can also actually ship libnvdla_compiler.so.
PawelPeczek-Roboflow
added a commit
that referenced
this pull request
May 14, 2026
… mounts on JP62/JP71, bump JP51 OpenCV (#2321) * Build OpenCV from source with GStreamer + bake NVIDIA gst plugins into JP62 The pip opencv-python wheel ships without GStreamer, and the l4t-cuda runtime base used for the JP62 image has no multimedia stack, so cv2.VideoCapture on Jetson silently falls through to the plain v4l2 ioctl path -- pulling raw 4K YUYV, doing CPU YUV->BGR, and never reaching NVIDIA's hardware engines. Three pieces, all needed together: 1. Compile OpenCV from source in the builder stage with WITH_GSTREAMER=ON, WITH_FFMPEG=ON, WITH_LIBV4L=ON, WITH_CUDA=ON. Replaces the pip-installed opencv-python/opencv-contrib-python with a wheel built from python_loader. Adds a build-time cv2.getBuildInformation() check so a regression in the detected flags fails the build instead of silently shipping CPU-only cv2. 2. Add the GStreamer runtime + PyGObject stack to the runtime stage (libgstreamer1.0-0, plugins-base/good, python3-gi, gir typelibs) so that downstream Python code using "from gi.repository import Gst" to construct hardware capture/encode pipelines can import and run inside the container. 3. Copy NVIDIA's GStreamer plugins (libgstnv*.so) plus the tegra runtime libs from the JetPack builder stage into the runtime image, instead of relying on nvidia-container-passthrough host mounts. Self-contained -- works without any host-side changes deployed. Branched off v1.2.7 to keep the diff minimal. * Pass CMAKE_POLICY_VERSION_MINIMUM=3.5 to OpenCV cmake CMake 4.x removed compatibility with cmake_minimum_required(VERSION <3.5). OpenCV 4.10.0's OpenCVGenPkgconfig.cmake still uses the old declaration, so the configure step aborts with: CMake Error at .../OpenCVGenPkgconfig.cmake:113 (cmake_minimum_required): Compatibility with CMake < 3.5 has been removed from CMake. Same workaround the existing onnxruntime build step uses. * Bump OpenCV to 4.13.0 and reorder JP62 build OpenCV 4.13.0 (released 2025-12-31) is the current latest stable; it builds cleanly under CMake 4.x without the cmake_minimum_required workaround the 4.10.0 source needed. Aligns both Jetson images. JP62-specific: - Move the from-source OpenCV step to AFTER the inference packages install so the cv2 we install (built with GStreamer + FFmpeg) is the last thing to touch /usr/local/lib/python3.10/dist-packages/cv2/. Previously the inference_core/cli/sdk pip install would refetch opencv-python (as a transitive dep) and clobber our from-source cv2. - Drop CMAKE_POLICY_VERSION_MINIMUM=3.5 -- not needed on 4.13.0. * Drop the bake-in attempt for NVIDIA gst plugins The l4t-jetpack:r36.4.0 builder image does not ship the tegra runtime libs as a regular populated /usr/lib/aarch64-linux-gnu/tegra directory. Those libs are tied to the host JetPack BSP and only appear inside the container at run time via nvidia-container-runtime's CSV-driven host mounts. So `COPY --from=builder /usr/lib/aarch64-linux-gnu/tegra ...` errors during the build with "not found". This image now scopes to enabling the cv2 side of the GStreamer stack (WITH_GSTREAMER=ON OpenCV, python-gi, base gstreamer runtime + GIR typelibs). The host system has to provide the NVIDIA plugins for any nv* pipeline element to resolve at run time. * mkdir tegra/ at end of builder so libnvdla glob COPY can lstat parent BuildKit needs to lstat the source parent dir before evaluating a glob COPY pattern, even when "no match" is allowed. The libnvdla fix in PR #2201 copies tegra/libnvdla_compiler.so*, but l4t-jetpack:r36.4.0 doesn't always ship /usr/lib/aarch64-linux-gnu/tegra/ as a real directory -- that path is normally populated by nvidia-container-runtime at container start. Recent builds intermittently fail at this step depending on layer cache state. mkdir -p the path so BuildKit can process the COPY. The glob will copy the real file if it's there, or silently no-op if not. Doesn't paper over the underlying flakiness of the libnvdla source path, but lets this PR's build progress. * Correct libnvdla_compiler.so source path from tegra/ to nvidia/ The actual location in nvcr.io/nvidia/l4t-jetpack:r36.4.0 is /usr/lib/aarch64-linux-gnu/nvidia/libnvdla_compiler.so, not tegra/. The original COPY in PR #2201 used the wrong path; the glob silently matched zero files, so the libnvdla fix shipped an empty layer and ONNX Runtime kept falling back to CPU on Jetson 6.x. Same one-line correction as PR #2306. Bundling it here so this PR's build can also actually ship libnvdla_compiler.so. * Set OpenCV python install path so the loader stub finds cv2 at runtime Without OPENCV_PYTHON3_INSTALL_PATH / PYTHON3_INCLUDE_DIR / PYTHON_VERSION the python_loader config.py bakes the BUILDER path (/build/opencv/<ver>/release/lib/python3) into sys.path. That path only exists in the builder stage, so in the runtime image the loader stub fails to find the native cv2 .so, falls back to importlib.import_module("cv2") which re-enters the same stub package, and Python raises: ImportError: ERROR: recursion is detected during loading of "cv2" binary extensions. Check OpenCV installation. JP51's PR #2100 passes these flags explicitly; replicating the same set here so the loader references /usr/local/lib/python3.10/dist-packages. * Rescue install-tree cv2 config files; parameterize for porting Two related changes to make the from-source OpenCV build correct and reusable across jp5.1 / jp6.2 / jp7.1: 1. Rescue the install-tree config files. After ninja install, the files at ${INSTALL_PATH}/cv2/config*.py have install-tree paths (correct). The pip install of the wheel built from /build/.../python_loader then overwrites them with BUILD-TREE config files (paths like /build/opencv/opencv-X.Y.Z/release/lib/python3) which only exist in the builder stage. In a multi-stage runtime image those paths are dead, so the loader's bootstrap can't find the native cv2 .so, re-imports the stub package, and Python raises: ImportError: recursion is detected during loading of "cv2" Fix by saving config*.py between ninja install and pip wheel/install, restoring after. 2. Pull per-platform variables out of the cmake invocation into ARGs at the top of the builder (OPENCV_CUDA_ARCH, OPENCV_PYTHON_INSTALL_PATH, OPENCV_PYTHON_INCLUDE_DIR, OPENCV_PYTHON_VERSION). Replicating this OpenCV block in jp5.1.1 / jp7.1.0 dockerfiles is now a copy-paste plus four ARG value changes; no surgery inside the cmake invocation. Adds a build-time sanity check that simulates the runtime stage by stripping /build/ from sys.path before importing cv2. If the install- tree rescue ever regresses, the docker build fails here -- not 90 minutes later on the device. * Add BuildKit cache mounts to Jetson builds and split Depot projects Pins syntax to dockerfile:1.7 on both jp62 and jp71 so ARG interpolation in cache-mount ids works. Adds apt cache mounts (sharing=locked, id scoped per JetPack + builder/runtime + arch) to every apt RUN so .debs and lists persist across builds. Disables docker-clean and turns on Keep-Downloaded-Packages so the apt cache is actually used. Drops the trailing rm of /var/lib/apt/lists/* since that path is now the cache mount. Adds per-build-tree cache mounts on the expensive C++ compiles: PyTorch, torchvision, onnxruntime (build/cuda12 on jp62, build/Linux on jp71), OpenCV (release/ on jp62), with the TRT .deb download also cached on jp62. Each id is scoped by version + arch + SM so a version bump starts clean instead of reusing a stale build tree. The OpenCV step on jp62 stays positioned last in the builder because the inference wheel installs upstream are not --no-deps and an earlier opencv-python pull would clobber the from-source build. The build-dir cache mount makes the forced rebuild after COPY-invalidation a fast ninja no-op rather than a 30-60 min recompile. Points jp62 and jp71 workflows at the new dedicated Depot projects (2rp7mfjw7q for jp62, v1xzfwkc4b for jp71) so they no longer share an NVMe cache pool and evict each other. * Split git-clone out of cache-mounted build RUNs The cache mount on build/ auto-creates its parent directory before the RUN body executes, so a combined 'git clone ... && cd ... && make' fails with 'destination path already exists and is not an empty directory'. First jp62 attempt with these cache mounts failed at the PyTorch step: fatal: destination path 'pytorch' already exists and is not an empty directory. Split each of PyTorch / torchvision / onnxruntime on jp62 and jp71 into two RUNs: a clone RUN with no mount, then a build RUN whose cache mount overlays build/ on the cloned source layer. * Bump all GitHub Actions to latest major versions Bumps: - actions/cache v3 -> v5 - actions/checkout v2/v3/v4 -> v6 - actions/create-github-app-token v1 -> v3 - actions/setup-node v3 -> v6 - actions/setup-python v2/v5 -> v6 - actions/upload-artifact v4 -> v7 - aws-actions/amazon-ecr-login v1 -> v2 - aws-actions/configure-aws-credentials v2 -> v6 - dcarbone/install-jq-action v2.1.0 -> v3.2.0 - digicert/ssm-code-signing v1.0.0 -> v1.2.1 - docker/login-action v2/v3 -> v4 - docker/setup-buildx-action v2 -> v4 - docker/setup-qemu-action v2 -> v4 - google-github-actions/auth v2 -> v3 - google-github-actions/setup-gcloud v2 -> v3 depot/* already pinned to floating v1 (latest major). actions/upload-release-asset and pypa/gh-action-pypi-publish kept on their existing pins (v1 already latest; pypi-publish uses the release/v1 stable-channel ref recommended by upstream). * Bump jp71 flash-attn MAX_JOBS from 1 to 2 PR #2068 reduced this to 1 because MAX_JOBS=4 was starving the Depot BuildKit gRPC connection during flash-attn CUDA kernel compilation. 2 is a middle ground: flash-attn's nvcc instances peak ~3-4 GB each against PyTorch 2.10 headers, so 2 concurrent fits comfortably on a 32 GB arm builder while halving the wall time vs serial. With the cache mounts from c5701f2 in place, the cost of a regression here is ~30-60 min of flash-attn rebuild, not the full 4hr build. jp62 stays at MAX_JOBS=4 because it's on PyTorch 2.6 (lighter headers) and that build is already known-good. --------- Co-authored-by: Paweł Pęczek <146137186+PawelPeczek-Roboflow@users.noreply.github.com>
2 tasks
grzegorz-roboflow
pushed a commit
that referenced
this pull request
Sep 4, 2026
…egra/ (#2924) "Build and Push Jetson 6.X Container" has failed on every push to main since 2026-06-19 with: COPY --from=builder /usr/lib/aarch64-linux-gnu/tegra/libnvdla_compiler.so* /usr/local/cuda/lib64/ ERROR: lstat .../usr/lib/aarch64-linux-gnu/tegra: no such file or directory In l4t-jetpack r36.x the library lives at /usr/lib/aarch64-linux-gnu/nvidia/libnvdla_compiler.so. The tegra/ dir is populated by nvidia-container-runtime host mounts at run time only, so it is absent in the builder stage and BuildKit cannot lstat the glob's parent. #2321 made the same one-line correction for Dockerfile.onnx.jetson.6.2.0 but left the 6.0.0 Dockerfile on the old path (#2306 covered both and was superseded). This also means the runtime image now actually ships libnvdla_compiler.so, which the ORT TensorRT EP needs to load (#2201). Claude-Session: https://claude.ai/code/session_01QxRB1NT6sXZpDiMcTDBg9t Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
COPY --from=buildersource path from/usr/lib/aarch64-linux-gnu/tegra/libnvdla_compiler.so*to/usr/lib/aarch64-linux-gnu/nvidia/libnvdla_compiler.so*in bothDockerfile.onnx.jetson.6.0.0andDockerfile.onnx.jetson.6.2.0.nvcr.io/nvidia/l4t-jetpack:r36.x, so the glob silently matched zero files and produced an empty layer. That layer cached forever, so the fix in PR Fix missing libnvdla_compiler.so in Jetson 6.x TRT runtime #2201 never actually shipped a workinglibnvdla_compiler.sointo the runtime image.Why this matters
On Jetson 6.x devices, ONNX Runtime's TensorRT execution provider needs
libnvdla_compiler.so. Without it:```
[E:onnxruntime:Default] Failed to load library libonnxruntime_providers_tensorrt.so:
libnvdla_compiler.so: cannot open shared object file: No such file or directory
EP Error ... Falling back to ['CPUExecutionProvider'] and retrying.
```
The model then runs entirely on CPU, ~3x slower (we observed 21 FPS camera vs. 1.3 FPS inference on an AI1/Orin NX).
Verification
Inspected
nvcr.io/nvidia/l4t-jetpack:r36.4.0directly:```
$ docker run --rm --platform linux/arm64 nvcr.io/nvidia/l4t-jetpack:r36.4.0
sh -c 'find / -name libnvdla*'
/usr/lib/aarch64-linux-gnu/nvidia/libnvdla_compiler.so ← actual location
```
Test plan
workflow_dispatchwith a custom tag and verify/usr/local/cuda/lib64/libnvdla_compiler.so*is present in the resulting image.TensorrtExecutionProviderinstead of falling back to CPU.