[CUDA] Add win-arm64 packaging and size options to plugin pipeline

[Tianlei Wu (tianleiwu)]

Description

Extends the CUDA plugin EP packaging pipeline with Windows ARM64 support, replaces the shared x64/ARM64 architecture lists with per-platform lists that reflect the GPUs each package actually targets, and adds two build-time knobs for controlling binary size. The plugin .so/.dll is currently very large (~720 MB uncompressed on the CUDA 12.8 Linux leg, ~90% of which is .nv_fatbin), and the architecture list is the single biggest lever on that number, so it needs to be tuned per platform rather than shared.

Summary of Changes

Windows ARM64 packaging

File	Change
plugin-cuda-pipeline.yml	Add build_windows_arm64; rename invalidAArch64Config to invalidArm64Config and extend it to cover Windows ARM64, since NVIDIA only ships Windows-on-ARM CUDA for 13.x
plugin-cuda-packaging-stage.yml	Add build_windows_arm64 and arm64_cuda_version (13.1), wire the ARM64 stage and its artifacts into NuGet and Foundry Local zip packaging
plugin-win-cuda-stage.yml	Add an arm64 arch path: ARM64 agent pool, win-arm64/ CUDA SDK blob prefix, separate cuDNN folder, native ARM64 toolset

Per-platform CUDA architecture lists

cmake_x64_cuda_archs / cmake_arm64_cuda_archs are split into four independent lists, since Windows x64, Linux x64, Windows ARM64, and Linux aarch64 serve very different GPU populations:

Parameter	CUDA 12.8	CUDA 13.x
cmake_windows_x64_cuda_archs	61,75,86,89,120	75,80,86,89,120
cmake_windows_arm64_cuda_archs	n/a	120,121
cmake_linux_x64_cuda_archs	75,80,86,89,90,120	75,80,86,89,90,120
cmake_linux_aarch64_cuda_archs	n/a	89,90,100,103,120,121

Notable decisions:

• 120-virtual dropped everywhere. The compute_120 PTX measured 176 MB, 27% of the entire .nv_fatbin — by far the most expensive single entry. It also cannot carry the NVFP4 kernels, which are only valid as real sm_120a (cuobjdump -ptx | grep -c e2m1x2 returns 0), so it was paying full price for partial coverage.
• Linux aarch64 targets the platforms that actually exist on ARM: GH200 (90), GB200 (100), GB300 (103), DGX Spark GB10 (121), plus discrete cards in ARM chassis (89, 120). 103 is required alongside 100 because ORT normalizes 100 to 100a-real, and a targets are locked to their exact SM.
• 75 dropped from Linux aarch64 — Turing was never paired with an ARM host in practice.

CUDA architecture normalization

File	Change
cmake/external/cuda_configuration.cmake	ARCHITECTURES_WITH_ACCEL: add 103 and 121, drop 101 (removed by NVIDIA after CUDA 12.9). Without this, 103 and 121 would be built as plain targets and would silently lose the CUTLASS block-scaled/TMA kernels, which are gated on __CUDA_ARCH_FEAT_SM1xx_ALL.

Build size controls

Two new pipeline parameters, both plumbed through the packaging stage to all four platform stages:

Parameter	Default	Effect
enable_cuda_fatbin_size_compression	false	Sets the new onnxruntime_CUDA_FATBIN_COMPRESS_SIZE cmake option, forcing -Xfatbin=-compress-all -compress-mode=size on the CUDA 12.8 leg. CUDA >= 13.0 already does this unconditionally, so the parameter only changes 12.8.
enable_fpa_intb_gemm	true	Sets the existing onnxruntime_USE_FPA_INTB_GEMM cmake option. fpA_intB GEMV/GEMM is ~141 MB of device code (22% of .nv_fatbin), second only to flash attention.

onnxruntime_CUDA_FATBIN_COMPRESS_SIZE fails configuration on CUDA < 12.8 rather than silently passing an unsupported flag to nvcc.

Windows composes these via FatbinCompressOption / FpaIntBGemmOption job variables appended to the build.py invocations, mirroring the existing $(TelemetryOption) pattern. Linux composes them into EXTRA_CMAKE_DEFINES, which build_cuda_plugin_package.sh already forwards.

Packaged binary hardening and verification

File	Change
cmake/onnxruntime_providers_cuda_plugin.cmake	Compile onnxruntime_providers_cuda.rc into the plugin DLL on Windows so the packaged binary carries version info; set SKIP_BUILD_RPATH on Linux so the build machine's CUDA path is not embedded in a binary that ships as-is
plugin-linux-cuda-stage.yml	Fail the build if the plugin .so has an empty RPATH/RUNPATH component or a hard-coded CUDA path
plugin-win-cuda-stage.yml	Fail the build if the plugin DLL is missing required version-info properties

Testing

• Pipeline changes are validated by running the CUDA plugin packaging pipeline. Both new parameters default to current behavior (enable_cuda_fatbin_size_compression: false, enable_fpa_intb_gemm: true), so a default run produces the same build flags as before this PR aside from the architecture list changes.

• The cmake -compress-mode selection logic was verified in isolation across four combinations:

  Toolkit	Option	Result
  12.8	OFF	-Xfatbin=-compress-all
  12.8	ON	-Xfatbin=-compress-all -compress-mode=size
  13.1	OFF	-Xfatbin=-compress-all -compress-mode=size
  12.6	ON	configure-time fatal error, as designed

• The new RPATH and DLL version-info checks are self-verifying: they fail the packaging stage rather than publishing a bad artifact.

Motivation and Context

The primary consumer is Foundry Local (vision, audio, and mostly LLM models), which ships this plugin to end-user machines, so download size matters directly.

Trade-offs worth flagging for reviewers:

• -compress-mode=size raises the minimum driver to the CUDA 12.4 level (Linux >= 550.54.14, Windows >= 551.61); older drivers cannot decompress the fatbin at all. It also increases module load time (measured ~0.8 ms to ~4.3 ms for a ~6.5 MB SASS module) and adds a few percent to nvcc time. This is why the parameter defaults to false and is opt-in per run.
• The architecture lists are all -real with no virtual entry, so any GPU whose compute capability is not explicitly listed gets cudaErrorNoKernelImageForDevice (209) instead of falling back to JIT. This is deliberate given the PTX cost, but it means new architectures must be added explicitly.
• enable_fpa_intb_gemm: false is not yet validated end to end. The fpA_intB path is opt-in at run time via ORT_FPA_INTB_GEMM / ep.cuda.fpa_intb_gemm, but matmul_nbits.cc forces it on whenever weights are prepacked, independent of that flag. The fallback path should be exercised before shipping a package built with this off.

Checklist

• No breaking changes to default pipeline behavior (both new parameters default to existing behavior)
• cmake option gated on toolkit version with an explicit error rather than a silent no-op
• Tests added/updated — not applicable; changes are build/packaging configuration