net: close DMA-BUF fd on the regMrDmaBuf-failure fallback path

[EylonKrause]

Summary

sendProxyRegBuffer and recvProxyRegBuffer (src/transport/net.cc) leak the DMA-BUF file descriptor on the regMrDmaBuf-failure fallback path.

Both functions open a DMA-BUF fd via cuMemGetHandleForAddressRange and register it with regMrDmaBuf:

if (resources->useDmaBuf) {
  int dmabuf_fd;                                 // declared inside the block
  ...
  CUCHECKGOTO(cuMemGetHandleForAddressRange(&dmabuf_fd, ...), ret, peermem);
  NCCLCHECKGOTO(regMrDmaBuf(..., dmabuf_fd, &handle), ret, peermem);
  (void)close(dmabuf_fd);                        // closed ONLY on this success path
  needReg = false;
}
peermem:                                         // no close here
  if (needReg) { ... regMr(...) fallback ... }   // can succeed -> function returns OK

When regMrDmaBuf fails, NCCLCHECKGOTO jumps to peermem with dmabuf_fd still open and needReg still true, so the code falls back to a plain regMr. If that fallback succeeds, the function returns success while the DMA-BUF fd is never closed — a per-registration fd leak on the fallback path. Under repeated buffer registration this exhausts the process fd table.

Fix

Hoist dmabuf_fd to the enclosing scope, initialize it to -1, drop the inline close, and close once at the peermem label on every exit path — guarded by != -1 so the cuMemGetHandleForAddressRange-failure path (where no fd was opened) does not close a bogus descriptor.

This mirrors the idiom already used in collNetRegBuffer (src/transport/coll_net.cc), which declares int dmabuf_fd = -1; at function scope and performs a guarded close at its cleanup label. The change makes the two proxy-side reg paths consistent with the collnet path.

Testing

• -fsyntax-only compile of src/transport/net.cc is clean (validates the hoisted-scope + peermem fall-through close: no double-close, no jump-crosses-initialization).
• Behavior mirrors the already-accepted coll_net.cc guarded-close pattern.
• Full functional exercise requires a multi-GPU DMA-BUF-capable path (nv_peermem fallback), which I cannot run on a single-GPU host; flagging for maintainer CI on the DMA-BUF path.

Signed-off-by: EylonKrause eylon1909@gmail.com

[EylonKrause]

Disclosure: this contribution was authored with an AI coding assistant (Claude) and reviewed before submission.