Skip to content

--threads accepts unsatisfiable values and panics or aborts during channel allocation and thread creation #2078

Description

@leeewee

fd --threads N (-j N) parses N into an unbounded NonZeroUsize (src/cli.rs:556) and subsequently uses it to:

  1. construct a channel with capacity 2 * threads; and
  2. create threads execution workers when non-batched --exec / -x is used.

Excessive but syntactically valid values therefore cause allocation or thread-creation failures to surface as panics or process aborts rather than normal user-facing errors:

# Panic mechanism Site Reached by Example trigger Result
1 Channel-capacity /2 * multiply overflow src/walk.rs:646 bounded(2 * config.threads) every run (with or without --exec) fd -j 9223372036854775807 . capacity overflow, exit 101 (release)
2 Thread-spawn exhaustion src/walk.rs:425 scope.spawn(…) Non-batched --exec / -x fd -j 200000 -x echo . failed to spawn thread → abort, exit 134

Reproduction 1: channel-capacity failure

The results channel is created using twice the requested thread count (src/walk.rs:636):

let (tx, rx) = bounded(2 * config.threads); 

On a 64-bit release build:

$ fd -j 9223372036854775807 .
thread 'main' panicked at alloc/src/raw_vec/mod.rs:28:5:
capacity overflow

$ echo $?
101

And under an -C overflow-checks=on build, the 2 * config.threads itself panics first (attempt to multiply with overflow, walk.rs:636) once N > usize::MAX / 2.

For a smaller but still enormous value, the capacity can be representable while the underlying allocation is not satisfiable:

$ fd -j 99999999999999 .
memory allocation of 3199999999999968 bytes failed

$ echo $?
134

Reproduction 2: execution-worker thread exhaustion

When non-batched --exec is enabled, fd creates execution workers according to the requested thread count (src/walk.rs:425):

for _ in 0..threads {
    let handle = scope.spawn(|| exec::job(/* ... */));
    // ...
}

Scope::spawn unwraps the OS thread-creation result, so once the OS refuses (EAGAIN / RLIMIT_NPROC, typically well under a million threads) it panics instead of degrading gracefully.

$ fd -j 200000 -x echo .
thread 'main' panicked at std/src/thread/scoped.rs:206:46:
failed to spawn thread: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }
$ echo $?
134

This is primarily a robustness and availability issue. An accidental, generated, or otherwise excessive --threads value can cause fd to panic or abort instead of exiting with a controlled error.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions