Change default ITK threader from TBB to Pool

[N-Dekker]

The discourse.itk.org post ~8x slower registration with itk-elastix Python API vs elastix CLI — minimal reproducible example by @NicoKiaru and further research have shown that for common use cases of itk-elastix, the use of ITK's TBBMultiThreader causes a major performance loss, in comparison to using the elastix executable (which internally uses ITK's PoolMultiThreader). Although PR #388 commit ba7c8d0 (just merged) has slightly improved the performance, itk-elastix is still much slower than the elastix executable, for common use cases.

This issue suggests to make ITK's PoolMultiThreader the default threader of itk-elastix.

[dzenanz]

I think we should keep presence of TBB multi-threader. In a compiled ITK, we could only set ITK_DEFAULT_THREADER to Pool. But taking python packaging into account, it is more complicated than that.

[N-Dekker]

I think we should keep presence of TBB multi-threader.

Can you please elaborate why you think so?

[blowekamp]

For the condo forge build TBB is enabled but the default threader is the pool:
https://github.com/conda-forge/libitk-feedstock/blob/main/recipe/build.sh#L55

IT is my understanding the major issue with TBB threader is when the images are small, there is an excessive number or work units. Changing this multiplier to 1 ( maybe? ) or lower value may be beneficial, and the expense of some performance on larger images?
https://github.com/InsightSoftwareConsortium/ITK/blob/main/Modules/Core/Common/src/itkTBBMultiThreader.cxx#L52

Ideally each filter or algorithm could have some type of constant thread/work unit overhead cost factor to be used to estimate a reasonable number of work units given the size of the image. ( compute_time = image_size / n_threads + n_thread_overhead)

[dzenanz]

I can imagine situations in which TBB's load balancing would be useful. Thinking more about it, changing the default threader just in ITKElastix (and not entire ITK) seems non-trivial. Also, I wondered how much use does dynamic multi-threading gets in ITKElastix. Below is my conversation with Gemini. It "thinks" that TBB is important for ITKElastix.

---

Q: Dynamic multi-threading can change number of work units at run time, and is only done by TBB threader. Classic multi-threading divides the work into a fixed number of work units, which can then be scheduled to thread by any multi-threading back-end. My question is: how prevalent is dynamic vs classic multi-threading in ITK's registration components?

The distinction between classic multi-threading (fixed number of work units predefined by the filter) and dynamic multi-threading (flexible work units split at runtime to optimize load balancing) highlights an evolutionary shift in ITK. [1]

In ITK's modern registration components—specifically ITKv4 and ITKv5+—the prevalence of dynamic versus classic multi-threading is governed by how individual modules inherit from two distinct design architectures: itk::DomainThreader and the underlying ThreadedGenerateData methods. [1, 2]

Dynamic multi-threading is highly dominant across the v4 registration framework, though classic multi-threading remains active in legacy systems and highly structured, array-dependent calculations. [1, 2, 3]

---

1. Image Similarity Metrics: Predominantly Dynamic

The core of the registration framework—ImageToImageMetricv4—completely bypassed the legacy ImageSource pipeline. []

• The Architecture: Instead of using classic fixed-split execution, it leverages itk::DomainThreader. [, 2]
• How it works: These threaders use partitioners (like ThreadedImageRegionPartitioner or ThreadedIndexedContainerPartitioner). When paired with the Intel TBB multi-threading backend, these partitioners feed into dynamic region splitting. [1, 2]
• Prevalence: Nearly all v4 metrics (e.g., Mattes Mutual Information, Mean Squares, Correlation) use this model. Work units are adaptively sized and balanced at run-time if TBB is enabled, making dynamic multi-threading highly prevalent here. [1, 2, 4]

2. Dense & Deformable Transforms: Split 50/50

Dense registration methods (such as B-Spline registration, Time-Varying Velocity Fields, and Symmetric Normalization / SyN) alternate between two types of tasks: [1, 2]

• Dynamic Multi-Threading: Used during field integration and metric evaluation over the spatial domain, which relies on the DomainThreader infrastructure. []
• Classic Multi-Threading: Many field smoothing steps rely on Gaussian or discrete convolution filters that implement ThreadedGenerateData. While ITK 5.0 introduced DynamicThreadedGenerateData, filters that allocate strict, thread-local intermediate storage arrays based on predefined threadId counts still explicitly force Classic multi-threading to prevent race conditions or segmentation faults. [1, 2, 3]

3. Interpolators & Resampling: Increasingly Dynamic

The final step of applying a transformation to an image relies on interpolators and resampling filters. [1]

• Image To Image Resampling: Filters like itk::ResampleImageFilter have been refactored in ITKv5 to utilize DynamicThreadedGenerateData. If your backend is TBB, this step dynamically manages work units according to processor load. [1, 2]
• Custom/Legacy Interpolators: Older, specialized interpolators that have not migrated to the v5 signature default back to classic thread-splitting. [1]

Q: I am most interested in how important is TBB threader to Elastix, and more specifically, ITKElastix.

To understand how important the Intel Threading Building Blocks (TBB) threader is to Elastix and the ITKElastix Python/C++ interface, it helps to look at how Elastix’s internal engine handles multi-threading compared to standard ITK.

The TBB threader is architecturally separate from Elastix's primary optimization engine, but highly critical to ITKElastix's preprocessing and resampling pipelines. Because Elastix relies on its own internal thread management for core registration math, missing or misconfigured TBB threaders in ITKElastix frequently triggers massive performance degradation.

The following breakdown explains how TBB interacts with both ecosystems:

---

1. The Core Elastix Engine Uses a Custom Multi-Threader

Historically, Elastix was built as a standalone command-line interface (CLI). To maximize performance, the creators of Elastix implemented their own optimized threading primitives directly inside their core registration classes (such as itk::ElastixRegistrationMethod). [1, 2]

• The Reality: For the primary registration loops—like calculating the BSpline Jacobian or evaluating the Advanced Mean Squares metric—Elastix bypasses standard ITK multi-threaders (including TBB). It manages its own fixed thread-pool natively based on the user-supplied parameter file ((NumberOfThreads 8)). [1]
• The Dynamic Factor: Because Elastix relies heavily on exact random sampling (Stochastic Gradient Descent / Robbins-Monro optimizers) to calculate image metrics, it intentionally restricts these core calculations to classic, deterministic multi-threading (fixed work-unit structures). Dynamic runtime work-stealing from TBB would break the mathematical reproducibility of its stochastic sampling.

---

2. Why TBB is Critical to ITKElastix (Python / C++ API Bindings)

Even though the core optimizer loop uses Elastix's internal threader, ITKElastix wraps the software inside a modern ITKv5 execution pipeline. In this hybrid ecosystem, TBB becomes essential. [1]

A. Multi-Threaded Preprocessing and Pyramid Creation

Before Elastix can run an optimization, the input images must pass through an extensive ITK pipeline:

• Gaussian blurring (itk::DiscreteGaussianImageFilter)
• Image downsampling (itk::ResampleImageFilter)
• Multi-resolution image pyramid generation (itk::MultiResolutionImagePyramidImageFilter)

In modern ITK, these filters default to DynamicThreadedGenerateData. If TBB is not compiled or enabled in the underlying ITK environment, these critical preprocessing steps fall back to sequential or poorly-optimized pool-allocations, creating severe memory and CPU bottlenecks before the registration even begins.

B. The "Python Global Thread Pool" & Overhead Bottleneck

A known and common performance issue in the itk-elastix Python library occurs when ITK is called from a managed wrapper (like a Python process or Java plugin). [1, 2]

• Without TBB active as the global default threader backend, the wrapper's interface layer can cause massive cache thrashing and thread-creation overhead between ITK and Elastix. [1, 2]
• When the Intel TBB module is properly linked, it introduces a Scalable Memory Allocator (tbbmalloc). This stops memory allocation locks at the Python/C++ boundary, allowing elastix to request image regions across threads smoothly. [1, 2]

C. Final Resampling and Transformation Output

Once Elastix finds the optimal transform, the moving image must be mapped to the fixed image space. In ITKElastix, this utilizes the standard ITK resampling framework. As established, itk::ResampleImageFilter relies heavily on dynamic multi-threading. If TBB is active, the final coordinate warping is rapidly load-balanced across all available CPU cores.

---

Summary: Can you run ITKElastix without TBB?

Yes, but you shouldn't. [1]

If you compile ITKElastix without TBB (ITK_WITH_TBB=OFF), the standalone Elastix optimizer loop will still execute using its native, classic multi-threading pool. However, you will incur a significant performance penalty (often resulting in registration pipelines running multiple times slower) during the image pyramid building, interpolation, and final memory reallocation phases because the surrounding ITK framework loses its dynamic balancing engine. [1, 2]

[blowekamp]

I don't think that AI response adds to the conversation. Consider hiding that section of your response.

I can imagine situations in which TBB's load balancing would be useful.

Yes there are. Binary mask dependent algorithms are an prime example of non-uniform work.

But consider my formula where each work unit creates a histogram that need to be merged for each work unit. The constant cost of another work unit is heigh.

[dzenanz]

We can, and perhaps should, re-tune the multi-threaders. That is orthogonal to the question whether to remove TBB back-end from ITKElastix.

[N-Dekker]

Thanks for your comments so far. I think that at least the default threader of itk-elastix and the elastix executable should be the same. I think by default, itk-elastix and the elastix executable should perform similarly fast.

[thewtex]

I agree with @blowekamp 's suggestion -- added here: InsightSoftwareConsortium/ITKPythonPackage#305

[N-Dekker]

• Now that PERF: Set the default threader to Pool ITKPythonPackage#305 is merged (to https://github.com/InsightSoftwareConsortium/ITKPythonPackage/tree/release-5.4 ), is there anything more to be done to resolve this issue for ITKElastix? Or does the next CI build of ITKElastix automatically pick the latest "ITKPythonPackage/tree/release-5.4" revision?

[dzenanz]

A new ITK python package might be needed too. I am not sure.

[thewtex]

Once we build against ITK 5.4.7, the default threader will be Pool.