Memory-limited NestedLoopJoin silently drops unmatched rows when it spills

[viirya]

Describe the bug

A memory-limited NestedLoopJoinExec silently drops unmatched rows when it spills. The query succeeds and returns fewer rows than the same query with an ample memory pool — there is no error.

This is pre-existing and not a regression from #24675: that PR only touches nested_loop_join.rs's spill gate, and its gate does not cover this path.

To Reproduce

Two tables registered from in-memory batches (l: 200 rows, r: 90 rows, join keys drawn from a small domain so most rows match), target_partitions = 1, batch_size = 16, comparing an unlimited pool against a 64-byte pool:

-- LEFT ANTI, via NOT EXISTS
SELECT l.k, l.v FROM l WHERE NOT EXISTS (SELECT 1 FROM r WHERE l.v > r.w);
-- ample: 2 rows, memory-limited: 0 rows

SELECT l.k, r.w FROM l LEFT JOIN r ON l.v > r.w;
-- ample: 9336 rows, memory-limited: 9334 rows

SELECT l.k, r.w FROM l FULL JOIN r ON l.v > r.w;
-- ample: 9339 rows, memory-limited: 9337 rows

The lost rows are the unmatched ones (e.g. 6|NULL, 8|NULL for the LEFT JOIN). INNER and explicit RIGHT JOIN are unaffected. The join spills in the failing runs (spill_count > 0).

Note the failing shapes reach the operator as swapped join types (Right, RightAnti, RightSemi, or Full), so the rows needing unmatched emission sit on the operator's probe side.

Root cause

In memory-limited mode the left side is processed in chunks and the right side is replayed per chunk. Per-right-batch match bitmaps are therefore merged into SpillStateActive::global_right_bitmaps and their emission is deliberately deferred to NLJState::EmitGlobalRightUnmatched, which is only reachable from handle_emit_left_unmatched's left_exhausted == true branch.

But handle_buffering_left contains this early exit:

if active.pending_batches.is_empty() {
    // No data at all — go directly to Done
    self.left_exhausted = true;
    self.state = NLJState::Done;
    return ControlFlow::Continue(());
}

Despite the comment, this also fires on the load after the final left chunk, when the left side is exhausted and nothing remains to buffer. Going straight to Done discards the accumulated bitmaps, so any probe row that no chunk matched is never emitted.

Instrumenting the failing LEFT JOIN case shows 39 bitmap merges but zero emissions, and the completion path only ever running with left_exhausted = false.

Expected behavior

Memory-limited execution returns the same rows as unlimited execution — or fails with ResourcesExhausted. It must not return fewer rows silently.