Skip to content

Commit 943d026

Browse files
panjf2000vibe-sudoCopilot
authored
patch: v2.12.1 (#398)
* chore: use my referral link of DO * docs: correct WithNonblocking comment about its returned error (#390) * fix: make Reboot() wait for pool to fully release before resetting state (#395) * fix: resolve data race on poolCommon.once field using atomic.Pointer The data race occurred between Reboot() writing p.once = &sync.Once{} and worker goroutines reading p.once.Do(...) concurrently. Fix: change the once field from *sync.Once to atomic.Pointer[sync.Once] so all reads (Load) and writes (Store) are atomic. Agent-Logs-Url: https://github.com/panjf2000/ants/sessions/b2f67c6c-ed78-4758-9185-643b3e09d143 Co-authored-by: panjf2000 <7496278+panjf2000@users.noreply.github.com> * fix: make Reboot() wait for all workers to exit before resetting pool The root cause of the data race is that Reboot() resets pool state (once, allDone) while workers are still shutting down and accessing those same fields. Fix: Reboot() now waits on the allDone channel (which is closed when the last worker exits) before resetting pool state. This ensures no worker goroutine is running when the fields are reassigned, eliminating the data race without needing atomic wrappers. Reverts the atomic.Pointer[sync.Once] approach in favor of this fundamental fix. Agent-Logs-Url: https://github.com/panjf2000/ants/sessions/7b1d6e25-546a-40ef-b31d-33e824186b31 Co-authored-by: panjf2000 <7496278+panjf2000@users.noreply.github.com> * fix: also wait for purge/ticktock goroutines to exit in Reboot() Ensures the old purge and ticktock goroutines have fully exited (set their done flags) before Reboot() resets those flags and starts new goroutines. This prevents a logic race where the old goroutine could set purgeDone/ticktockDone=1 after Reboot() resets them to 0. Agent-Logs-Url: https://github.com/panjf2000/ants/sessions/480234c8-a50f-4dc3-9352-0b337ebfc5d3 Co-authored-by: panjf2000 <7496278+panjf2000@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: panjf2000 <7496278+panjf2000@users.noreply.github.com> * chore: improve variable naming in binarySearch (#393) * fix: release created pools on NewMultiPool failure (#394) * chore: replace 1<<31-1 with math.MaxInt32 (#392) * fix: move once.Do(close(allDone)) from Reboot() to Release() (#397) * fix: use atomic.Pointer for once field to prevent data race in Reboot The data race occurred between Reboot() writing p.once = &sync.Once{} and worker goroutines reading p.once to call once.Do(close(allDone)). By using atomic.Pointer[sync.Once], all reads (Load) and writes (Store) to the once field are properly synchronized. Agent-Logs-Url: https://github.com/panjf2000/ants/sessions/4a1d1b13-0a7f-4167-ae12-429e7719ea32 Co-authored-by: panjf2000 <7496278+panjf2000@users.noreply.github.com> * fix: use sync.WaitGroup to ensure all worker goroutines fully exit before Reboot Revert the superficial atomic.Pointer fix. The real issue is that Reboot() proceeded after allDone (running count hit 0) while worker goroutines were still executing deferred cleanup code. Add workerWg sync.WaitGroup to poolCommon. Each worker goroutine calls wg.Add(1) before spawning and defer wg.Done() as the outermost defer (runs last). Reboot() calls workerWg.Wait() after <-allDone to block until every goroutine has fully exited before resetting pool state. Agent-Logs-Url: https://github.com/panjf2000/ants/sessions/758f404a-fab6-4a4e-8f42-ea4fdfd2ab9a Co-authored-by: panjf2000 <7496278+panjf2000@users.noreply.github.com> * fix: call workerWg.Add(1) before addRunning(1) for defensive ordering Agent-Logs-Url: https://github.com/panjf2000/ants/sessions/758f404a-fab6-4a4e-8f42-ea4fdfd2ab9a Co-authored-by: panjf2000 <7496278+panjf2000@users.noreply.github.com> * fix: move once.Do(close(allDone)) to Release() instead of Reboot() This is the fundamental fix: Release() is the proper place to close allDone when no workers are running, since it's the operation that transitions the pool to CLOSED state. If workers are still running, the last one to exit closes allDone in its defer as before. This eliminates the race between Reboot() and running workers on once.Do, and removes the unnecessary workerWg complexity. Agent-Logs-Url: https://github.com/panjf2000/ants/sessions/0d78b737-d4ef-4dbd-b021-3e532757c1f9 Co-authored-by: panjf2000 <7496278+panjf2000@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: panjf2000 <7496278+panjf2000@users.noreply.github.com> --------- Co-authored-by: hehaifeng526 <54569318+vibe-sudo@users.noreply.github.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: panjf2000 <7496278+panjf2000@users.noreply.github.com>
2 parents 24eac6c + deeaf50 commit 943d026

8 files changed

Lines changed: 62 additions & 34 deletions

File tree

README.md

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -347,8 +347,4 @@ If you have `ants` integrated into projects, feel free to open a pull request re
347347

348348
## 🔋 Sponsorship
349349

350-
<p>
351-
<a href="https://www.digitalocean.com/">
352-
<img src="https://opensource.nyc3.cdn.digitaloceanspaces.com/attribution/assets/PoweredByDO/DO_Powered_by_Badge_blue.svg" width="201px">
353-
</a>
354-
</p>
350+
[![DigitalOcean Referral Badge](https://web-platforms.sfo2.cdn.digitaloceanspaces.com/WWW/Badge%203.svg)](https://www.digitalocean.com/?refcode=5d8774f42124&utm_campaign=Referral_Invite&utm_medium=Referral_Program&utm_source=badge)

README_ZH.md

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -346,8 +346,4 @@ pool.Reboot()
346346

347347
## 🔋 赞助商
348348

349-
<p>
350-
<a href="https://www.digitalocean.com/">
351-
<img src="https://opensource.nyc3.cdn.digitaloceanspaces.com/attribution/assets/PoweredByDO/DO_Powered_by_Badge_blue.svg" width="201px">
352-
</a>
353-
</p>
349+
[![DigitalOcean Referral Badge](https://web-platforms.sfo2.cdn.digitaloceanspaces.com/WWW/Badge%203.svg)](https://www.digitalocean.com/?refcode=5d8774f42124&utm_campaign=Referral_Invite&utm_medium=Referral_Program&utm_source=badge)

ants.go

Lines changed: 38 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -402,6 +402,15 @@ func (p *poolCommon) Release() {
402402
// There might be some callers waiting in retrieveWorker(), so we need to wake them up to prevent
403403
// those callers blocking infinitely.
404404
p.cond.Broadcast()
405+
406+
// If there are no running workers at the time of Release, close allDone immediately
407+
// so that Reboot() or ReleaseContext() won't block on <-p.allDone indefinitely.
408+
// If workers are still running, the last one to exit will close allDone in its defer.
409+
if p.Running() == 0 {
410+
p.once.Do(func() {
411+
close(p.allDone)
412+
})
413+
}
405414
}
406415

407416
// ReleaseTimeout is like Release but with a timeout, it waits all workers to exit before timing out.
@@ -438,12 +447,6 @@ func (p *poolCommon) ReleaseContext(ctx context.Context) error {
438447
purgeCh = p.allDone
439448
}
440449

441-
if p.Running() == 0 {
442-
p.once.Do(func() {
443-
close(p.allDone)
444-
})
445-
}
446-
447450
for {
448451
select {
449452
case <-ctx.Done():
@@ -465,14 +468,36 @@ func (p *poolCommon) ReleaseContext(ctx context.Context) error {
465468
// Release() to ensure that all workers are stopped and resource are released
466469
// before rebooting, otherwise you may run into data race.
467470
func (p *poolCommon) Reboot() {
468-
if atomic.CompareAndSwapInt32(&p.state, CLOSED, OPENED) {
469-
atomic.StoreInt32(&p.purgeDone, 0)
470-
p.goPurge()
471-
atomic.StoreInt32(&p.ticktockDone, 0)
472-
p.goTicktock()
473-
p.allDone = make(chan struct{})
474-
p.once = &sync.Once{}
471+
if atomic.LoadInt32(&p.state) != CLOSED {
472+
return
475473
}
474+
475+
// Wait for all workers to exit. The allDone channel is closed either
476+
// by Release() (if no workers were running) or by the last exiting worker.
477+
<-p.allDone
478+
479+
// Wait for the purge and ticktock goroutines to exit completely,
480+
// so that their deferred purgeDone/ticktockDone stores don't
481+
// race with the resets below.
482+
if !p.options.DisablePurge {
483+
for atomic.LoadInt32(&p.purgeDone) != 1 {
484+
runtime.Gosched()
485+
}
486+
}
487+
for atomic.LoadInt32(&p.ticktockDone) != 1 {
488+
runtime.Gosched()
489+
}
490+
491+
if !atomic.CompareAndSwapInt32(&p.state, CLOSED, OPENED) {
492+
return
493+
}
494+
495+
atomic.StoreInt32(&p.purgeDone, 0)
496+
p.goPurge()
497+
atomic.StoreInt32(&p.ticktockDone, 0)
498+
p.goTicktock()
499+
p.allDone = make(chan struct{})
500+
p.once = &sync.Once{}
476501
}
477502

478503
func (p *poolCommon) addRunning(delta int) int {

multipool.go

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,10 @@ func NewMultiPool(size, sizePerPool int, lbs LoadBalancingStrategy, options ...O
108108
for i := 0; i < size; i++ {
109109
pool, err := NewPool(sizePerPool, options...)
110110
if err != nil {
111+
// Release all previously created pools to avoid resource leak
112+
for j := 0; j < i; j++ {
113+
pools[j].Release()
114+
}
111115
return nil, err
112116
}
113117
pools[i] = pool
@@ -120,7 +124,7 @@ func (mp *MultiPool) next(lbs LoadBalancingStrategy) (idx int) {
120124
case RoundRobin:
121125
return int(atomic.AddUint32(&mp.index, 1) % uint32(len(mp.pools)))
122126
case LeastTasks:
123-
leastTasks := 1<<31 - 1
127+
leastTasks := math.MaxInt32
124128
for i, pool := range mp.pools {
125129
if n := pool.Running(); n < leastTasks {
126130
leastTasks = n

multipool_func.go

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,10 @@ func NewMultiPoolWithFunc(size, sizePerPool int, fn func(any), lbs LoadBalancing
5555
for i := 0; i < size; i++ {
5656
pool, err := NewPoolWithFunc(sizePerPool, fn, options...)
5757
if err != nil {
58+
// Release all previously created pools to avoid resource leak
59+
for j := 0; j < i; j++ {
60+
pools[j].Release()
61+
}
5862
return nil, err
5963
}
6064
pools[i] = pool
@@ -67,7 +71,7 @@ func (mp *MultiPoolWithFunc) next(lbs LoadBalancingStrategy) (idx int) {
6771
case RoundRobin:
6872
return int(atomic.AddUint32(&mp.index, 1) % uint32(len(mp.pools)))
6973
case LeastTasks:
70-
leastTasks := 1<<31 - 1
74+
leastTasks := math.MaxInt32
7175
for i, pool := range mp.pools {
7276
if n := pool.Running(); n < leastTasks {
7377
leastTasks = n

multipool_func_generic.go

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,10 @@ func NewMultiPoolWithFuncGeneric[T any](size, sizePerPool int, fn func(T), lbs L
5151
for i := 0; i < size; i++ {
5252
pool, err := NewPoolWithFuncGeneric(sizePerPool, fn, options...)
5353
if err != nil {
54+
// Release all previously created pools to avoid resource leak
55+
for j := 0; j < i; j++ {
56+
pools[j].Release()
57+
}
5458
return nil, err
5559
}
5660
pools[i] = pool
@@ -63,7 +67,7 @@ func (mp *MultiPoolWithFuncGeneric[T]) next(lbs LoadBalancingStrategy) (idx int)
6367
case RoundRobin:
6468
return int(atomic.AddUint32(&mp.index, 1) % uint32(len(mp.pools)))
6569
case LeastTasks:
66-
leastTasks := 1<<31 - 1
70+
leastTasks := math.MaxInt32
6771
for i, pool := range mp.pools {
6872
if n := pool.Running(); n < leastTasks {
6973
leastTasks = n

options.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@ func WithMaxBlockingTasks(maxBlockingTasks int) Option {
9696
}
9797
}
9898

99-
// WithNonblocking indicates that pool will return nil when there is no available workers.
99+
// WithNonblocking indicates that pool will return ErrPoolOverload when there is no available workers.
100100
func WithNonblocking(nonblocking bool) Option {
101101
return func(opts *Options) {
102102
opts.Nonblocking = nonblocking

worker_loop_queue.go

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -124,8 +124,7 @@ func (wq *loopQueue) refresh(duration time.Duration) []worker {
124124
}
125125

126126
func (wq *loopQueue) binarySearch(expiryTime int64) int {
127-
var mid, nlen, basel, tmid int
128-
nlen = len(wq.items)
127+
n := len(wq.items)
129128

130129
// if no need to remove work, return -1
131130
if wq.isEmpty() || expiryTime < wq.items[wq.head].lastUsedTime() {
@@ -143,21 +142,21 @@ func (wq *loopQueue) binarySearch(expiryTime int64) int {
143142

144143
// base algorithm is a copy from worker_stack
145144
// map head and tail to effective left and right
146-
r := (wq.tail - 1 - wq.head + nlen) % nlen
147-
basel = wq.head
145+
r := (wq.tail - 1 - wq.head + n) % n
146+
base := wq.head
148147
l := 0
149148
for l <= r {
150-
mid = l + ((r - l) >> 1) // avoid overflow when computing mid
149+
mid := l + ((r - l) >> 1) // avoid overflow when computing mid
151150
// calculate true mid position from mapped mid position
152-
tmid = (mid + basel + nlen) % nlen
151+
tmid := (mid + base + n) % n
153152
if expiryTime < wq.items[tmid].lastUsedTime() {
154153
r = mid - 1
155154
} else {
156155
l = mid + 1
157156
}
158157
}
159158
// return true position from mapped position
160-
return (r + basel + nlen) % nlen
159+
return (r + base + n) % n
161160
}
162161

163162
func (wq *loopQueue) reset() {

0 commit comments

Comments
 (0)