Memory leaks in loops with Promise

[julien-f]

• Version: Node v4.4.4 / Node v6.1.0
• Platform: Linux 4.2.0-1-amd64 deps: update openssl to 1.0.1j #1 SMP Debian 4.2.6-3 (2015-12-06) x86_64 GNU/Linux

;(function loop () {
  return Promise.resolve().then(loop)
})()

The code above increasingly consumes memory until it crashes with:

<--- Last few GCs --->

   16059 ms: Scavenge 1406.1 (1457.9) -> 1406.1 (1457.9) MB, 19.4 / 0 ms (+ 2.6 ms in 1 steps since last GC) [allocation failure] [incremental marking delaying mark-sweep].
   18001 ms: Mark-sweep 1406.1 (1457.9) -> 1406.1 (1457.9) MB, 1941.5 / 0 ms (+ 3.7 ms in 2 steps since start of marking, biggest step 2.6 ms) [last resort gc].
   19928 ms: Mark-sweep 1406.1 (1457.9) -> 1406.1 (1457.9) MB, 1927.5 / 0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x30f9e32b4629 <JS Object>
    1: PromiseSet(aka PromiseSet) [native promise.js:~38] [pc=0x21bc625235b] (this=0x30f9e32041b9 <undefined>,n=0x23be73509901 <a Promise with map 0x2d3fc3316dc9>,q=0,r=0x30f9e32041b9 <undefined>,t=0x23be73509961 <JS Array[0]>,u=0x23be73509941 <JS Array[0]>)
    2: PromiseInit(aka PromiseInit) [native promise.js:~53] [pc=0x21bc624d6e9] (this=0x30f9e32041b9 <undefined>,n=0x23be73509901 <a Promis...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory

With Bluebird, the used memory never goes above 30MB and the program does not crash:

global.Promise = require('bluebird')

;(function loop () {
  return Promise.resolve().then(loop)
})()

[mscdex]

/cc @nodejs/v8

[Fishrock123]

Is this because each promise keeps a reference to the next?

[ChALkeR]

@Fishrock123 Doesn't look like that, looks like mark-sweep isn't called for whatever reason.

Could be a duplicate of #6180.
Perhaps it's even fixed by #6398 in the current master, but I can't check atm. Update: it's not, #6180 isn't fixed yet.

[jeisinger]

filed https://bugs.chromium.org/p/v8/issues/detail?id=5002

Firefox also runs out of memory, so maybe it's a spec thing

according to the crash, MS is executed.

[ChALkeR]

@jeisinger It's not a spec thing, adding a manual gc every e.g. 1e7 iterations fixes it.
Looks very similar to #6180 — scavege is triggered instead of mark-sweep, but the memory is reclaimed only by mark-sweep, not by scavenge.

Update: I made a mistake in the test, shame on me =).

[julien-f]

@jeisinger If it's a spec thing it's an issue because I don't know a better way to do a loop with promises :/

@petkaantonov What's your opinion on this?

[ChALkeR]

@jeisinger No idea why the mark-sweep at the end doesn't reclaim most of that, though.

[ChALkeR]

@Fishrock123 @jeisinger I made a mistake in the test, ignore my previous comments. Sorry for the inconvenience.

[Fishrock123]

If it's a spec thing it's an issue because I don't know a better way to do a loop with promises :/

Unbounded recursive loops are typically not safe in JavaScript at the current time, promise or not apparently.

[ChALkeR]

Yes, it looks like all Promise objects are retained in the heap. There are twice more of them than the number of iterations, btw.

[vkurchatkin]

twice more of them than the number of iterations, btw.

makes sense, 2 new promises are created on each iteration

[julien-f]

Unbounded recursive loops are typically not safe in JavaScript at the current time, promise or not apparently.

Yes but there are no other way to do async loops than using recursion (without tricks likes setImmediate() to break the stack).

[ChALkeR]

makes sense, 2 new promises are created on each iteration

Ah, yes, that is correct.