FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory

Question

Node version is v0.11.13

Memory usage during crash according to sudo top not raises over 3%

Code that reproduces this error:

var request = require('request')
var nodedump = require('nodedump')

request.get("http://pubapi.cryptsy.com/api.php?method=marketdatav2",function(err,res)
{
    var data
    console.log( "Data received." );
    data = JSON.parse(res.body)
    console.log( "Data parsed."   );
    data = nodedump.dump(data)
    console.log( "Data dumped."   ); 
    console.log( data )
})

To check if that a recursion stack size problem I have ran next code with --stack-size=60000 parameter

var depth = 0;

(function recurse() {
    // log at every 500 calls
    (++depth % 500) || console.log(depth);
    recurse();
})();

and have got

264500 
Segmentation fault

Then I ran code which gives me FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory with the same --stack-size=60000 parameter and haven't got Segmentation fault.

So I conclude CALL_AND_RETRY_LAST has nothing common with the recursion stack size.

How could I solve this problem? I believe there is enough free memory on my computer to finish this task successfully.

There are similar questions on stackoverflow but none of this questions are about CALL_AND_RETRY_LAST that's why I created separate question.

Your code works fine for me with Node.js 0.10.28. It took 1200 MB of virtual memory and printed an unbelievable amount of junk to the screen, but it seemed to "work." — John Zwinck, Sep 29 '14 at 07:26
The junk is exactly what I desired to get. Bad thing is that node 0.10.* does not support harmony generators thus I can't use it for this purposes. — user619271, Sep 29 '14 at 08:16

score 127 · Accepted Answer · edited Jun 12 '19 at 07:36

127

If you have a look at the source: github/v8, it seems that you try to reserve a very big object.According to my experience it happens if you try to parse a huge JSON object, but when I try to parse your output with JSON and node0.11.13, it just works fine.

You don't need more --stack-size, you need more memory: --max_new_space_size and/or --max_old_space_size.

The only hint I can give you beside that is trying another JSON-parser and/or try to change the input format to JSON line instead of JSON only.

edited Jun 12 '19 at 07:36

sajadre

1,141
2
15
30

answered Sep 29 '14 at 07:40

CFrei

3,552
1
15
29

12

Thank you very much! Defining --max_old_space_size=2000000 which I believe means ~2GB had solved my issue. Varying --max_new_space_size don't seemed to have any effect at all. – user619271 Sep 29 '14 at 08:20
I also took a look on the v8 source but didn't understood what's going on there at all. So my chances to solve this without your help are probably tend to zero. So thank you again. – user619271 Sep 29 '14 at 08:29
1

@theWanderer4865 what do you mean with "try to change the input format to JSON line instead of JSON only."? Could you please elaborate? Thank you. – titusmagnus Aug 07 '15 at 21:38
@titusmagnus I didn't answer the question maybe OP can explain more. My guess is he may be referring to new line delimited JSON but I'm not 100% – theWanderer4865 Aug 08 '15 at 13:01
16

@user619271 --max_old_space_size=2048 is 2gb – Timeless Feb 01 '16 at 01:45
Is there a way to know where exactly in my code this is happening? I think increasing memory is not the only solution, perhaps restructuring the algorithm can help solve the issue. Thanks – Diego Nov 13 '17 at 15:24
Thanks for the hint, for me it was reading huge documents from MongoDB which are actually JSON files. – samurai jack Jan 26 '18 at 01:49
1

Haha.. @Timeless – technophyle Sep 25 '18 at 03:36
while executing "nodejs --max_old_space_size=2000 update-farsi-json-file.js" getting this error "FATAL ERROR: invalid array length Allocation failed - JavaScript heap out of memory" where update-farsi-json-file.js is a JS script that uses "csv-to-array" module to read CSV into an array of rows. link to issue: https://stackoverflow.com/questions/60740102/npm-module-csv-to-array-does-not-read-entire-csv-file-given-as-input @CFrei – Nishant Kumar Mar 19 '20 at 10:26
@titusmagnus , I believe he or she is referring to the JSON Lines format, which is a version of JSON that is formatted to allow "streaming", i.e. you can process it a little bit at a time so as to lower the amount of memory you're using at any given moment. There's more about that here: https://jsonlines.org/ – Todd Walton Jan 14 '22 at 17:42

sol404 · Answer 2 · 2019-07-24T08:47:04.613

61

$ sudo npm i -g increase-memory-limit

Run from the root location of your project:

$ increase-memory-limit

This tool will append --max-old-space-size=4096 in all node calls inside your node_modules/.bin/* files.

Node.js version >= 8 - DEPRECATION NOTICE

Since NodeJs V8.0.0, it is possible to use the option --max-old-space-size. NODE_OPTIONS=options...

$ export NODE_OPTIONS=--max_old_space_size=4096

edited Jul 24 '19 at 08:47

answered Sep 23 '17 at 11:58

sol404

1,653
13
15

hello, @sol-ibit. What does 4096 argument means? – estebanpdl Sep 28 '18 at 00:06
1

Hi @estebanpdl, The solution to run your Node.js app with increased memory is to start the process with an additional V8 flag: --max-old-space-size. You need to append your desired memory size in megabytes. The following command will start your application with a memory limit of 4 GB. – sol404 Sep 28 '18 at 10:40
2

Setting the variable with `export NODE_OPTIONS=--max_old_space_size=4096` did the trick for me! Thanks! – Tarator Apr 29 '19 at 19:05

score 47 · Answer 3 · edited Jun 01 '16 at 20:01

47

To solve this issue you need to run your application by increasing the memory limit by using the option --max_old_space_size. By default the memory limit of Node.js is 512 mb.

node --max_old_space_size=2000  server.js

edited Jun 01 '16 at 20:01

jfunk

7,176
4
37
38

answered Feb 26 '16 at 11:22

Jijo Paulose

1,896
18
20

6

Is there a step-by-step guide of how to do this please? Where do I paste this faithful code: `node --max_old_space_size=2000 server.js` – Mr. Benedict Aug 08 '17 at 20:06
@Mr.Benedict you do this when you start your application. – easymoden00b Aug 11 '17 at 13:22
2

node --max_old_space_size=2048 server.js ... is more appropriate ... works thanks – danday74 Jan 25 '18 at 16:25
1

I have a React app on google cloud. How can I do this? Can anyone guide, please? – Shubham Kushwah Jan 02 '19 at 07:54
1

@easymoden00b It's not `node server.js`, rather `npm start`. How can I do this? – Shubham Kushwah Jan 02 '19 at 07:55
@easymoden00b Do I have to change it in the `package.json` or in the **Google App Engine** somewhere? – Shubham Kushwah Jan 02 '19 at 07:55
@ShubhamKushwah Yes, you can change the `npm start` command in your `package.json` file from `node server.js` to `node --max-old-space-size=32000 server.js` or whatever limit you want. Note that it doesn't matter if you use underscores or hyphens in the "max-old-space-size" option. – May 15 '19 at 17:54
@Joe There's no server.js file. It's create-react-app structure. Can I do it like this with npm: `npm run start --max-old-space-size=32000` – Shubham Kushwah May 16 '19 at 19:51

Ben Creasy · Answer 4 · 2018-08-06T02:45:36.967

33

Note: see the warning in the comments about how this can affect Electron applications.

As of v8.0 shipped August 2017, the NODE_OPTIONS environment variable exposes this configuration (see NODE_OPTIONS has landed in 8.x!). Per the article, only options whitelisted in the source (note: not an up-to-date-link!) are permitted, which includes "--max_old_space_size". Note that this article's title seems a bit misleading - it seems NODE_OPTIONS had already existed, but I'm not sure it exposed this option.

So I put in my .bashrc:
export NODE_OPTIONS=--max_old_space_size=4096

edited Aug 06 '18 at 02:45

answered Feb 20 '18 at 23:41

Ben Creasy

3,825
4
40
50

1

Word of warning: do not set this as an environment variable on your Windows machine. It breaks _everything_. It broke GitKraken, Slack, the VS2017 installer (which is chromium-based), Azure-related VS2017 extensions, and even the "Extensions and Updates" menu item in VS2017. – Roman Starkov Aug 06 '18 at 01:17
1

Take a look at https://github.com/electron/electron/issues/12695 - looks like Electron won't crash from NODE_OPTIONS in 2.0.3 - but I set NODE_OPTIONS using smartcd only in specific project directories to avoid this issue https://github.com/cxreg/smartcd – Ben Creasy Aug 06 '18 at 02:44
Ah, that probably explains why Discord and VS Code were unaffected. – Roman Starkov Aug 06 '18 at 08:22
7

How to do that on a windows machine? – Mahmoud Ezzat Nov 20 '18 at 16:20

jfunk · Answer 5 · 2019-01-11T16:57:28.897

31

I found that max_new_space_size is not an option in node 4.1.1 and max_old_space_size alone did not solve my problem. I am adding the following to my shebang and the combination of these seems to work:

#!/usr/bin/env node --max_old_space_size=4096 --optimize_for_size --max_executable_size=4096 --stack_size=4096

[EDIT]: 4096 === 4GB of memory, if your device is low on memory you may want to choose a smaller amount.

[UPDATE]: Also discovered this error while running grunt which previously was run like so:

./node_modules/.bin/grunt

After updating the command to the following it stopped having memory errors:

node --max_old_space_size=2048 ./node_modules/.bin/grunt

edited Jan 11 '19 at 16:57

answered May 12 '16 at 18:21

jfunk

7,176
4
37
38

4

I added --max_old_space_size=8192 , it still didn't solve the issue but did hang my pc :( – Dhyey Aug 24 '16 at 07:34
I lowered the amount of memory in the example from 8Gb to 4Gb, thanks for the feedback. – jfunk Aug 24 '16 at 14:10

SuperStar518 · Answer 6 · 2019-05-14T19:24:56.293

23

The increase-memory-limit module has been deprecated now. As of Node.js v8.0 shipped August 2017, we can now use the NODE_OPTIONS env variable to set the max_old_space_size globally.

export NODE_OPTIONS=--max_old_space_size=4096

Ref URL: https://github.com/endel/increase-memory-limit

edited May 14 '19 at 19:24

answered Aug 15 '18 at 02:09

SuperStar518

2,814
2
20
35

Where do you set this? I am using a Azure DevOps build and builds are failing in windows agents – int-i Aug 01 '19 at 14:53
1

To avoid cross-platform difference, install `cross-env` module globally and run on terminal with such command - `cross-env PORT=8000 node --max-old-space-size=4096 server.js` – SuperStar518 Aug 01 '19 at 17:46
Azure Devops builds are run on MS agents on cloud, there are no consoles to execute them. I found another way to achieve these using powershell. Added it as an answer here. – int-i Aug 01 '19 at 18:55

score 13 · Answer 7 · answered Oct 18 '17 at 15:20

Just a variation on the answers above.

I tried the straight up node command above without success, but the suggestion from this Angular CLI issue worked for me - you create a Node script in your package.json file to increase the memory available to Node when you run your production build.

So if you want to increase the memory available to Node to 4gb (max-old-space-size=4096), your Node command would be node --max-old-space-size=4096 ./node_modules/@angular/cli/bin/ng build --prod. (increase or decrease the amount of memory depending on your needs as well - 4gb worked for me, but you may need more or less). You would then add it to your package.json 'scripts' section like this:

"prod": "node --max-old-space-size=4096 ./node_modules/@angular/cli/bin/ng build --prod"

It would be contained in the scripts object along with the other scripts available - e.g.:

"scripts": {
    "ng": "ng",
    "start": "ng serve",
    "build": "ng build",
    "test": "ng test",
    "lint": "ng lint",
    "e2e": "ng e2e",
    "prod": "node --max-old-space-size=4096./node_modules/@angular/cli/bin/ng build --prod"
}

And you run it by calling npm run prod (you may need to run sudo npm run prod if you're on a Mac or Linux).

Note there may be an underlying issue which is causing Node to need more memory - this doesn't address that if that's the case - but it at least gives Node the memory it needs to perform the build.

score 10 · Answer 8 · answered Dec 24 '18 at 07:12

My working solution is:

Install cross-env
npm install --save-dev cross-env or npm install -g cross-env.
File package.json add new build script
e.g.
... "build:prod:ios": "cross-env NODE_OPTIONS='--max-old-space-size=8192' ionic cordova build ios --prod --release" ...
Use that command to build next time.
npm run build:prod:ios
Problem solved.

score 10 · Answer 9 · answered May 10 '21 at 11:44

10

You should also check if you accidentally installed the x86 version of node instead of x64. Happened to me, because nodejs.org preselected x86 on my x64 machine...

answered May 10 '21 at 11:44

denns

1,105
1
11
24

4

I made the same mistake! Thanks. – Pablo Chvx May 16 '21 at 23:51
3

Underrated comment. This solved my issue. – Dan Gayle Jun 03 '21 at 23:17
1

Very underrated comment. I was wondering why nodejs was under Program Files (x86) whilst trying various ways in setting the max_old_space_size. Uninstalled x86 and installed x64; solved my issue. – Key007 Aug 30 '22 at 05:32

score 6 · Answer 10 · answered Nov 15 '19 at 09:16

6

In a Windows Machine run below command

set NODE_OPTIONS=--max_old_space_size=4096

answered Nov 15 '19 at 09:16

kartick shaw

915
13
5

Getting `FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory` – fabpico Apr 11 '22 at 12:59

score 3 · Answer 11 · answered Aug 01 '19 at 17:07

Anyone getting this error with Azure build pipelines, try the below step to change environment variable of build agent

Add an Azure build pipeline task -> Azure powershell script:Inlinescript before Compile with below settings

- task: AzurePowerShell@3
  displayName: 'Azure PowerShell script: InlineScript'
  inputs:
    azureSubscription: 'NYCSCA Azure Dev/Test (ea91a274-55c6-461c-a11d-758ef02c2698)'
    ScriptType: InlineScript
    Inline: '[Environment]::SetEnvironmentVariable("NODE_OPTIONS", "--max_old_space_size=16384", "Machine")'
    FailOnStandardError: true
    azurePowerShellVersion: LatestVersion

This one would be better as a comment (since it doesn't answer the OP), but it's still a very useful hint! — Charles Roberto Canato, Nov 10 '20 at 01:02

score 2 · Answer 12 · edited Jul 19 '17 at 11:43

2

I lost some days with this problem.... until I found that in some file I was importing one static file, a built file. It make the build to never end. Something like:

import PropTypes from "../static/build/prop-types";

Fixing to the real source solved all the problem.

Sharing my solution. :)

edited Jul 19 '17 at 11:43

oligofren

20,744
16
93
180

answered Sep 19 '16 at 11:02

Tiago Gouvêa

15,036
4
75
81

2

could you explain your solution a little more? I'm having this same issue and it is crashing my application. Everything works fine for the first few minutes of the app running, but then it finally crashes from running out of memory. But during that period Sequelize works fine. – Porlune Jan 08 '17 at 05:12
1

What happend, is that my IDE (WebStorm) automaticly included the BUILDED js on my sources... when one file (on my sources) was changed, it build all files into a new js output. But, if my sources have references to builded files, it never stops to build. Check what file are including a incorrect file. – Tiago Gouvêa Jan 10 '17 at 11:48

score 2 · Answer 13 · answered Apr 19 '18 at 23:50

I was seeing this issue when I was creating a bundle to react-native. Things I tried and didn't work:

Increasing the node --max_old_space_size, intrestingly this worked locally for me but failed on jenkins and I'm still not sure what goes wrong with jenkins
Some places mentioned to downgrade the version of node to 6.9.1 and that didn't work for me either. I would just like to put this here as it might work for you.

Thing that did work for me: I was importing a really big file in the code. The way I resolved it was by including it in the ignore list in .babelrc something like this:

{
    "presets": ["react-native"],
    "plugins": ["transform-inline-environment-variables"],
    "ignore": ["*.json","filepathToIgnore.ext"]
}

It was a .js file which did not really needed transpiling and adding it to the ignore list did help.

Saad Mahmood · Answer 14 · 2019-02-27T11:21:28.610

2

I was facing this issue in ionic and tried many solutions but solved this by running this.

For MAC: node --max-old-space-size=4096 /usr/local/bin/ionic cordova build android --prod

For Windows: node --max-old-space-size=4096 /Users/{your user}/AppData/Roaming/npm/node_modules/ionic/bin/ionic cordova build windows --prod

edited Feb 27 '19 at 11:21

answered Jan 04 '19 at 06:52

Saad Mahmood

167
11

score 2 · Answer 15 · edited Apr 09 '19 at 13:12

npm install -g increase-memory-limit

increase-memory-limit

OR

Navigate to %appdata% -> npm folder or C:\Users\{user_name}\AppData\Roaming\npm
Open ng.cmd in your favorite editor
Add --max_old_space_size=8192 to the IF and ELSE block

now ng.cmd file looks like this after the change:

@IF EXIST "%~dp0\node.exe" (
  "%~dp0\node.exe" "--max_old_space_size=8192" "%~dp0\node_modules\@angular\cli\bin\ng" %*
) ELSE (
  @SETLOCAL
  @SET PATHEXT=%PATHEXT:;.JS;=;%
  node "--max_old_space_size=8192" "%~dp0\node_modules\@angular\cli\bin\ng" %*
)

score 2 · Answer 16 · answered Jul 04 '20 at 08:08

#!/usr/bin/env node --max-old-space-size=4096 in the ionic-app-scripts.js dint work

But after Modifying: the following file it worked

node_modules/.bin/ionic-app-scripts.cmd

By adding:

@IF EXIST "%~dp0\node.exe" ( "%~dp0\node.exe" "%~dp0..@ionic\app-scripts\bin\ionic-app-scripts.js" %* ) ELSE ( @SETLOCAL @SET PATHEXT=%PATHEXT:;.JS;=;% node --max_old_space_size=4096 "%~dp0..@ionic\app-scripts\bin\ionic-app-scripts.js" %* )

R15 · Answer 17 · 2021-03-19T16:01:57.633

I replaced

ng build --prod

with

node --max_old_space_size=8192 ./node_modules/@angular/cli/bin/ng build --prod

So instead of using ng build --prod just use above command starts with node... To generate production build there is not need to use ng server --prod.

If you don't want to use --prod, just remove it.

So these are the commands to use in vs code terminal

node --max_old_space_size=8192 ./node_modules/@angular/cli/bin/ng build --prod
node --max_old_space_size=8192 ./node_modules/@angular/cli/bin/ng serve

score 1 · Answer 18 · answered Feb 04 '19 at 09:29

1

this error occurs when required memory allocated for execution is less than memory required for running process. By default, Node memory size is 512 mb to increase this you need to type the following command :

node --max-old-space-size= <NewSize> <fileName>

answered Feb 04 '19 at 09:29

Khandelwal-manik

353
3
13

1

what is the filename here? – Rashedul.Rubel Jun 28 '19 at 11:19

score 0 · Answer 19 · answered Mar 29 '18 at 13:53

0

An alternative solution is to disable the AOT compiler:

ng build --prod --aot false

answered Mar 29 '18 at 13:53

Kyle Krzeski

6,183
6
41
52

1

can u elaborate why false ? – Codebrekers Feb 14 '19 at 11:36

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory

19 Answers19

Linked