How to handle Javascript heap out of memory - MongoDB, Node.js, React.js

Question

I'm working on a project where I need to display weather data to a site that I built. I'm pulling the data from weather sensors and storing it in a Mongo DB so that I can display the data from the past 24 hours. This works except when I import collected data from the past 2 years into the database and whenever I run the site it crashes due to 'JavaScript heap out of memory'. I checked my network and saw none of my routes are pulling any data giving me a 500 status code. I've checked my code to see if I'm doing something wrong in pulling the data but can't find any issues. I've included snips of my code and the error I'm in my terminal. If there's anything else I can do to provide more information please let me know. Thanks so much.

Example of one of my Routes

.get('/get-temp', (req, res, next) => {
 const WeatherData = new mongoose.model('temp', MCDataSchema, 'Weather');
 // {'Time Stamp': {"$gte":startDate, "$lt":endDate}}
 WeatherData.find({
   'Time Stamp': {
     "$gte": past24Hours
   }
 }, (err, result) => {
   if (err) {
     console.error(err);
   } else {
     var tempList = new Array();
     var tempObj;
     var nextTimeMin;
     result.forEach(object => {
       if (object['Time Stamp'] !== 'NaN') {
         if (tempList.length == 0 || (parseInt(object['Time Stamp']) >= nextTimeMin)) {
           tempList.push({
             timestamp: object['Time Stamp'],
             temp: object.AirTemp_C
           })
           tempObj = object;
           let time = parseInt(tempObj['Time Stamp']);
           nextTimeMin = time + 3456000;
         }
       }
     });
     res.send(tempList);
   }
 })
})

Terminal Error

<--- Last few GCs --->

[5188:000001642D522470]    28211 ms: Scavenge 2047.0 (2050.2) -> 2046.4 (2050.4) MB, 8.6 / 0.0 ms  (average mu = 0.242, current mu = 0.277) allocation failure
[5188:000001642D522470]    28269 ms: Scavenge 2047.0 (2050.4) -> 2046.6 (2050.4) MB, 12.7 / 0.0 ms  (average mu = 0.242, current mu = 0.277) allocation failure
[5188:000001642D522470]    28343 ms: Scavenge 2047.3 (2050.4) -> 2046.9 (2050.7) MB, 6.9 / 0.0 ms  (average mu = 0.242, current mu = 0.277) allocation failure


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 00007FF7C278730D]
    1: StubFrame [pc: 00007FF7C2712E38]
Security context: 0x00ccd16c08d1 <JSObject>
    2: deserializeObject(aka deserializeObject) [00000036B17982B1] [C:\Users\Eamon Duffy\Github\qu-weather\backend\node_modules\bson\lib\bson\parser\deserializer.js:~43] [pc=000000871BCA3A46](this=0x011a1e5804b1 <undefined>,0x01ea8d802a81 <Uint8Array map = 000001BC01364749>,323974,0x01ea8d802a51 <Object map = 0000033...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

Try https://stackoverflow.com/questions/38558989/node-js-heap-out-of-memory#38560292 — D. SM, Apr 27 '20 at 03:09
I've tried all those solutions that everyone posted in that thread in the past before making this post and none have made a difference in my problem. — dizzleduff, Apr 27 '20 at 20:34
It would help you to get better answers if you include everything you have tried, *and the results of each experiment*, in your question. — D. SM, Apr 27 '20 at 20:57
Yes, we later found out the server we were running it on didn't have enough memory. Once we wiped it of old content, we were successful. — dizzleduff, Feb 27 '21 at 03:33

How to handle Javascript heap out of memory - MongoDB, Node.js, React.js

0 Answers0