Nodejs heap out of memory during findOne Sequelize method with eager loading

Question

My NodeJS application currently uses a Sequelize object for working across multiple stages. This object is the result of a Model.findOne operation, which includes several other models, similar to a left join with multiple tables in plain SQL.

transaction  = await Transaction.findOne({
            where: {
                id: transaction.id
            },
            include: transactionIncludes //this includes other models such as client, orderDetail, orderLog, etc
        })
    }
    return transaction

Sometimes, my application gets to mount this transaction object fine. However, when I do try to scale it, I often get these OOM errors such as:

<--- Last few GCs --->

[736:0x60632a0]    72273 ms: Mark-sweep (reduce) 2036.3 (2045.7) -> 2036.2 (2045.7) MB, 13.2 / 0.0 ms  (+ 0.1 ms in 4 steps since start of marking, biggest step 0.1 ms, walltime since start of marking 184 ms) (average mu = 0.943, current mu = 0.951) final[736:0x60632a0]    72342 ms: Mark-sweep (reduce) 2041.9 (2051.4) -> 2041.9 (2051.4) MB, 10.9 / 0.0 ms  (+ 0.5 ms in 4 steps since start of marking, biggest step 0.3 ms, walltime since start of marking 64 ms) (average mu = 0.931, current mu = 0.833) finali

<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb2c2b0 node::Abort() [node]
 2: 0xa4025c node::FatalError(char const*, char const*) [node]
 3: 0xd1d11e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xd1d497 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xed68f5  [node]
 6: 0xee7d3d v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 7: 0xee8bf7 v8::internal::Heap::FinalizeIncrementalMarkingIfComplete(v8::internal::GarbageCollectionReason) [node]
 8: 0xeec6b0 v8::internal::IncrementalMarkingJob::Task::RunInternal() [node]
 9: 0xddae0b non-virtual thunk to v8::internal::CancelableTask::Run() [node]
10: 0xb9a7f4 node::PerIsolatePlatformData::RunForegroundTask(std::unique_ptr<v8::Task, std::default_delete<v8::Task> >) [node]
11: 0xb9c669 node::PerIsolatePlatformData::FlushForegroundTasksInternal() [node]
12: 0x159d8a6  [node]
13: 0x15afe14  [node]
14: 0x159e1f8 uv_run [node]
15: 0xa65fe5 node::SpinEventLoop(node::Environment*) [node]
16: 0xb6ed86 node::NodeMainInstance::Run(node::EnvSerializeInfo const*) [node]
17: 0xaef83a node::Start(int, char**) [node]
18: 0x7f8cda3cb09b __libc_start_main [/lib/x86_64-linux-gnu/libc.so.6]
19: 0xa62f6c  [node]
Aborted (core dumped)

The dumped core sits at around 2 gigs of memory. I'd like to solve this without increasing RAM.

Edit: someone has asked me to show includes with model associations, here they are:

[redacted]

show `transactionIncludes` and also how all models are associated with each other. — Anatoly, Dec 29 '21 at 19:54
hello, I have updated my question to include model associations. Thank you for your attention. — Pedro Maioli, Dec 29 '21 at 22:19
You showed `emissaoIncludes` instead of `transactionIncludes` — Anatoly, Dec 30 '21 at 09:29
I forgot to change const name. This is the correct object, sorry. — Pedro Maioli, Dec 30 '21 at 10:50
I don't see `VooIncludes` that is used in `transactionIncludes` — Anatoly, Dec 30 '21 at 11:11

score 1 · Accepted Answer · answered Dec 30 '21 at 11:19

If you have more than one hasMany associations included in a Sequelize query (on the same level or on the nested level) then you need to indicate separate: true for all of them so Sequelize will perform all such included queries separately to avoid multiplying records:

export const transactionIncludes = [
    {model: CartaoPagamentos, as: 'cartaopagamento', type: 'hasOne', foreignKey: 'transaction_id'},
    {model: Cliente, as: 'cliente', type: 'belongsTo', foreignKey: 'cliente_id'},
    {model: Fila, as: 'fila', type: 'hasOne', foreignKey: 'transaction_id'},
    {model: Voo, as: 'voo', type: 'hasMany', foreignKey: 'transaction_id', include: VooIncludes, separate: true }, 
    {model: Passageiros, as: 'passageiro', type: 'hasMany', foreignKey: 'transaction_id', separate: true },
    {model: Cartaomilhas, as: 'cartaomilha', type: 'hasOne', foreignKey: 'transaction_id'},
    {model: Robot, as: 'robot', type: 'belongsTo', foreignKey: 'robo_id'},
];

Also please look at my answer here (the part about hasMany to understand better what's going on under the hood when you indicate more than one hasMany in Sequelize queries without the separate: true option.

Nodejs heap out of memory during findOne Sequelize method with eager loading

1 Answers1