Artillery JavaScript heap out of memory

Question

The laodtest fails to run after changing the arrivalrate from 10 to 100.

Artillery: 1.6.0-27 Artillery Pro: not installed Node.js: v10.15.0 OS: darwin/x64

:test $ artillery run -o report.json artillery.yml
Started phase 0, duration: 10s @ 10:01:42(+0000) 2019-03-10
.
<--- Last few GCs --->

[62621:0x102803200] 9478 ms: Mark-sweep 1392.4 (1401.5) -> 1392.3 (1401.5) MB, 20.1 / 0.0 ms (average mu = 0.439, current mu = 0.002)

last resort GC in old space requested [62621:0x102803200] 9498 ms: Mark-sweep 1392.3 (1401.5) -> 1392.3 (1401.5) MB, 20.6 / 0.0 ms (average mu = 0.277, current mu = 0.001) last resort GC in old space requested

<--- JS stacktrace --->

==== JS stack trace =========================================

0: ExitFrame [pc: 0x38a6205dbe3d]

Security context: 0x1da57481e6e1
1: byteLength [0x1da5274066f1] [buffer.js:526] [bytecode=0x1da597d26509 offset=126](this=0x1da5d7c5fbc1 <JSFunction

Buffer (sfi = 0x1da573a14251)>,string=0x1da597e082b9 ,encoding=0x1da5d92026f1 ) 2: arguments adaptor frame: 1->2 3: setContentLength(aka setContentLength) [0x1da5201841e9] [/Users//.nv...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: 0x10003b125 node::Abort() [/Users//.nvm/versions/node/v10.15.0/bin/node]
2: 0x10003b32f node::OnFatalError(char const*, char const*) [/Users//.nvm/versions/node/v10.15.0/bin/node]
3: 0x1001a8e85 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char

const*, bool) [/Users//.nvm/versions/node/v10.15.0/bin/node] 4: 0x1005742a2 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users//.nvm/versions/node/v10.15.0/bin/node] 5: 0x10057d7a4 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users//.nvm/versions/node/v10.15.0/bin/node] 6: 0x10054f055 v8::internal::Factory::NewRawOneByteString(int, v8::internal::PretenureFlag) [/Users//.nvm/versions/node/v10.15.0/bin/node] 7: 0x1006811a8 v8::internal::String::SlowFlatten(v8::internal::Handlev8::internal::ConsString, v8::internal::PretenureFlag) [/Users//.nvm/versions/node/v10.15.0/bin/node] 8: 0x1001c6c1d v8::String::Utf8Length() const [/Users//.nvm/versions/node/v10.15.0/bin/node] 9: 0x10004eaac node::Buffer::(anonymous namespace)::ByteLengthUtf8(v8::FunctionCallbackInfov8::Value const&) [/Users//.nvm/versions/node/v10.15.0/bin/node] 10: 0x10023170f v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo*) [/Users//.nvm/versions/node/v10.15.0/bin/node] 11: 0x100230c51 v8::internal::MaybeHandlev8::internal::Object v8::internal::(anonymous namespace)::HandleApiCallHelper(v8::internal::Isolate*, v8::internal::Handlev8::internal::HeapObject, v8::internal::Handlev8::internal::HeapObject, v8::internal::Handlev8::internal::FunctionTemplateInfo, v8::internal::Handlev8::internal::Object, v8::internal::BuiltinArguments) [/Users//.nvm/versions/node/v10.15.0/bin/node] 12: 0x1002302f0 v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) [/Users//.nvm/versions/node/v10.15.0/bin/node] 13: 0x38a6205dbe3d Abort trap: 6

My tests look like this.

.yml

config:
  target: "<URL_REMOVED"
  processor: "./getData.js"
  phases:
    - duration: 10
      arrivalRate: 100
scenarios:
  - flow:
      - function: "getData"
      - post:
          url: "/api/v2/auth"
          json:
            productId: "56729b6b77c82288f746c0cf"
          capture:
            json: "$.data.token"
            as: "token"
      - post:
          url: "/api/v2/sessions"
          headers:
            Authorization: 'Bearer {{token}}'
          json:
            productId: "56729b6b77c82288f746c0cf"
            jobId: "{{jobId}}"
          capture:
            json: "$.data.session._id"
            as: "sessionId"
      - post:
          url: "/api/v2/sessions/{{sessionId}}/document"
          headers:
            Authorization: "Bearer {{token}}"
          json:
            side: "front"
            payload: "{{frontDocument}}"
      - get:
          url: "/api/v2/sessions/{{sessionId}}/metrics/front"
          headers:
            Authorization: "Bearer {{token}}"
      - get:
          url: "/api/v2/sessions/{{sessionId}}/classification"
          headers:
            Authorization: "Bearer {{token}}"
      - get:
          url: "/api/v2/sessions/{{sessionId}}/end"
          headers:
            Authorization: "Bearer {{token}}"

getData.js

'use strict';
var faker = require('faker');
var FRONT_ID = require("./resources/id/front.json");

module.exports = {
    getData
};

function getData(userContext, events, done) {

    let jobId = faker.random.uuid()
    userContext.vars.jobId = jobId;
    userContext.vars.frontDocument = FRONT_ID.base64;
    return done();
}

Answer 1

Your nodejs, the instance running artillery, is out of RAM. Its default limit is about 1.4G.

Artillery, on *nix, probably gets started from /usr/bin/artillery when you install it with npm install -g.

That file's first line is, very likely,

#!/usr/bin/env node

Try changing that to

#!/usr/bin/env node  --max-old-space-size=8192

to get heap space of 8G. But don't grab more heap space than the machine running artillery has physical RAM, or you'll thrash. Read this: how to increase nodejs default memory?

Edit: the question remains: why does your artillery process blow out its heap. It looks like it ran for about 9s before it crashed. It may be that your system under test cannot keep up with the load you place on it with an arrival rate of 100. It may be that artillery keeps creating and queueing in-memory post and get requests, and they don't finish so they don't get released. Do your performance logs give you any hints about this?

The whole point of load-testing is to find the breaking point of your system under test. You wouldn't test a boat by loading ten sandbags into it, then saying "cool it works" and then dumping 100 sandbags into it. The boat would just sink. Why test a server that way? All you learn is it has a capacity somewhere between 10 and 100. Instead, ramp up the load, sandbag by sandbag, until the system starts to stumble.

This is why artillery (and indeed most load-testing systems) has a way to ramp up the load.

Why not try something like this:

- duration: 120
  arrivalRate: 10
  name: "Two minutes, ten arrivals/sec"
- duration: 600
  arrivalRate: 10
  rampTo: 100
  name: "Ten minutes, gradual ramp to 100 arrivals/sec"

Once you know the arrival rate at which your system stumbles, you can do more detailed tests as needed. And, you can try to rework your code to make it faster, if necessary.

Also, a 10-second test isn't generally long enough to gather worthwhile data. You want to know your system's capacity at a steady high load.