16

The Node.js documentation on Buffer says:

Raw data is stored in instances of the Buffer class. A Buffer is similar to an array of integers but corresponds to a raw memory allocation outside the V8 heap. A Buffer cannot be resized.

Okay, so buffers are not stored in the V8 heap, but… where does Node.js actually store them? I can hardly imagine that it's on the stack, is it?

In other words: What exactly does

a raw memory allocation outside the V8 heap

actually mean?

Golo Roden
  • 140,679
  • 96
  • 298
  • 425
  • 6
    Maybe by "outside the V8 heap" it means outside the normal pool used for JavaScript objects. It could of course be allocated (via low-level `malloc()`) in the surrounding process. – Pointy Jul 15 '15 at 12:01
  • 3
    ref link : https://github.com/joyent/node/blob/master/src/node_buffer.cc#L191 – Hacketo Jul 15 '15 at 12:37
  • To fix the version of the ref link: https://github.com/joyent/node/blob/6036e4f5a833b27fb474f3bf891c405606fdce19/src/node_buffer.cc#L191 – Golo Roden Jul 15 '15 at 12:38
  • So, in other words: If it's using `malloc`, Node.js puts buffers onto the heap, it's just not V8's heap, but some other heap. Right? – Golo Roden Jul 15 '15 at 13:00
  • @Pointy If you turn your comment into an answer, and include the ref link and answer my last question (the one in the commit), I will happily accept your answer as marked :-) – Golo Roden Jul 15 '15 at 13:39
  • 1
    @GoloRoden well. hacketo even provided you the actual source code of nodejs that uses malloc. thats what i hate about stackoverflow.. you cannot accept multiple answers. – GottZ Jul 15 '15 at 16:02
  • 1
    related: [use of smalloc in io.js](http://stackoverflow.com/q/28066317/1048572) – Bergi Jul 15 '15 at 21:41

3 Answers3

2

out side of v8 heap area. read the below link will help you.

https://nodejs.org/api/smalloc.html#smalloc_smalloc

Nalla Srinivas
  • 913
  • 1
  • 9
  • 17
  • 1
    Smalloc was [deprecated in May 2015](https://nodejs.org/en/blog/weekly-updates/weekly-update.2015-05-08/) and is no longer used; the link in this answer is now a 404 error. – user56reinstatemonica8 Dec 19 '18 at 11:08
0

The idea of Buffer is to handle, process raw binary data which may appear when node.js and external data sources like network, filesystem are communicating. For such purposes Node use system API calls (mmap() in unix, VirtualAlloc() in windows mainly used for allocating anonymous memory regions -> physical memory of the computer system, not V8)

Before Node.js 4.0: Buffers were allocated in the heap memory of the V8 engine, along with JavaScript objects. They were subject to garbage collection and had the same memory management as other JavaScript objects.

Node.js 4.0 and later: Buffers larger than a certain threshold (~8 KB) are allocated outside the V8 heap in a separate memory pool called "Buffer pool" or "Backing store." (related topic) This memory is managed by Node.js directly using native system calls or APIs. The advantage of allocating larger Buffers outside the V8 heap is to reduce pressure on the garbage collector and improve performance for handling binary data. Smaller Buffers that are below the threshold are still allocated within the V8 heap and are subject to garbage collection.

Node.js 12.0 and later: The Buffer.poolSize property was introduced, allowing you to configure the size of the Buffer pool. By default, it is set to 8192 bytes (8 KB). You can adjust this value based on your application's needs.

So summary to your questions:

  1. where: Node.js stores buffers outside of V8 heap if size of buffer hits the limit which might be configured using Buffer.poolSize property
  2. why: to make processing of large binary data more easier

Hope that helps :)

int0x80
  • 21
  • 3
-3

Although i am a beginner in node but what i understand is raw memory allocator implies it just solely produce a block of memory on which it constructs buffer object.The data will be lost as you kill the node process.

Sahil
  • 75
  • 5