2

In the case of a JS array, it's possible to create one with predefined length. If we pass the length to the constructor, like new Array(itemCount), a JS engine could pre-allocate memory for the array, so it won't be needed to reallocate it while new items are added to the array.

Is it possible to pre-allocate memory for a Map or Set? It doesn't accept the length in the constructor like array does. If you know that a map will contain 10 000 items, it should be much more efficient to allocate the memory once than reallocate it multiple times.

Here is the same question about arrays.


There is discussion in the comments, whether the predefined array size has an impact on the performance or not. I've created a simple test to check it, the result is:

  • Chrome - filling an array with a predefined size is about 3 times faster
  • Firefox - there is no much difference, but the array with predefined length is a little bit faster
  • Edge - filling of a predefined size array is about 1.5 times faster

There is a suggestion to create the entries array and pass it to the Map, so the map can take the length from the array. I've created such test too, the result is:

  • Chrome - construction of a map which accepts entries array is about 1.5 times slower
  • Firefox - there is no difference
  • Edge - passing of entries array to the Map constructor is about 1.5 times faster
TylerH
  • 20,799
  • 66
  • 75
  • 101
Valeriy Katkov
  • 33,616
  • 20
  • 100
  • 123
  • 1
    *"a JS engine could pre-allocate memory for the array"* Could it really though? An array can contain heterogeneous data. While I have to admit not having insights into browser engines, it doesn't look like there is *that* much that can be done there. But of course I could be completely wrong. Edit: I had a look at the linked question, but I remain doubtful whether the performance differences are actually due to memory allocation or something else. – Felix Kling Jan 30 '20 at 15:12
  • Do you understand, how hashmaps work? Also, "preallocating", aka `Array(count)` also probably does not do what you think it does. – ASDFGerte Jan 30 '20 at 15:12
  • 1
    @ASDFGerte do you? Hashmaps have a size too, and growing them is even more complicated than growing arrays .... – Jonas Wilms Jan 30 '20 at 15:24
  • 1
    I believe the previous comments are taking umbrage with the idea that `new Array(size)` allocates memory for the elements in the array. It doesn't. It allocates memory for the "slots" in which the array elements will be placed. This does lead to performance benefits for arrays. `Map`s and `Set`s use different algorithms for storage of their data, if I'm not mistaken, and may not get the same performance benefits. – Heretic Monkey Jan 30 '20 at 15:25
  • @HereticMonkey *it does* how do you know that? Is your argument based on a benchmark made in 2015? – Jonas Wilms Jan 30 '20 at 15:29
  • 1
    Calling `new Map(array)` could take the `array`s length as a predicate for the size. Not sure if it does. – Jonas Wilms Jan 30 '20 at 15:33
  • @JonasWilms Where do I say "it does"? I say "It doesn't" referring to the fact that `new Array(size)` does not preallocate memory for its elements, because it can't. It doesn't know what's going to be in the array. How about, instead of trying to belittle others' statements and get all argumentative, you answer the question or present your own evidence? – Heretic Monkey Jan 30 '20 at 15:33
  • @HereticMonkey " It allocates memory for the "slots" in which the array elements will be placed. This does lead to performance benefits for arrays." I was referring to this part – Jonas Wilms Jan 30 '20 at 15:35
  • @JonasWilms Turns out, you can run those jsperf things on modern browsers. When I ran it in Edge Chromium it was 64.7% faster to set the size first... – Heretic Monkey Jan 30 '20 at 15:46
  • 1
    @JonasWilms an answer to what is actually happening, will be specific to a lot of implementation-dependent things. I am sometimes just poking, whether it makes any sense, to start going into the details this requires, or not. I am aware, that hashmaps grow too. The comment should not imply, that they don't. Heterogeneous data structures, and how engines do guesswork on a lot of parameters, e.g. size, or what content they will hold, and what structure will be internally allocated, is a large topic. – ASDFGerte Jan 30 '20 at 15:48
  • I've created a test to check whether the predefined size of an array has impact on the performance or not, and it looks like it has. I've added the test results to the question. I understand that it mostly depends on the engine realization, but I suppose that if an engine knows how may items a Map/Set will have, it could optimize the data structure somehow. – Valeriy Katkov Jan 30 '20 at 16:26
  • My (very naive) benchmark says that it takes *6ms* to add 10000 entries to a Map ... so how much time do you want to invest to get this down a bit ... Does it really matter? – Jonas Wilms Jan 30 '20 at 17:55
  • @JonasWilms Indeed, in most cases it doesn't matter. I just wonder is there a simple solution you can relay on or not... looks like not. You've suggested an interesting idea, to implement a map using a typed array, but indeed there should be a good reason to invest to it. I hope that your idea will be useful for someone, thank you! – Valeriy Katkov Jan 30 '20 at 18:30

2 Answers2

4

There is no way to influence the way the engine allocates memory, so whatever you do, you cannot guarantee that the trick you are using works on every engine or even at another version of the same engine.

There are however typed arrays in JS, they are the only datastructures with a fixed size. With them, you can implement your own hashmap with fixed size.

A very naive implementation is slightly faster² on inserting, not sure how it behaves on reads and others (also it can only store 8bit integers > 0):

function FixedMap(size) {
   const keys = new Uint8Array(size),
         values = new Uint8Array(size);
   
   return {
     get(key) {
        let pos = key % size;
        while(keys[pos] !== key) { 
          if(keys[pos] % size !== key % size) 
              return undefined;
          pos = (pos + 1) % size;
        }
        return values[pos];
     },
     set(key, value) {
       let pos = key % size;
       while(key[pos] && key[pos] !== key) pos = (pos + 1) % size;
       keys[pos] = value;
       values[pos] = value;
     }
   };
}
   

console.time("native");

const map = new Map();

for(let i = 0; i < 10000; i++)
  map.set(i, Math.floor(Math.random() * 1000));
  
console.timeEnd("native");

console.time("self");

const fixedMap = FixedMap(10000);

for(let i = 0; i < 10000; i++)
  fixedMap.set(i, Math.floor(Math.random() * 1000));
console.timeEnd("self");

² Marketing would say It is up to 20% faster! and I would say It is up to 2ms faster, and I spent more than 10minutes on this ...

Jonas Wilms
  • 132,000
  • 20
  • 149
  • 151
  • For fixed-size/static maps, it might even be worth it to search for a perfect hash function. – Bergi Jan 30 '20 at 19:57
  • @bergi the one I used is kinda perfect ... the benchmark is kinda useless ... :) – Jonas Wilms Jan 30 '20 at 21:03
  • I like the answer, i think Javascript is a bit too loose on this, at least needs to provide a way to specify the value type, like in `Uint32Array`. – windmaomao Dec 15 '20 at 17:30
1

No, this is not possible. It's doubtful even that the Array(length) trick still works, engines have become much better at optimising array assignments and might have chosen a different strategy to (pre)determine the size of memory to allocate.

For Maps and Sets, no such trick exists. Your best bet will be creating them by passing an iterable to their constructor, like new Map(array), instead of using the set/add methods. This does at least offer the chance for the engine to take the iterable size as a hint - although filtering out duplicates can still lead to differences.
Disclaimer: I don't know whether any engines implement, or plan to implement, such an optimisation. It might not be worth the effort, especially if the current memory allocation strategy is already good enough to diminish any improvements.

Bergi
  • 630,263
  • 148
  • 957
  • 1,375
  • Thanks for the opinion! I've added a test to the question, which checks `Array(length)` trick and it seems it works. I tested the solution you've suggested as well, to put an array of entries to the Map constructor, in Chrome it works even worse than putting elements one by one, so I wouldn't recommend it. But I agree with the main thought of your answer, thanks. – Valeriy Katkov Jan 30 '20 at 17:25
  • 1
    @ValeriyKatkov What a shame. I guess that's enough of a difference to be reported as a bug… – Bergi Jan 30 '20 at 19:55