Is cache miss possible in NodeJS and how to get it?

Question

From Wikipedia, A cache miss is a failed attempt to read or write a piece of data in the cache, which results in a main memory access with much longer latency.

However, I don’t understand what kind of data we are talking about and how to reproduce this synthetically. Based on this answer, cache optimization is performed by adjusting the algorithm to avoid data fragmentation when accessing.

/* Array.prototype.flat polyfill */
Array.prototype.flat = function() {
    this.reduce((a, v) => Array.isArray(v) ? a.concat(v.flat()) : a.concat(v), []);
};
/* Cache test */
const len = 100;
const generateArr = (len) =>  {
    const mat = [...new Array(len)].map(() => 
        [...new Array(len)].map(() => Math.round(Math.random() * 10))
    );
    return new Uint8Array(mat.flat(Infinity))
};
const arr = generateArr(len)
/* {1, 2, 3, 4, 5, 6, 7, 8, 9, n} */
const testFriendly = () => {
    let total=0;    
    for (let x=0;x!=len;x+=10) {
        for (let y=0;y!=10;y++) {
            total+=arr[x+y];
        }
    }
};
/* {0, 10, 20, 30, 40, 50, 60, 70, 80, 90, 1, 11, 12, n}*/
const testUnfriendly = () => {
    let total=0;
    for (let y=0;y!=10;y++) {
        for (let x=0;x!=len;x+=10) {
            total+=arr[x+y];
        }
    }
};
const test = () => {
    console.time("Cache-friendly");
    for (let i=0; i!=7000; i++) {
        testFriendly();
    }
    console.timeEnd("Cache-friendly");
    console.time("Cache-unfriendly");
    for (let i=0; i!=7000; i++) {
        testUnfriendly();
    }
    console.timeEnd("Cache-unfriendly");
};
test()
test()
test()
test()

After the JIT completes, the friendly test runs faster, but not always. Is this a cache miss?

But can an application programmer get a cache miss on the NodeJS platform? Are there any anti-patterns in order to guarantee a cache miss and prevent similar in production code?

Answer 1

what kind of data we are talking about?

Basically all data.

There are three places were data can be stored (written to, read from) :

(1) Registers directly on the processor [a few bytes].

(2) RAM (with the caches in front of it) [a few giga/terrabytes]

(3) I/O (harddisks, SSDs, Network adapters) [unlimited]

As you cannot access (1) and (3) directly, all your data ends up in the RAM basically.

Is cache miss possible in NodeJS and how to get it?

Sure. In the optimal case, were all your data you work with is cached, and the whole cached is utilized, a few kB get stored in L1, a few hundred kB get stored in L2, and a few MB get stored in L3. So if you work with more data than it is able to fit into these caches, cache misses will definetly happen (They will most likely happen evertheless).

After the JIT completes, the friendly test runs faster, but not always. Is this a cache miss?

Well, let's have a look at the stack you are operating on:

JavaScript <- you are here
/* gets run by */
V8 / NodeJS <- optimizations do happen
/* is written in */
C++ <- optimizations do happen, the other question is at this level
/* gets compiled to */
bytecode <- optimizations do happen
/* gets finally run on */
The Processor <- here the caches come into play

So in other words: Unlike the C++ question, in NodeJS there is another level of optimization inbetween. That level might introduce other caches, might change the way data is represented in memory, or do other optimizations. You could analyze the bytecode that V8 executes, and optimize for that .. but the next V8 version might introduce another optimization, which makes your "optimizations" obsolete.

But can an application programmer get a cache miss on the NodeJS platform?

Sure. Somewhere under the hood, very likely, very often. But ... Can you do anything about it? No, not really. The V8 team or the C++ compiler implementors can, you cannot, really.

Are there any anti-patterns in order to guarantee a cache miss and prevent similar in production code?

Yeah well, don't write crazy code.

_{The estimations are taken from Nick Craver's blog, I strongly recommend reading that :)}