81

From the 11th Chapter(Performance and Scalability) and the section named Context Switching of the JCIP book:

When a new thread is switched in, the data it needs is unlikely to be in the local processor cache, so a context-switch causes a flurry of cache misses, and thus threads run a little more slowly when they are first scheduled.

  1. Can someone explain in an easy to understand way the concept of cache miss and its probable opposite (cache hit)?
  2. Why context-switching would cause a lot of cache miss?
Gray
  • 115,027
  • 24
  • 293
  • 354
Geek
  • 26,489
  • 43
  • 149
  • 227
  • 6
    Cache hit - **found** in cache, without having to go "further" (into memory, disk, etc). There isn't much else to it. – Anirudh Ramanathan Sep 01 '13 at 14:25
  • 2
    See also [What Every Programmer Should Know About Memory](http://www.akkadia.org/drepper/cpumemory.pdf). It's old but still relevant, except for the part about prefetch threads and so much software prefetching. That part mostly only applies to Pentium4, but everything else about multi-level caches and laying out your data for locality and sequential access still applies. – Peter Cordes Dec 13 '16 at 18:23

5 Answers5

116

Can someone explain in an easy to understand way the concept of cache miss and its probable opposite (cache hit)?

A cache miss, generally, is when something is looked up in the cache and is not found – the cache did not contain the item being looked up. The cache hit is when you look something up in a cache and it was storing the item and is able to satisfy the query.

Why context-switching would cause a lot of cache miss?

In terms of memory, each processor has a memory cache – a high speed copy of small portions of main memory. When a new thread is context switched into a processor, the local cache memory is empty or it doesn't correspond to the data needed for the thread. This means that all (or most) memory lookups made by that new thread result in cache misses because the data that it needs is not stored in the local memory cache. The hardware has to then make a number of requests to main memory to fill up the local memory cache which causes the thread to initially run slower.

Gray
  • 115,027
  • 24
  • 293
  • 354
  • 3
    I added a second part to this question. – Geek Sep 01 '13 at 14:29
  • "each processor has a memory cache" - Is this memory cache different to the L1, L2 etc caches or is it the same? – Rajeev Mehta Jun 19 '17 at 14:02
  • 1
    The same @RajeevMehta. – Gray Jun 19 '17 at 14:41
  • I'm just doing some testing on `x-cache`. And I'm seeing some unexpected behavior. Even though it's the very first time I'm `GET`ing an image, I still get `HIT`.1. **Does `HIT` mean `HIT` cache on client, or means `HIT` on anything but the server ie you can get a *cached* result from a CDN?** Because I got "HIT from cloudfront" and looked it up and found out that cloudfront is a CDN. – mfaani Jul 16 '17 at 15:57
  • 2. I'm guessing that if I ever got `MISS` for the first time for a public image (`cache-control: public`) on my own computer... then 'my friend in the next room' would likely get a `HIT` because now the proxies have it cached. Right? – mfaani Jul 16 '17 at 16:33
  • I'd be speculating @Honey. I'd ask some questions about CDN/Cloudfront. – Gray Jul 17 '17 at 14:07
  • You mean this is CDN specific?! I'm aware that `x-cache` isn't a standard header, but it's that your answer is *vague* in that does `x-cache HIT` mean hit cache from client or proxy? Don't you agree? – mfaani Jul 17 '17 at 14:30
  • 5
    I'm just saying that I don't know and that you should ask another SO question specifically about how CDN's handle the cache headers. My answers were about internal thread/processor memory caches . – Gray Jul 17 '17 at 15:01
5

Whenever the processor wants to fetch data from main memory, first it will look at the cache buffer to see whether the corresponding address is present in the buffer. If it is there, it will perform the operation by using the cache; no need to fetch from the main memory. This is called a "Cache hit".

If the address is not present in the cache, it is called a "Cache miss". If a cache miss has occurred, that means the processor has go to main memory to fetch the address and it takes some more time.

Michael Myers
  • 188,989
  • 46
  • 291
  • 292
babu R
  • 51
  • 1
  • 2
1

You should also observer that if a context switch causes a return of a previously run thread to active state on a processor with access to the cached data, there is a chance that the required "working set" is still in the cache. The probability of this being true depends on the cache size (and structure). It also depends on the workload: How much demand for cache was there during the threads idle or waiting period, and how long the idle or waiting period lasts.

Joe Temple
  • 11
  • 2
0

If the processor finds that the memory location is in the cache, we say that a cache hit otherwise we speak of a cache miss.

Gray
  • 115,027
  • 24
  • 293
  • 354
priya
  • 1
  • 1
-4

If the desired data is in L1, then it's a cache hit. And if the desired data is in another cache memory level then it's a cache miss.

Michael Myers
  • 188,989
  • 46
  • 291
  • 292
mirzairtiza
  • 1
  • 1
    An access that misses in L1 but hits in L2 wouldn't usually be called a "cache miss". It's an L1 miss, but not just an unqualified "cache miss". – Peter Cordes Dec 13 '16 at 18:21