1

Why is primary and cache memory divided into blocks?

Hi just got posed with this question, I haven't been able to find a detailed explanation corresponding to both primary memory and cache memory, if you have a solution It would be greatly appreciated :)

Thank you

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
Michael Cunningham
  • 11
  • 4
  • 1
    What would be the alternative, and why would that be bad? – Scott Hunter Dec 11 '21 at 19:44
  • Recommended reading: [What Every Programmer Should Know About Memory?](https://stackoverflow.com/a/8126441). For example DDR memory burst transfer size is 64 bytes, which is not coincidentally the size chosen for cache blocks in most CPUs for over a decade. Having cache work with 1 byte blocks would need just as much space for tag, dirty, pseudo-LRU eviction tracking, and so on per block, but you'd have 1/64th as much actual data per amount of metadata. – Peter Cordes Dec 12 '21 at 02:29

2 Answers2

1

Cache blocks exploit locality of reference based on two types of locality. Temporal locality, after you reference location x you are likely to access location x again shorty. Spatial locality, after you reference location x you are likely to access nearby locations, location x+1, ... shortly.

If you use a value at some distant data center x, you are likely to reuse that value and so it is copied geographically closer, 150ms. If you use a value on disk block x, you are likely to reuse disk block x and so it is kept in memory, 20 ms. If you use a value on memory page x, you are like to reuse memory page x and so the translation of its virtual address to its physical address is kept in the TLB cache. If you use a particular memory location x you are likely to reuse it and its neighbors and so it is kept in cache.

Cache memory is very small, L1D on an M1 is 192kB, and DRAM is very big, 8GB on an M1 Air. L1D cache is much faster than DRAM, maybe 5 cycles vs maybe 200 cycles. I wish this table was in cycles and included registers but it gives a useful overview of latencies:

https://gist.github.com/jboner/2841832

The moral of this is to pack data into aligned structures which fit. If you randomly access memory instead, you will miss in the cache, the TLB, the virtual page cache, ... and everything will be excruciatingly slow.

Olsonist
  • 2,051
  • 1
  • 20
  • 35
1

Most things in computer systems are divided into chunks of fixed sizes: bytes, words, cache blocks, pages.

One reason for this is that while hardware can do many things at once, it is hardware and thus, generally can only do what it was designed for.  Making bytes of blocks of 8 bits, making words blocks of 4 bytes (32-bit systems) or 8 bytes (64-bit systems) is something that we can design hardware to do, and mostly in parallel.

Not using fixed-sized chunks or blocks, on the other hand, can make things much more difficult for the hardware, so, data structures like strings — an example of data that is highly variable in length — are usually handled with software loops.

Usually these fixed sizes are in powers of 2 (32, 64, etc..) — because division and modulus, which are very useful operations are easy to do in binary for powers of 2.

In summary, we must subdivide data into blocks because, we cannot treat all the data as one lump sum (hardware wise at least), and, treating all data as individual bits is also too cumbersome.  So, we chunk or group data into blocks as appropriate for various levels of hardware to deal with in parallel.

Erik Eidt
  • 23,049
  • 2
  • 29
  • 53