Confused about Intel Optane DC SSD usage as extra RAM with IMDT?

Question

I'm a little confused about Intel Optane DC. I want that my Optane DC will be able to perform as DRAM and storage both. On the one hand, I understood that only "Intel Optane DC Persistent Memory DIMM" is able to perform as DRAM.That it because he has 2 modes (Memory mode and App-Direct Mode). On the other hand, in this link: https://www.intel.com/content/www/us/en/products/docs/memory-storage/solid-state-drives/optane-ssd-dc-p4800x-mdt-brief.html I read that "Together, DRAM and Intel® Optane™ SSDs with Intel® Memory Drive Technology emulate a single volatile memory pool". I'm confused, is Intel Optane DC SSD is able to perform as DRAM or only Intel persistent Memory DIMM?

Answer 1

Yes you can use a P4800x with Intel's IMDT (Intel Memory Drive Technology) software to give the illusion of more RAM by using the Optane DC SSD as swap space. This is what you want. IMDT sets up a hypervisor that gives the OS the illusion of DRAM + SSD as physical memory, instead of just letting the OS use it as swap space normally.

Apparently this works well when you already have enough physical RAM for most of your working set, and IMDT has smart prefetching algorithms that try to page in ahead of when a page will be needed.

One advantage to running the OS under the IMDT hypervisor instead of just using the SSD as swap space is that it will get the OS to use some of that extra space for pagecache (aka disk caching), instead of needing special code to use (some of) an SSD as cache for a slower disk.

---

But no, it's not Optane DC Persistent Memory, that's something else.

See also a SuperUser answer for more about Optane vs. Optane DC PM. And Hadi Brais added some nice sections to it about IMDT for Optane SSDs.

P4800x is connected over PCI-express (as you can see in pictures on https://www.anandtech.com/show/11930/intel-optane-ssd-dc-p4800x-750gb-handson-review) for example. So it's not an NV-DIMM; you can't stick it in a DIMM socket and have the CPU access it over the memory bus. The form-factor isn't DIMM.

As far as hardware, there are 3 things with the Optane brand name:

• Consumer grade "Optane" SSDs. Just a fast PCIe NVMe using 3D XPoint memory instead of NAND flash.

• Enterprise "Optane DC" SSDs. Just a fast PCIe NVMe using 3D XPoint memory. Not fundamentally different from the consumer stuff, just faster and higher power-consumption. P4800x is this.

  The "expand your RAM" functionality here is pure software, fairly similar (and possibly worse) than just creating a swap partition on it and letting the OS handle paging to it. Especially if you weren't using virtualization already.

• Enterprise "Optane DC Persistent Memory" (PM for short). 3D XPoint memory that's truly mapped (by hardware) into physical address space for access with ordinary load/store instruction, without going through a driver for each read/write. e.g. Linux mmap(MAP_SYNC) and using clflush or clwb asm instructions in user-space to commit data to persistent storage.

  PM is still slower than DRAM, though, so if you just want volatile memory you might still use it as swap space like IMDT. One key use-case for DC PM is giving databases the ability to commit to persistent storage without going through the OS. This allows out-of-order execution around I/O, as well as much lower overhead.

See articles like https://www.techspot.com/news/79483-intel-announces-optane-dc-persistent-memory-dimms.html which put Optane DC Persistent Memory above Optane DC in the classic pyramid storage hierarchy.

AFAIK, Optane DC PM devices only exist in a DIMM form-factor, not PCIe (and uses something like DDR4 signalling). This requires special support from the CPU because modern CPUs integrate the memory controller.

In theory you could have a PCIe device that exposed some persistent storage in a PCIe memory region. Those are part of physical address space and can be configured as write-back cacheable. (Or can they? Mapping MMIO region write-back does not work) So they could be memory-mapped into userland virtual address space. But I don't think any PCIe Optane DC Persistent Memory devices exist, probably because PCIe command latency is (much) higher than over the DDR4 bus. Bandwidth is also lower. So it makes sense to use it as fast swap space (copying in a whole page), not as write-back cacheable physical memory where you could have cache misses waiting a very long time.

(Margaret Bloom also comments re: block size of writes maybe being a problem.)

i.e. you don't want a "hot" part of your working set on memory that the CPU accesses over the PCIe bus. You probably don't even want that for Optane DC PM.

---

Optane / 3D XPoint is always persistent storage; it' up to software whether you take advantage of that or just use it as slower volatile RAM.

It's not literally DRAM, that has a specific technical meaning (dynamic = data stored in tiny capacitors that need refreshing frequently). 3D XPoint isn't dynamic, and isn't even volatile. But you can use it as equivalent because 3D XPoint memory has very good write endurance (it doesn't wear out like NAND flash). If people talk about using Optane as more DRAM, they're using the term to just mean volatile RAM, filling the same role that DRAM traditionally fills.