6

I have a program which uses mmap() and shared memory to efficiently access a large database file. I would like to experiment with huge pages to see if it speeds things up.

I thought that a quick and easy way would be to copy the database file into Linux's hugetlbfs directory and make a symlink to it in the old location.

However, this does not work because the cp command cannot write to the file. I suspect that files can only be created by calling the ftrunc() and mmap() system calls to write into it. I will probably try writing a copy tool that does this, unless I get an answer describing an existing tool.

I'm looking for any other good methods to do shared memory maps using huge pages in Linux.

Rachid K.
  • 4,490
  • 3
  • 11
  • 30
Zan Lynx
  • 53,022
  • 10
  • 79
  • 131
  • "Transparent hugepages" might give you all the benefits without any special coding or copying all of the data into the hugetlb ramdisk. This feature is standard in recent kernels, and both Red Hat and Suse have back ported it to their latest enterprise releases (RHEL 6 update 2 and SLES 11 SP2, respectively) – Nemo Apr 27 '12 at 00:49
  • 1
    @Nemo: According to the docs, transparent hugepages currently only work with private anonymous memory. I was hoping for something to work with shared file mapping. – Zan Lynx Apr 27 '12 at 22:51
  • Hm, sounds like that is part of "future work". Well, the legacy hugetlbfs is only anonymous pages, too (a ramdisk, which is basically the same thing). So it looks like Linux does not presently support what you are trying to do. – Nemo Apr 28 '12 at 21:43
  • 2
    @Nemo: Yes, but in the ramdisk they are named anonymous pages and should be able to be opened in shared access. – Zan Lynx Apr 29 '12 at 22:53
  • Oracle [recommends](https://docs.oracle.com/en/database/oracle/oracle-database/19/ladbi/disabling-transparent-hugepages.html) *not* using transparent huge pages. Their recommendation is probably worth reading, especially if you're concerned about dynamic memory allocation that might be out of your control. Such a recommendation can easily be considered general. – Jeff Holt May 19 '23 at 23:05
  • So, Oracle recommends to disable them _for their database product_. The explanation given is very terse. As long as I do not need to run an Oracle database server, I rather keep the default that appears to be fine from the standpoint of kernel developers and all major distributors (including Oracle, according to the article you linked). – ypnos May 19 '23 at 23:20

2 Answers2

0

An old question now. But seeing as no one has answered and I'm actually wanting to experiment with huge page support too (for different reasons). I'll provide an answer.

Although huge pages are now transparent in modern kernels you can still get more control.

These functions may be what you're looking for.

get_huge_pages(), free_huge_pages(), get_hugepage_region(), free_hugepage_region()

You'll need to install libhugetlbfs which is a wrapper for the hugetlbfs.

Here's a Linux Weekly article that you may find helpful. Huge pages - part 1 (Introduction)

hookenz
  • 36,432
  • 45
  • 177
  • 286
0

Huge pages are a way to save TLB misses by mapping several consecutive physical memory pages (e.g. 4KB long) into a single huge one (e.g. 2 MB, 1 GB...). Two mechanisms are proposed by Linux:

  • Transparent huge pages (THP) managed transparently inside the kernel;
  • User controlled huge pages managed by the user space applications.

Added to that, an optional user space library called libhugetlbfs is available to provide a user friendly API and tools to use huge pages for the heap, the code and data segment of statically linked programs. This library hides the "raw" interface provided by the kernel. Depending on the applications, this may be useful or not enough flexible.

For the data base example problem of the question, the idea would be to create files in a hugetlbfs file system (it is actually a kind of RAM file system based an huge memory pages), extend the files (with ftrunc()) to a multiple of huge page size, mmap the files into memory and use those memory zones as a buffers to read/write database files. The performances are increased because less TLB entries are needed to access the memory pages.

This answer provide details on the way to map huge pages with the Linux "raw" interface.

Rachid K.
  • 4,490
  • 3
  • 11
  • 30