17

According to redis docs, it's advisable to disable Transparent Huge Pages.

Would the guidance be the same if the machine was shared between the redis server and the application.

Moreover, for other technologies, I've also read guidance that THP should be disabled for all production environments when setting up the server. Is this kind of pre-emptiveness applicable to redis as well, or one must first strictly monitor latency issues before deciding to turn off THP?

Hassan Baig
  • 15,055
  • 27
  • 102
  • 205

5 Answers5

12

Note: At the time this answer was originally written, turning off THP was still advised all the time. Things have changed since this and it is no longer advisable to turn off THP unless you are persisting to disk. I am only leaving this answer here for posterity's sake.

Turn it off. The problem lies in how THP shifts memory around to try and keep or create contiguous pages. Some applications can tolerate this, most databases cannot and it causes intermittent performance problems, some pretty bad. This is not unique to Redis by any means.

For your application, especially if it is JAVA, set up real HugePages and leave the transparent variety out of it. If you do that just make sure you allocate memory correctly for the app and redis. Though I have to say, I probably would not recommend running both the app and redis on the same instance/server/vm.

NoSQLKnowHow
  • 4,449
  • 23
  • 35
  • Well to be exact, I've got a Django (Python) application, although I fathom that won't change anything vis-a-vis your answer, would it? – Hassan Baig Mar 04 '17 at 11:22
  • No, it would not. – NoSQLKnowHow Mar 05 '17 at 17:31
  • 1
    Thanks Kirk! Btw to further clarify, I need to do **both** `echo never > /sys/kernel/mm/transparent_hugepage/enabled` and `echo never > /sys/kernel/mm/transparent_hugepage/defrag`, correct? – Hassan Baig Mar 06 '17 at 00:09
  • Moreover, in your experience, does req/s throughput suffer as a result of turning off THP? – Hassan Baig Mar 06 '17 at 01:10
  • More than likely, you need both places. You will need to have some kind of init script that starts on boot that will do it every time. – NoSQLKnowHow Mar 06 '17 at 21:27
  • Each statement/claim of this answer is incorrect or untrue. That, which is claimed with no evidence, can be dismissed without any evidence. – Maxim Egorushkin Jun 23 '23 at 22:40
10

Turning off transparent hugepages is a bad idea, and redis no longer recommends it.

What you should do instead is make sure transparent_hugepage is not set to always. (This is what recent versions of redis check for.) You can check the current value of the setting with:

$ cat /sys/kernel/mm/transparent_hugepage/enabled

And correct it like so:

# echo madvise >/sys/kernel/mm/transparent_hugepage/enabled

Although no action is likely to be necessary, since madvise is typically the default setting in recent linux distros.

Some background:

  • transparent_hugepage = always: can force applications to use hugepages unless they opt out with madvise. This has a lot of problems and is rarely enabled.
  • transparent_hugepage = never: does not fulfill allocations with hugepages, even if the application requests it with madvise
  • transparent_hugepage = madvise: allows applications to opt-in to hugepages. This is normally a good idea because hugepages can improve performance in some applications, but this setting doesn't force them on applications that, like redis, don't opt in
Kaz Wesley
  • 161
  • 1
  • 5
  • It is still recommended by Redis documentation when using Redis persistence: https://redis.io/docs/management/optimization/latency/#latency-induced-by-transparent-huge-pages – Jose Nobile Dec 09 '22 at 22:52
4

It is rather annoying that searching for "transparent huge pages" yields top results about how to disable them because Redis and some other databases cannot handle transparent huge pages without performance degradation.

These applications should do any of:

  • Use prctl(PR_SET_THP_DISABLE, ...) call to opt out from using transparent huge pages.
  • Be started by a script which does this call for them prior to fork/exec the database process. PR_SET_THP_DISABLE get inherited by child processes/threads for exactly this scenario when an existing application cannot be modified.

prctl(PR_SET_THP_DISABLE, ...) has been available since Linux 3.15 circa 2014, so that there is little excuse for those databases to not mention this solution, instead of giving this poor/panic advice to their users to disable transparent huge pages for the entire system.

3 years after this question was asked, Redis got disable-thp config option to make prctl(PR_SET_THP_DISABLE, ...) call on its own, by default.

My production memory-intensive processes go 5-15% faster with /sys/kernel/mm/transparent_hugepage/enabled set to always. Many popular desktop applications benefit from always transparent huge pages immensely.

This is why I cannot appreciate those search results for "transparent huge pages" spammed with Redis adviсe to disable them. That's a panic advice from Redis, not the best practice.

Maxim Egorushkin
  • 131,725
  • 17
  • 180
  • 271
-1

The overhead THP imposes occurs only during memory allocation, because of defragmentation costs.

If your redis instance has a (near-)constant memory footprint, you can only benefit from THP. Same applies to java or any other long-lived service that does its own memory management. Pre-allocate memory once and benefit.

milan
  • 2,355
  • 2
  • 23
  • 38
-6

why playing such echo-games when there is a kernel-param you can boot with?

transparent_hugepage=never

Harald Reindl
  • 21
  • not sure why this answer got downvoted. Here's another reference https://unix.stackexchange.com/a/99172/29023 – chachan Jan 08 '18 at 17:22
  • perhaps you didn't need to call the other answer "echo-games" and show more details about what you wanted to say as the link above – chachan Jan 08 '18 at 17:24
  • sorry, but when one is not capable to copy&paste "transparent_hugepage=never" into Google or don't know what a kernel parameter is he's not the audience anyways – Harald Reindl Jan 09 '18 at 18:00
  • 4
    Simple. The OP raised a number of concerns and _none_ of them were addressed by this answer. – Amir Aug 19 '18 at 11:39
  • 1
    i refered to "echo never > /sys/kernel" which is not persistent while "transparent_hugepage=never" as kenerl param is a) persistent and b) as early as possible set – Harald Reindl Aug 20 '18 at 12:05
  • This is to be set in `sysctl.conf`? – Hassan Baig Feb 04 '19 at 02:31
  • The question in "When", not "How". For recent distro, I think [the answer](https://stackoverflow.com/a/64945381/186347) by Kaz Wesley (i.e., do not turn off) is more appropriate. – ntd Jan 07 '21 at 07:32