8

I found sometimes I got this error, but I see the python program is stilling running (not get terminated), wondering if it is just a warning or an error? I have searched for similar discussion, but not found a definite answer.

tcmalloc: large alloc (bytes, result) = 1073750016 0x7f8a3fc00000 @ 0x55aefbc75b29 0x55aefbc74bbb 0x55aefc178c65 0x7f8bf019d86d 0x7f8bf019f908 0x7f8bf019cac8 0x 0x 0x 0x
Lin Ma
  • 9,739
  • 32
  • 105
  • 175

2 Answers2

16

As per https://www.novell.com/support/kb/doc.php?id=7012805

"This message isn't really an error per se, it's more just informational.

There is an environmental variable, TCMALLOC_LARGE_ALLOC_REPORT_THRESHOLD, which by default is set to 1073741824 bytes. ~1GB So if there is an allocation that is == or exceeds 1GB of memory, this message will be displayed in the ndsd.log.

If there is a great deal of memory in the server, such as greater than 8G - 32GB and/or a fairly large eDirectory database you may see this message more readily. You can just modify the threshold to be greater than the typical alloc that your seeing in the ndsd.log."

I'm not sure this is specifically correct given I'm unaware of the environment you're using, I'd say the explanation can be generalized as a warning that 1GB of memory has been allocated by the running application.

Further info: https://github.com/gperftools/gperftools/issues/360

Edit: As per the OPS comments, it appears that the same error was produced when allocating 1gb and 21gb so it would be safe to assume that there is a programmatic error that is using all available memory, I would go back and review the code.

Madison Courto
  • 1,231
  • 13
  • 25
  • Thanks Madison, will it cause program to terminate? – Lin Ma Sep 19 '18 at 07:06
  • 1
    Not the warning no, but obviously if you were to run out of memory it would. – Madison Courto Sep 19 '18 at 07:36
  • Thanks for the explain Madison, if such warning happens, does it mean something is not right in my program -- e.g. allocate too much memory? Or it is ok if I really need to allocate such big memory space. – Lin Ma Sep 19 '18 at 08:10
  • 1
    It’s perfectly fine if it’s a memory intensive application, if it doesn’t crash I’d assume you’re not running out so there is no need to worry. – Madison Courto Sep 19 '18 at 08:15
  • @MadisonCourto I get this massage and do you have any ideas how to change the memory size? – GeekLee Oct 23 '18 at 01:44
  • @FavonianKong It's an environment variable so you can set it to whatever you like but interestingly it automatically increases the threshold for the message by a factor of 1.125 which equates to ~1000mb every 60 messages, more on this here: https://issues.apache.org/jira/browse/IMPALA-1426 depending on your distro you can set the var using export TCMALLOC_LARGE_ALLOC_REPORT_THRESHOLD but this will not be persistent, for persistence look here https://stackoverflow.com/questions/13046624/how-to-permanently-export-a-variable-in-linux hope this helps. – Madison Courto Oct 23 '18 at 02:46
  • @MadisonCourto Thanks for ur reply, I tried to `echo` the environment variable before export it like `echo $TCMALLOC_LARGE_ALLOC_REPORT_THRESHOLD` but it did not output anything. Then I used the export to set the variable and run my project but it still output the same warning, which confuse me very much, is that anything I do the wrong way? – GeekLee Oct 23 '18 at 13:40
  • @FavonianKong I'm not sure what distro you are using and I feel this question is better asked on a linux/unix community but "printenv" could print all the env vars for you. https://www.cyberciti.biz/faq/linux-list-all-environment-variables-env-command/ – Madison Courto Oct 23 '18 at 23:49
  • @MadisonCourto I set `TCMALLOC_LARGE_ALLOC_REPORT_THRESHOLD=21329330176 ` before running my Python script but it still crashes with `tcmalloc: large alloc 21329330176 bytes == 0x18e144000 @ 0x7f736dbc2001 0x7f736b6f6b85 0x7f736b759b43 0x7f736b75ba86 0x7f736b7f3868 0x5030d5 0x506859 0x504c28 0x506393 0x634d52 0x634e0a 0x6385c8 0x63915a 0x4a6f10 0x7f736d7bdb97 0x5afa0a`, how can I handle it? – yamini goel Feb 15 '19 at 09:35
  • @yaminigoel I'm not certain but it appears you have allocated 21gb of memory, do you have 21gb to allocate? – Madison Courto Mar 04 '19 at 02:41
-3

it's out of memory, see the picture below. I get this message when run out of memory.

enter image description here

You get this message because it's using up all your memory.

enter image description here

Chafik Boulealam
  • 516
  • 5
  • 10