7

The line profiling output of google-pprof claims that most of the running time of my numerical C++ program is being spent in a function called __nss_database_lookup (see below). Apparently that function is for handling things like the passwd file on UNIX systems. My C++ program should only be doing numerical calculations, allocating memory, and passing a few custom C++ data types around.

What's going on? Is the appearance of that function a mirage, a mere artefact of how google-pprof works? Or is it actually being called and wasting two thirds of my program's running time? If it is being called, what could be calling it? Has something mistakenly called it in one of my C++ classes? How would I track that down?

I'm using Ubuntu 20.04, g++-7 and g++-9.

Total: 1046 samples
     665  63.6%  63.6%      665  63.6% __nss_database_lookup ??:0
     107  10.2%  73.8%      193  18.5% <function1> file.h:1035
      92   8.8%  82.6%       92   8.8% <function2> file.h:...
      87   8.3%  90.9%       87   8.3% <function3> file.h:995
      17   1.6%  92.5%      734  70.2% <function4> file.h:1128
...

(Function and file names obscured for confidentiality reasons)

Tom Ellis
  • 9,224
  • 1
  • 29
  • 54
  • 2
    This kind of report does not say anything about what part of code calls `__nss_database_lookup`. To get whole picture, you can build [CPU Flame Graphs](http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html) using perf. – ks1322 Mar 23 '21 at 12:23
  • While you're futzing around with pprof, just [*do this*](https://stackoverflow.com/a/378024/23771) and find out what's *really* going on. – Mike Dunlavey Mar 24 '21 at 17:21

1 Answers1

16

A friend of mine met the similar issue today. Though it has been a while after you raised the question, but I still would like to answer it so that anyone else who reaches here can get some hints.

This is because some local symbols (which corresponds to static local functions in C/C++) are called, and these symbols don't have their entries in the symbol table, and their text (code) is placed after __nss_database_lookup. So your perf tool treats them as a part of __nss_database_lookup.

For example, your program may call memcpy, and memcpy calls __memmove_unaligned_avx_erms, which is a local symbol in glibc and isn't exported in dynamic symbol table, and its code is placed after __nss_database_lookup coincidentally together with other local symbols. And your perf tool can find nothing about __memmove_unaligned_avx_erms, so it just thinks __nss_database_lookup is called.

A potential solution is to install libc-dbg package (the package name may vary on various distros), and if your perf tool is smart enough to automatically load the debug info, it may annotate symbols correctly. (My friend checked that it took some effects on perf tool)

zsrkmyn
  • 547
  • 1
  • 5
  • 20
  • It seems that gperftools/pprof is not smart enough to load the libc-dbg symbols :-/ – Alex Jan 24 '23 at 00:38