5

I am trying to profile ffmpeg using the Linux perf command in docker. In the first run, the output is fine with all function names displayed by "perf report". In the second run, it only shows addresses rather than function names. I am using a script that runs the ffmpeg twice in a docker.

The commands used in the script is:

perf record ffmpeg -y -i input.mp4 -c:v libx264 -b:v 1500k output1.mp4

sleep 5

perf record ffmpeg -y -i input.mp4 -c:v libx264 -b:v 1500k output1.mp4

The command to run the script:

docker run -it --privileged -v $(pwd):/cwd myDocker:someTag /cwd/my_script.sh

I noticed in the second run, it cannot find the libx264 library in perf.data.

The outputs of "perf report" are shown below. Any kind of help is greatly appreciated.

Output 1:

# Overhead  Command  Shared Object       Symbol
# ........  .......  ..................  .........................................................
#

 6.35%  ffmpeg   libx264.so.148      [.] x264_cabac_block_residual_rd_internal_ssse3_lzcnt
 3.91%  ffmpeg   libx264.so.148      [.] x264_cabac_encode_decision_bmi2
 3.70%  ffmpeg   ffmpeg              [.] quantize_and_encode_band_cost_UPAIR
 2.94%  ffmpeg   libx264.so.148      [.] x264_me_search_ref

Output 2:

# Overhead  Command  Shared Object       Symbol
# ........  .......  ..................  .........................................................
#

 1.00%  ffmpeg    (deleted)         [.] 0x00000000000c6e87
 0.90%  ffmpeg    (deleted)         [.] 0x00000000000c82c1
 0.71%  ffmpeg    (deleted)         [.] 0x00000000000c82e2
 0.69%  ffmpeg    (deleted)         [.] 0x00000000000c6e98

I appreciate it if someone can provide a clue.

Thank you in advance.

EDIT 1: I also read How can I get perf to find symbols in my program. That post does not answer my question as it does not address my situation.

EDIT 2: I also looked at perf can find symbol in the kernel ,but can not find symbol in my program. How to fix it?. The setting for my experiment is correct.

EDIT 3: When I run the script outside container, both iterations work perfectly fine. The only problem is when I run in in the docker container.

EDIT 4: After the first run, I ensured that both the binary and shared library object are visible to the ls and stat commands.

EDIT 5: After the first run, I ensured that both the binary and shared library object are visible to the nm and objdump commands. In addition, these commands show correct symbols.

user3784418
  • 71
  • 1
  • 5
  • Seems like perf can't find the libraries to get the symbols - is the data recorded in the same container that is trying to get the report? – Leonardo Dagnino Mar 15 '22 at 22:04
  • Thank you. Yes, same container. But how perf can run the libraries for the second time if it cannot find them? – user3784418 Mar 16 '22 at 19:26
  • That's strange, I thought you were maybe recording on a different container which would explain that situation. I also don't have any ideas, haven't used perf much. You could try reading the `perf.data` file since the filenames are stored plaintext in there (amidst a lot of binary data) – Leonardo Dagnino Mar 16 '22 at 21:20
  • Thank you for your comment. I already did so using perf script -D, which shows that the file is "(deleted)". That is, perf record cannot see the file while recording. – user3784418 Mar 16 '22 at 22:09

1 Answers1

1

The same happened to me.

To fix it, copy your executable to the same path just with a suffix of (deleted). In your case it will probably be:

sudo cp "/usr/bin/ffmpeg" "/usr/bin/ffmpeg (deleted)"

If you will strace the perf command, you will see that this is the file path it is looking for. Once you will do the copy, it will find the symbols there.

It seems like perf calls readlink on some /proc path that is already invalid, and thus gets the process exe path with this " (deleted)" suffix. It comes from kernel readlink implementation code, see here about it: Broken symlinks and a mysterious (deleted)

Oren Kishon
  • 509
  • 6
  • 8
  • 1
    You can use `ln` to make a hard-link instead of an actual copy of the file. Or maybe `ln -s` to make a symlink, which will survive upgrades of the binary without being affected or getting out of sync. – Peter Cordes Feb 24 '23 at 19:34