2

I am using the Intel CMT-CAT distribution (cache management technology - cache allocation technology) at https://github.com/intel/intel-cmt-cat on Ubuntu 18.04.

One of the programs in the distribution (_pqos_api_lock) calls lockf.c from the C std library glibc. Lockf produces a segmentation fault (SIGSEGV). Further debugging shows that it segfaults at the point where rdx switches from 0 to 0x7fffffffe0c8 (140737488347336).

Here is the output from my gdb session:

Breakpoint 1, lockf (fd=4, cmd=cmd@entry=1, len=len@entry=0) at lockf.c:36
36      lockf.c: No such file or directory.
(gdb) ni
39      in lockf.c
(gdb) i r rdi
rdi            0x4      4
(gdb) i r rsi
rsi            0x1      1
(gdb) i r rdx
rdx            0x0      0
(gdb) ni
36      in lockf.c
(gdb) i r rdi
rdi            0x4      4
(gdb) i r rsi
rsi            0x1      1
(gdb) i r rdx
rdx            0x0      0
(gdb) ni
42      in lockf.c
(gdb) i r rdi
rdi            0x4      4
(gdb) i r rsi
rsi            0x1      1
(gdb) i r rdx
rdx            0x0      0
(gdb) ni
36      in lockf.c
(gdb) i r rdi
rdi            0x4      4
(gdb) i r rsi
rsi            0x1      1
(gdb) i r rdx
rdx            0x0      0
(gdb) ni
36      in lockf.c
(gdb) i r rdi
rdi            0x4      4
(gdb) i r rsi
rsi            0x1      1
(gdb) i r rdx
rdx            0x0      0
(gdb) ni
0x00007ffff78f07bb      36      in lockf.c
(gdb) i r rdi
rdi            0x4      4
(gdb) i r rsi
rsi            0x1      1
(gdb) i r rdx
rdx            0x0      0
(gdb) ni
0x00007ffff78f07c0      36      in lockf.c
(gdb) i r rdi
rdi            0x4      4
(gdb) i r rsi
rsi            0x1      1
(gdb) i r rdx
rdx            0x0      0
(gdb) ni
46      in lockf.c
(gdb) i r rdi
rdi            0x4      4
(gdb) i r rsi
rsi            0x1      1
(gdb) i r rdx
rdx            0x0      0
(gdb) ni
39      in lockf.c
(gdb) i r rdi
rdi            0x4      4
(gdb) i r rsi
rsi            0x1      1
(gdb) i r rdx
rdx            0x0      0
(gdb) ni
0x00007ffff78f07c8      39      in lockf.c
(gdb) i r rdi
rdi            0x4      4
(gdb) i r rsi
rsi            0x1      1
(gdb) i r rdx
rdx            0x7fffffffe0c8   140737488347336
(gdb) ni

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff78f07c8 in lockf (fd=4, cmd=cmd@entry=1,     len=len@entry=0)
    at lockf.c:39
39      in lockf.c

At each new instruction, I show the state of rdi, rsi and rdx (the three registers used for a call to lockf). I am able to see debug line numbers in lockf, but not the source code lines. I want to see the source code lines to understand why I get this error.

I downloaded source for lockf.c from https://code.woboq.org/userspace/glibc/io/lockf.c.html, but the line numbers do not appear to match what's installed on my system. WIthout the source lines of the lockf.c version installed locally, I can't debug why it segfaults.

The libc6-dbg package is already installed and it's the latest version (according to Ubuntu). The installed version of glibc is 2.27-3:

$ ldd --version
ldd (Ubuntu GLIBC 2.27-3ubuntu1) 2.27

So to sum it up, given the scenario described above, how can I see the source code lines in lockf.c (or any other glibc function) from a debug session in gdb.

Finally, I'm calling these programs from a shared object, but I don't think that makes a difference in this case.

UPDATE:

At the point where gdb goes into lockf.c, I issued info source:

info source Current source file is lockf.c Compilation directory is /build/glibc-OTsEL5/glibc-2.27/io Source language is c. Producer is GNU C11 7.3.0 -mtune=generic -march=x86-64 -g -O2 -O3 -std=gnu11 -fgnu89-inline -fmerge-all-constants -frounding-math -fstack-protector-strong -fPIC -fexceptions -ftls-model=initial-exec -fstack-protector-strong. Compiled with DWARF 2 debugging format. Does not include preprocessor macro info.

But there is no /build/glibc-OTsEL5/glibc-2.27/io on my Ubuntu 18.04.

RTC222
  • 2,025
  • 1
  • 20
  • 53
  • isn't it a [function _pqos_api_lock](https://github.com/intel/intel-cmt-cat/blob/master/lib/cap.c#L200) not a program? – KamilCuk Mar 15 '20 at 22:36
  • I'm looking at that now. – RTC222 Mar 15 '20 at 22:38
  • 1
    `I downloaded source for lockf.c` - you have to download the same file as you have installed. [lattera/glibc/../lockf.c](https://github.com/lattera/glibc/blob/master/io/lockf.c) "looks" fine, but download the source package for the package you have installed on your distribution. And the side note: I highly doubt that the reason for crash comes from standard library. [ubuntu has glibc-sources](https://packages.ubuntu.com/pl/bionic/glibc-source) – KamilCuk Mar 15 '20 at 22:41
  • 1
    Are you sure that the root cause of the crash is in the `lockf` function? It's more likely that that it's somewhere in the code calling `lockf`. – Some programmer dude Mar 15 '20 at 22:41
  • @KamilCuk - your first link may be useful. I will look at the source from lattera to see. Some programmer dude - the crash occurs during the switch block in lockf.c. I don't suspect a flaw with lockf.c but it may help me understand why rdx switches to such a bizarre number and produces a segfault. – RTC222 Mar 15 '20 at 22:44
  • @KamilCuk - the lattera link does not appear to be the same source because the debug session shown above hits line 36, which is a close brace on that source. That's why I'd like gdb to show the source code lines and not just line numbers in lockf.c. – RTC222 Mar 15 '20 at 22:49
  • 1
    Here's how to get the correct glibc source and have gdb use it, on Ubuntu : https://stackoverflow.com/a/48287761/2554472 – Mark Plotnick Mar 15 '20 at 22:56
  • Mark, that looks good. I'll work with it now and post back. Thanks. – RTC222 Mar 15 '20 at 23:11
  • @Mark Plotnick - in your answer that you linked to, you say "2) download the source code corresponding to the installed version of the C library." How do I do that? I've been unable to find the corresponding source. I created a folder /opt/glibc_source and issued "apt source libc6" and got Reading package lists... Done E: You must put some 'source' URIs in your sources.list." Then "find $PWD -maxdepth 1 -type d -name 'glibc*' results in "/opt/glibc_source/" The package dpkg-dev is installed and is the latest version. – RTC222 Mar 15 '20 at 23:47
  • Also, I updated my question above to show the result of info source. At this point, I've followed your other post but don't know where to get the source code to download to the new folder. – RTC222 Mar 15 '20 at 23:49
  • with `apt-get install glibc-sources`. `But there is no` so create this directory and put the sources there. But `gdb` has the [source search path](https://sourceware.org/gdb/current/onlinedocs/gdb/Source-Path.html) functionality, so you could put it in some dir and point gdb to it. – KamilCuk Mar 15 '20 at 23:53
  • The source is at https://ftp.gnu.org/gnu/libc/ (version 2.27). I'm untar-ing it now. – RTC222 Mar 16 '20 at 00:03
  • 1
    @RTC222 I'll have to add how to enable disabled source repositories to my answer. For now, https://askubuntu.com/questions/158871/how-do-i-enable-the-source-code-repositories shows two ways to do it, by GUI or by editing the `/etc/apt/sources.list` file and running `apt update`. – Mark Plotnick Mar 16 '20 at 17:33
  • @Mark Plotnick - I look forward to your updated answer. For now, I downloaded the source for glic 2.27 (the version installed on my server) and added its path to gdb. It works for debugging, but it would be best if I could just point to wherever the existing source is located. – RTC222 Mar 17 '20 at 16:32
  • 1
    @RTC222 I've updated my answer. – Mark Plotnick Mar 17 '20 at 16:38

0 Answers0