6

I encountered a weird situation where performing pointer arithmetic involving dynamically linked symbols leads to incorrect results. I'm unsure if there are simply missing some linker parameters or if it's a linker bug. Can someone explain what's wrong in the following example?

Consider the following code (lib.c) of a simple shared library:

#include <inttypes.h>
#include <stdio.h>

uintptr_t getmask()
{
  return 0xffffffff;
}

int fn1() 
{
  return 42;
}

void fn2() 
{
  uintptr_t mask; 
  uintptr_t p;

  mask = getmask();
  p = (uintptr_t)fn1 & mask; 
  printf("mask: %08x\n", mask);
  printf("fn1:  %p\n",   fn1); 
  printf("p:    %08x\n", p);
}

The operation in question is the bitwise AND between the address of fn1 and the variable mask. The application (app.c) just calls fn2 like that:

extern int fn2();

int main()
{
  fn2();

  return 0;
}

It leads to the following output ...

mask: ffffffff
fn1:  0x2aab43c0
p:    000003c0

... which is obviously incorrect, because the same result is expected for fn1 and p. The code runs on an AVR32 architecture and is compiled as follows:

$ avr32-linux-uclibc-gcc -Os -Wextra -Wall -c -o lib.o lib.c
$ avr32-linux-uclibc-gcc -Os -Wextra -Wall -shared -o libfoo.so lib.o
$ avr32-linux-uclibc-gcc -Os -Wextra -Wall -o app app.c -L. -lfoo

The compiler thinks, it is the optimal solution to load the variable mask into 32 bit register 7 and splitting the &-operation into two assembler operations with immediate operands.

$ avr32-linux-uclibc-objdump -d libfoo.so

000003ce <fn1>:
 3ce:   32 ac           mov     r12,42
 3d0:   5e fc           retal   r12

000003d2 <fn2>:
 ...
 3f0:   e4 17 00 00     andh    r7,0x0
 3f4:   e0 17 03 ce     andl    r7,0x3ce

I assume the immediate operands of the and instructions are not relocated to the loading address of fn1 when the shared library is loaded into the applications address space:

  • Is this behaviour intentional?
  • How can I investigate whether problem occurs when linking the shared library or when loading the executable?

Background: This is not an academic questions. OpenSSL and LibreSSL use similar code, so changing the C source is not an option. The code runs well on other architectures and certainly there is an unapparent reason for doing bitwise operations on function pointers.

PRABA
  • 460
  • 1
  • 6
  • 19
hamari
  • 61
  • 2
  • 2
    `return 0xffffffff` -> `return ~(uintptr_t )0` ? – Bathsheba Nov 24 '16 at 09:21
  • 1
    What would `print(" fn1-x : %08x\n", (uintptr_t) fn1);` give? – Serge Ballesta Nov 24 '16 at 09:28
  • What do you get if you compile without optimizations? – Jabberwocky Nov 24 '16 at 09:36
  • 1
    @Bathsheba why would `return 0xffffffff` be different from `return ~(uintptr_t)0` on a 32 bit environnment? – Jabberwocky Nov 24 '16 at 09:38
  • You seem to be missing -fPIC when compiling lib.c (I'm surprised you didn't get a warning from compiler). Also format argument for p is wrong (I suggest to use %llx and cast to long long to avoid messing with inttypes.h). – yugr Nov 24 '16 at 09:40
  • `uintptr_t` is only defined for object pointers, not function pointers. Function pointers can not be cast to any other pointer type, including void. So your code is undefined behavior. Especially function pointers into shared libraries will not behave the way you expect them to because of how relocation is done. – Art Nov 24 '16 at 09:58
  • 1
    @Art `uintptr_t` is an integer, not a pointer. Function pointers may be converted to integers.. 6.3.2.3/6: "Any pointer type may be converted to an integer type." – Lundin Nov 24 '16 at 10:17
  • I can't reproduce this. What happens if you change to `printf("p: %08" PRIuPTR "\n", p);`? – Lundin Nov 24 '16 at 10:24
  • @yugr: The shared objects files are binary equal no matter whether using -fPIC or not. I think the compiler produces PIC by default. – hamari Nov 24 '16 at 11:51
  • @Lundin: Same result as above with `PRIuPTR`. The output will be merely decimal, but the value is the same. Have you tried on an AVR32 architecture? Which compiler and binutils do you use? On Intel the code works fine, since the relocation process is instruction-set dependant. – hamari Nov 24 '16 at 12:25
  • Nope I tried on a 32 bit ARM with gcc. I suppose there is something fishy with your compiler port. – Lundin Nov 24 '16 at 12:35
  • @Michael Walz: Without size-optimization the code works as expected in this simple case. But I encountered this problem in LibreSSL. Recompiling with changed compiler settings doesn't solve the problem there. If you are interested, the code in question is located in line 933 of this file https://github.com/libressl-portable/openbsd/blob/84ca37c024bbc9197512e41eaf0136ae0cefbc1a/src/lib/libcrypto/bn/bn_nist.c – hamari Nov 24 '16 at 12:49
  • `fn` is a function pointer, so you should cast it to `(void*)` before passing to `%p` – phuclv Nov 27 '16 at 01:44
  • 1
    All of your `printf` statements cause undefined behaviour by using the wrong format specifier. So the output is meaningless. – M.M Nov 27 '16 at 02:55

1 Answers1

0

after correcting all the 'slopiness' in the code, the result is:

#include <inttypes.h>
#include <stdio.h>

int fn1( void );
void fn2( void );
uintptr_t getmask( void );

int main( void )
{
  fn2();

  return 0;
}

uintptr_t getmask()
{
  return 0xffffffff;
}

int fn1() 
{
  return 42;
}

void fn2() 
{
  uintptr_t mask; 
  uintptr_t p;

  mask = getmask();
  p = (uintptr_t)fn1 & mask; 
  printf("mask: %08x\n", (unsigned int)mask);
  printf("fn1:  %p\n",   fn1); 
  printf("p:    %08x\n", (unsigned int)p);
}

and the output (on my linux 64bit computer) is:

mask: ffffffff
fn1:  0x4007c1
p:    004007c1
user3629249
  • 16,402
  • 1
  • 16
  • 17