0

I am trying to understand the behavior of printf in this example. of course the main issue here is that we are returning a pointer to a value on the stack that was popped after the function Boo returned.

I compiled with gcc. In test1: I got 7 printed twice which was expected. And a garbage value on the second printf in test2. but when I compiled with gcc -O3 I got 0 printed on both cases and a compiler warning about returning address of local variable.

test.c: In function ‘Foo’: test.c:8:12: warning: function returns address of local variable [-Wreturn-local-addr] return t; ^ test.c:5:9: note: declared here int j;

Can someone help me explain how does the behavior of printf that causes this behavior?

int *Boo(int i, int *p)
{
    int j;
    int *t = &j;
    *t = i + *p;
    return t;
}

void Foo(int x)
{
    if (x == 0) { return;}
    Foo(x - 1);
}

//test1
int main(void)
{
    int x = 5;
    int *t = Boo(2, &x);
    printf("%d", *t);
    printf("%d", *t);

    return 0;
}

//test2
int main(void)
{
    int x = 5;
    int *t = Boo(2, &x);
    
    printf("%d", *t);
    Foo(8);
    printf("%d", *t);

    return 0;
}
Noor Yag
  • 43
  • 7
  • 2
    The reason for the observed behaviour is not `printf()`... it's the programmer's fault who allowed UB – pmg Jan 20 '21 at 17:25
  • 2
    Because undefined behavior is undefined. So a compiler might do different weird things especially if compiling with different settings. – Eugene Sh. Jan 20 '21 at 17:25
  • *I got 7 printed twice which was expected.* It's expected **not** to print 7 on either occasion. If you did get 7 it's only because the location had not yet been re-used. – Weather Vane Jan 20 '21 at 17:39
  • thank you, [link](https://stackoverflow.com/questions/37087286/c-program-crashes-when-adding-an-extra-int/37087465#37087465) this does answer the first part, but I was just curious if there is an explanation to why specifically on optimization printf will print 0s only, I tried looking into assembly code but I could not deduce anything from that. – Noor Yag Jan 20 '21 at 17:44
  • You could single-step with a debugger to observe what is happening. – Weather Vane Jan 20 '21 at 17:51

1 Answers1

0

GCC sees that Boo returns a pointer to a local variable, and that therefore any attempt to use this pointer is undefined behavior. That means, according to the C standard, that the compiler can do whatever it wants, and in such cases GCC will often generate code that's "efficient" even if it is totally unrelated to what the programmer may have intended. It inlines the call to Boo, which has no visible effects and therefore is optimized away, and it picks an "efficient" way to provide an argument for printf, which happens to be the constant 0.

See on godbolt. The integer argument to printf goes in esi, which comes from ebp which is zeroed. I guess there's a missed optimization in that it saves the same 0 in ebp across the call to printf to reload into esi, instead of just re-zeroing esi afterward. But the whole analysis is kind of pointless since, again, the behavior is undefined.

Boo itself is optimized into a function that just returns a NULL pointer and does nothing else, which again is legal because of undefined behavior, but that version of Boo is not called by the program. Foo is also optimized into a function which returns immediately without recursing; the compiler can tell that the recursive calls have no effect. (And if a negative argument were passed to Foo, you'd have undefined behavior due to signed integer overflow, so the compiler need not handle that case.)

Nate Eldredge
  • 48,811
  • 6
  • 54
  • 82