10

As pointed out in an answer to this question, the compiler (in this case gcc-4.1.2, yes it's old, no I can't change it) can replace struct assignments with memcpy where it thinks it is appropriate.

I'm running some code under valgrind and got a warning about memcpy source/destination overlap. When I look at the code, I see this (paraphrasing):

struct outer
{
    struct inner i;
    // lots of other stuff
};

struct inner
{
    int x;
    // lots of other stuff
};

void frob(struct inner* i, struct outer* o)
{
    o->i = *i;
}

int main()
{
    struct outer o;

    // assign a bunch of fields in o->i...

    frob(&o.i, o);
    return 0;
}

If gcc decides to replace that assignment with memcpy, then it's an invalid call because the source and dest overlap.

Obviously, if I change the assignment statement in frob to call memmove instead, then the problem goes away.

But is this a compiler bug, or is that assignment statement somehow invalid?

Community
  • 1
  • 1
bstpierre
  • 30,042
  • 15
  • 70
  • 103
  • 5
    I'm pretty sure you meant to write frob(..., **&o**), and not (..., **0**). – Andy Finkenstadt Mar 23 '11 at 22:01
  • It's a compiler bug. The entire gcc 4.x series is full of crap like this, including breaking `LIST_ENTRY` / `LIST_HEAD` type punning which predates gcc 1.0. Join the gcc-4 resistance! – Heath Hunnicutt Mar 23 '11 at 22:02
  • @Andy - My fat fingers. Thanks. – bstpierre Mar 24 '11 at 02:46
  • 2
    @Heath: The *entire* 4.x series? Are you seriously suggesting that we move back to 3.x? Or did you invent a time machine and you have access to the awesome futuristic 5.x series? – Adam Rosenfield Mar 24 '11 at 03:32
  • Accepted Jens' answer as I think it most clearly defines the issue. R.. is probably right in thinking that this is a gcc bug (at least a lurking bug) but in my particular combination it isn't a problem. At the very least I changed it to memmove to get valgrind to shut up. Thanks for the answers. – bstpierre Mar 25 '11 at 12:55
  • Still there in gcc 8: https://godbolt.org/z/v5f9vE – not-a-user Jul 17 '20 at 15:23

3 Answers3

4

As far as I can tell, this is a compiler bug. i is allowed to alias &o.i according to the aliasing rules, since the types match and the compiler cannot prove that the address of o.i could not have been previously taken. And of course calling memcpy with overlapping (or same) pointers invokes UB.

By the way note that, in your example, o->i is nonsense. You meant o.i I think...

R.. GitHub STOP HELPING ICE
  • 208,859
  • 35
  • 376
  • 711
  • "invokes UB" - well, it would if a program did it. gcc might conceivably be relying on some guarantee by glibc that defines the result. Maybe I misremember, but I think I've seen kernel comments before to the effect that an aligned `memcpy` is guaranteed to be a forward copy, or something. Or maybe comments in an implementation of `memmove`? Might not have been GNU at all, though. – Steve Jessop Mar 23 '11 at 22:12
  • glibc is part of my C implementation, though. So when you checked, I hope you checked *really closely* how gcc is documented to behave with C library implementations whose documented behavior differs from the documented behavior of glibc, because in the event that there is some relevant guarantee, it becomes quite important. – Steve Jessop Mar 23 '11 at 22:41
  • 2
    Nonetheless, the compiler itself can't invoke "undefined behaviour". It's up to the combination of the compiler and standard library to correctly implement the standard. If you used gcc with a C library where the `memcpy()` caused a problem in this instance, you'd have a nonconforming implementation - whether that was the fault of gcc or the library is a matter for the gcc and library authors to argue out between them. – caf Mar 23 '11 at 22:46
  • @SteveJessop How can the compiler know how the linker will resolve memcpy? I might swap in my own `memcpy` (or complete libc) that conforms to the C standard and explodes on overlap. – not-a-user Jul 17 '20 at 15:29
  • @not-a-user: two separate things. Firstly, the standard permits the compiler to call `memcpy`. That's all well and good, but the compiler isn't necessarily using that permission. Instead, the "as-if" rule permits the implementation to do anything it pleases, provided the result is correct. If glibc guarantees that the result is correct then the compiler can use glibc. And if the compiler requires that any stdlib you link its code against, must provide the same beyond-the-standard guarantee that glibc does, then it's you who has broken it by linking. Which is why I said to check the manual. – Steve Jessop Aug 12 '20 at 00:43
  • In short, conforming to the C standard might not be sufficient to be good enough to link gcc against your library. Maybe it is, but if the compiler docs add further requirements then you have to follow them or suffer the consequence. For example, just at a stupid level, the ABI isn't defined by the C standard, so you have to follow whatever ABI the linker tells you to and gcc is entitled to assume you've done that. I don't know whether gcc really says, "you must implement a glibc extension to the standard" for any standard functions, but if it does then you have to. – Steve Jessop Aug 12 '20 at 00:46
  • The reason for this is that the standard (well, as of 2011, I haven't kept up recently) doesn't distinguish between the "compiler" and the "standard library" except that it allows free-standing implementations with a minimal library. Any distinction between the two, or the interface for compiler-generated calls to standard functions (as opposed to calls in user's code) are implementation-specific details. Like caf says if two parts of the implementation blame each other for the bug then that's for them to sort out. The standard leaves them to it. – Steve Jessop Aug 12 '20 at 00:51
4

I think that you are mixing up the levels. gcc is perfectly correct to replace an assignment operation by a call to any library function of its liking, as long as it can guarantee the correct behavior.

It is not "calling" memcpy or whatsoever in the sense of the standard. It is just using one function it its library for which it might have additional information that guarantees correctness. The properties of memcpy as they are described in the standard are properties seen as interfaces for the programmer, not for the compiler/environment implementor.

Whether or not memcpy in that implementation in question implements a behavior that makes it valid for the assignment operation is another question. It should not be so difficult to check that or even to inspect the code.

Jens Gustedt
  • 76,821
  • 6
  • 102
  • 177
  • 1
    Right. The bug, if anywhere, is in valgrind - *you're* not allowed to use a `memcpy()` there, but the implementation is, since it's the one that provides `memcpy()`. – caf Mar 23 '11 at 22:44
  • So you're essentially saying that, in this case gcc *knows* that the particular implementation of `memcpy` it is using is overlap-safe? – bstpierre Mar 24 '11 at 02:54
  • @bstpierre, no, I am saying that it *should* know, and that using it is in itself not a violation of the standard. Whether or not it is effectively overlap safe (or their use in an assignment operator) should be subject to tests. – Jens Gustedt Mar 24 '11 at 06:56
  • In practice, any sane implementation of `memcpy` will be fine if the objects overlap exactly. It's only when they overlap partially that special consideration is needed. This is probably gcc's justification for assuming it just works. – R.. GitHub STOP HELPING ICE Jun 23 '13 at 03:46
  • But the `memcpy` symbol comes from libc, say glibc. The compiler cannot assume any particular behavior for overlapping arguments? – not-a-user Jul 17 '20 at 15:24
1

I suppose that there is a typo: "&o" instead of "0". Under this hypothesis, the "overlap" is actually a strict overwrite: memcpy(&o->i,&o->i,sizeof(o->i)). In this particular case memcpy behaves correctly.

Giuseppe Guerrini
  • 4,274
  • 17
  • 32