"constructing" a trivially-copyable object with memcpy

Question

In C++, is this code correct?

#include <cstdlib>
#include <cstring>

struct T   // trivially copyable type
{
    int x, y;
};

int main()
{
    void *buf = std::malloc( sizeof(T) );
    if ( !buf ) return 0;

    T a{};
    std::memcpy(buf, &a, sizeof a);
    T *b = static_cast<T *>(buf);

    b->x = b->y;

    free(buf);
}

In other words, is *b an object whose lifetime has begun? (If so, when did it begin exactly?)

Answer 1

This is unspecified which is supported by N3751: Object Lifetime, Low-level Programming, and memcpy which says amongst other things:

The C++ standards is currently silent on whether the use of memcpy to copy object representation bytes is conceptually an assignment or an object construction. The difference does matter for semantics-based program analysis and transformation tools, as well as optimizers, tracking object lifetime. This paper suggests that

1. uses of memcpy to copy the bytes of two distinct objects of two different trivial copyable tables (but otherwise of the same size) be allowed

2. such uses are recognized as initialization, or more generally as (conceptually) object construction.

Recognition as object construction will support binary IO, while still permitting lifetime-based analyses and optimizers.

I can not find any meeting minutes that has this paper discussed, so it seems like it is still an open issue.

The C++14 draft standard currently says in 1.8 [intro.object]:

[...]An object is created by a definition (3.1), by a new-expression (5.3.4) or by the implementation (12.2) when needed.[...]

which we don't have with the malloc and the cases covered in the standard for copying trivial copyable types seem to only refer to already existing objects in section 3.9 [basic.types]:

For any object (other than a base-class subobject) of trivially copyable type T, whether or not the object holds a valid value of type T, the underlying bytes (1.7) making up the object can be copied into an array of char or unsigned char.42 If the content of the array of char or unsigned char is copied back into the object, the object shall subsequently hold its original value[...]

and:

For any trivially copyable type T, if two pointers to T point to distinct T objects obj1 and obj2, where neither obj1 nor obj2 is a base-class subobject, if the underlying bytes (1.7) making up obj1 are copied into obj2,43 obj2 shall subsequently hold the same value as obj1.[...]

which is basically what the proposal says, so that should not be surprising.

dyp points out a fascinating discussion on this topic from the ub mailing list: [ub] Type punning to avoid copying.

Propoal p0593: Implicit creation of objects for low-level object manipulation

The proposal p0593 attempts to solve this issues but AFAIK has not been reviewed yet.

This paper proposes that objects of sufficiently trivial types be created on-demand as necessary within newly-allocated storage to give programs defined behavior.

It has some motivating examples which are similar in nature including a current std::vector implementation which currently has undefined behavior.

It proposes the following ways to implicitly create an object:

We propose that at minimum the following operations be specified as implicitly creating objects:

• Creation of an array of char, unsigned char, or std::byte implicitly creates objects within that array.

• A call to malloc, calloc, realloc, or any function named operator new or operator new[] implicitly creates objects in its returned storage.

• std::allocator::allocate likewise implicitly creates objects in its returned storage; the allocator requirements should require other allocator implementations to do the same.

• A call to memmove behaves as if it

  • copies the source storage to a temporary area

  • implicitly creates objects in the destination storage, and then

  • copies the temporary storage to the destination storage.

  This permits memmove to preserve the types of trivially-copyable objects, or to be used to reinterpret a byte representation of one object as that of another object.

• A call to memcpy behaves the same as a call to memmove except that it introduces an overlap restriction between the source and destination.

• A class member access that nominates a union member triggers implicit object creation within the storage occupied by the union member. Note that this is not an entirely new rule: this permission already existed in [P0137R1] for cases where the member access is on the left side of an assignment, but is now generalized as part of this new framework. As explained below, this does not permit type punning through unions; rather, it merely permits the active union member to be changed by a class member access expression.

• A new barrier operation (distinct from std::launder, which does not create objects) should be introduced to the standard library, with semantics equivalent to a memmove with the same source and destination storage. As a strawman, we suggest:

  // Requires: [start, (char*)start + length) denotes a region of allocated
  // storage that is a subset of the region of storage reachable through start.
  // Effects: implicitly creates objects within the denoted region.
  void std::bless(void *start, size_t length);
  

In addition to the above, an implementation-defined set of non-stasndard memory allocation and mapping functions, such as mmap on POSIX systems and VirtualAlloc on Windows systems, should be specified as implicitly creating objects.

Note that a pointer reinterpret_cast is not considered sufficient to trigger implicit object creation.

Answer 2

The code is legal now, and retroactively since C++98!

The answer by @Shafik Yaghmour is thorough and relates to the code validity as an open issue - which was the case when answered. Shafik's answer correctly refer to p0593 which at the time of the answer was a proposal. But since then, the proposal was accepted and things got defined.

Some History

The possibility of creating an object using malloc was not mentioned in the C++ specification before C++20, see for example C++17 spec [intro.object]:

The constructs in a C++ program create, destroy, refer to, access, and manipulate objects. An object is created by a definition (6.1), by a new-expression (8.5.2.4), when implicitly changing the active member of a union (12.3), or when a temporary object is created (7.4, 15.2).

Above wording does not refer to malloc as an option for creating an object, thus making it a de-facto undefined behavior.

It was then viewed as a problem, and this issue was addressed later by https://wg21.link/P0593R6 and accepted as a DR against all C++ versions since C++98 inclusive, then added into the C++20 spec, with the new wording:

[intro.object]

1. The constructs in a C++ program create, destroy, refer to, access, and manipulate objects. An object is created by a definition, by a new-expression, by an operation that implicitly creates objects (see below)...

...

11. Further, after implicitly creating objects within a specified region of storage, some operations are described as producing a pointer to a suitable created object. These operations select one of the implicitly-created objects whose address is the address of the start of the region of storage, and produce a pointer value that points to that object, if that value would result in the program having defined behavior. If no such pointer value would give the program defined behavior, the behavior of the program is undefined. If multiple such pointer values would give the program defined behavior, it is unspecified which such pointer value is produced.

The example given in C++20 spec is:

#include <cstdlib>
struct X { int a, b; };
X *make_x() {
   // The call to std​::​malloc implicitly creates an object of type X
   // and its subobjects a and b, and returns a pointer to that X object
   // (or an object that is pointer-interconvertible ([basic.compound]) with it), 
   // in order to give the subsequent class member access operations   
   // defined behavior. 
   X *p = (X*)std::malloc(sizeof(struct X));
   p->a = 1;   
   p->b = 2;
   return p;
}

---

As for the use of memcpy - @Shafik Yaghmour already addresses that, this part is valid for trivially copyable types (the wording changed from POD in C++98 and C++03 to trivially copyable types in C++11 and after).

---

Bottom line: the code is valid.

As for the question of lifetime, let's dig into the code in question:

struct T   // trivially copyable type
{
    int x, y;
};

int main()
{
    void *buf = std::malloc( sizeof(T) ); // <= just an allocation
    if ( !buf ) return 0;

    T a{}; // <= here an object is born of course
    std::memcpy(buf, &a, sizeof a);      // <= just a copy of bytes
    T *b = static_cast<T *>(buf);        // <= here an object is "born"
                                         //    without constructor    
    b->x = b->y;

    free(buf);
} 

Note that one may add a call to the destructor of *b, for the sake of completeness, before freeing buf:

b->~T();
free(buf);

though this is not required by the spec.

Alternatively, deleting b is also an option:

delete b;
// instead of:
// free(buf);

But as said, the code is valid as is.

Answer 3

From a quick search.

"... lifetime begins when the properly-aligned storage for the object is allocated and ends when the storage is deallocated or reused by another object."

So, I would say by this definition, the lifetime begins with the allocation and ends with the free.

Answer 4

Is this code correct?

Well, it will usually "work", but only for trivial types.

I know you did not ask for it, but lets use an example with a non-trivial type:

#include <cstdlib>
#include <cstring>
#include <string>

struct T   // trivially copyable type
{
    std::string x, y;
};

int main()
{
    void *buf = std::malloc( sizeof(T) );
    if ( !buf ) return 0;

    T a{};
    a.x = "test";

    std::memcpy(buf, &a, sizeof a);    
    T *b = static_cast<T *>(buf);

    b->x = b->y;

    free(buf);
}

After constructing a, a.x is assigned a value. Let's assume that std::string is not optimized to use a local buffer for small string values, just a data pointer to an external memory block. The memcpy() copies the internal data of a as-is into buf. Now a.x and b->x refer to the same memory address for the string data. When b->x is assigned a new value, that memory block is freed, but a.x still refers to it. When a then goes out of scope at the end of main(), it tries to free the same memory block again. Undefined behavior occurs.

If you want to be "correct", the right way to construct an object into an existing memory block is to use the placement-new operator instead, eg:

#include <cstdlib>
#include <cstring>

struct T   // does not have to be trivially copyable
{
    // any members
};

int main()
{
    void *buf = std::malloc( sizeof(T) );
    if ( !buf ) return 0;

    T *b = new(buf) T; // <- placement-new
    // calls the T() constructor, which in turn calls
    // all member constructors...

    // b is a valid self-contained object,
    // use as needed...

    b->~T(); // <-- no placement-delete, must call the destructor explicitly
    free(buf);
}