10

I would like to know what type introspection I can do to detect types that assignable by simply raw memory copy?

For example, as far I understand, built-in types tuples of built-in types and tuple of such tuples, would fall in this category. The motivation is that I want to transport raw bytes if possible.

T t1(...); // not necessarely default constructible 
T t2(...);

t1 = t2; // should be equivalent to std::memcpy(&t1, &t2, sizeof(T));
// t1 is now an (independent) copy of the value of t2, for example each can go out of scope independently

What type_trait or combination of type_traits could tell at compile time if assignment can be (in principle) replaced by memcpy?

I tried what would work for the types I would guess should fullfil this condition and to my surprise the only one that fit the behavior is not std::is_trivially_assignable but std::trivially_destructible. It makes sense to some level, but I am confused why some other options do not work with the expected cases.

I understand that there may not be a bullet proof method because one can always write a class that effectively is memcopyable, that cannot be "detected" as memcopyable, but I am looking for one that works for the simple intuitive cases.

#include<type_traits>
template<class T> using trait = 
    std::is_trivially_destructible
//  std::is_trivial
//  std::is_trivially_copy_assignable
//  std::is_trivially_copyable // // std::tuple<double, double> is not trivially copyable!!!
//  std::is_trivially_default_constructible
//  std::is_trivially_default_constructible
//  std::is_trivially_constructible
//  std::is_pod // std::tuple<double, double> is not pod!!!
//  std::is_standard_layout
//  std::is_aggregate
//  std::has_unique_object_representations
    <T>
;

int main(){
    static_assert((trait<double>{}), "");
    static_assert((trait<std::tuple<double, double>>{}), "");
    static_assert((not trait<std::tuple<double, std::vector<double>>>{}), "");
    static_assert((not trait<std::vector<double>>{}), "");
}

Of course my conviction that tuple should be memcopyable is not based on the standard but based on common sense and practice. That is, because this is generally ok:

std::tuple<double, std::tuple<char, int> > t1 = {5.1, {'c', 8}};
std::tuple<double, std::tuple<char, int> > t2;
t2 = t1;
std::tuple<double, std::tuple<char, int> > t3;
std::memcpy(&t3, &t1, sizeof(t1));
assert(t3 == t2);

As a proof of principle, I implemented this. I added a couple of conditions related to the size to avoid some possible misleading specialization of std::tuple.

template<class T> 
struct is_memcopyable 
: std::integral_constant<bool, std::is_trivially_copyable<T>{}>{};

template<class T, class... Ts> 
struct is_memcopyable<std::tuple<T, Ts...>> : 
    std::integral_constant<bool, 
        is_memcopyable<T>{} and is_memcopyable<std::tuple<Ts...>>{}
    >
{};

template<class T1, class T2> 
struct is_memcopyable<std::pair<T1, T2>> : 
    std::integral_constant<bool, 
        is_memcopyable<T1>{} and is_memcopyable<T2>{}
    >
{};

This is a very limited workaround because a class like:

struct A{ std::tuple<double, double> t; }; 

will still unfortunately be reported as non trivially copyable and non memcopyable.

alfC
  • 14,261
  • 4
  • 67
  • 118
  • 1
    Not that this answers your question but it need not be just one single type_trait. Perhaps write your own that is a combination of traits. e.g. (is_pod::value || is_trivially_assignable::value) – Joe Mar 05 '18 at 22:06
  • @Joe, yes, I asked for a combination of traits as well. I couldn't find one, I tried some. To my surprise `std::tuple` is not a `pod` nor `trivially_assignable`. – alfC Mar 05 '18 at 22:17
  • 2
    In that case, what is the source of your belief that tuple should be trivially assignable? This answer seems to say not: https://stackoverflow.com/questions/38779985/why-cant-stdtupleint-be-trivially-copyable – Joe Mar 05 '18 at 22:26
  • @Joe Good point. Tuples of memcopyable stuff should be memcopyable, that is my starting point. Either tuple or the traits should be implemented in such a way that this can be detected. – alfC Mar 05 '18 at 22:45
  • I understand your starting point, I just don't understand what is the basis of it. Does the standard say that they should? – Joe Mar 05 '18 at 22:49
  • Yes, I am not basing this on the standard but in common sense, see the added code in my question. These answers are helping me understand that this is not required by the standard and I have to add some assumptions in order to exploit the non-explicit memcopyability of tuples. It is a pity none of the traits captures this common sense property of tuples. – alfC Mar 06 '18 at 03:07
  • This is a very interesting question - why no upvotes? – davidbak Mar 06 '18 at 03:20
  • 1
    You could create your own type trait with as many specializations for std or library or user defined types as you'd like, with a fallback to something from std type_traits that might be acceptible even though it would have false negatives. (A comment, because this isn't an answer.) – davidbak Mar 06 '18 at 03:59
  • @davidbak people upvote less these days. I have a bunch of interesting questions (I am biased) with zero or just one vote. – alfC Mar 06 '18 at 04:45
  • @Joe you beat me to posting that link... – Francis Cugler Mar 06 '18 at 05:59
  • @davidbak, I created a custom trait, added in the last edit of the question. – alfC Mar 06 '18 at 06:09
  • 1
    I would think that the problem is that assigment operator of `std::pair` is not defaulted... so maybe you should suggest a fix to the standard... Meaningwhile, I would try to not depend on UB even though it would probably works as expected. – Phil1970 Mar 07 '18 at 01:25
  • 1
    @Phil1970: Existing discussions show that the designers are already aware of this. The assignment operator can't be defaulted for all pairs without breaking CopyAssignable pair-of-reference types, which are useful and used somewhat widely. So it would need specialization-linked-defaulting and that has backward compatibility costs too (although it doesn't break existing code, it does break ABI on some implementations). So it looks like the discussion will continue to weigh pros and cons. – Ben Voigt Mar 07 '18 at 04:51
  • @BenVoigt, now I wonder why the default "copy assignment" wouldn't just do the operation "=" on member references. Is that an exception to a defaulted copy? (It wouldn't be a copy of course, but it would work transparently and recursively). – alfC Mar 07 '18 at 09:31
  • 1
    @alfC: Types with non-static reference members default the assignment operators to deleted. Pair assigns the target objects of its members, not the references themselves (since there is no such thing as reference re-assignment) but this behavior is non-default. – Ben Voigt Mar 07 '18 at 15:35
  • @BenVoigt right, this is perhaps where the problem starts. It seems that this was a well intentioned heroic but misleading attempt to put semantics over syntax but at the end it doesn't help much. – alfC Mar 07 '18 at 16:01
  • @alfC: Well, it's actually there to support the `std::tie` pattern for multiple return values. See https://stackoverflow.com/a/16516315/103167 – Ben Voigt Mar 07 '18 at 19:06
  • @BenVoigt, yes, I know, support for `std::tie` is good. What I am saying is that the implementation of `std::pair` has to fight against the language to force the application of `operator=` on the reference members. In other words, `std::pair` is complicated because the language put a restriction that in insight was not very good. – alfC Mar 07 '18 at 19:47
  • @alfC: I don't agree. It's not a restriction, it's a choice to not default to questionable and surprising behavior. The language allows you to go both ways, you just have to specify. IMO defaulting to deletion is far better than opting you in by default to a copy assignment that yields a substantially different result than copy construction. – Ben Voigt Mar 08 '18 at 16:13

3 Answers3

11

The correct test is in fact std::is_trivially_copyable, which allows use of memcpy for both making a new object and modifying an existing one.

Although you may be surprised that these return false for types where your intuition tells you that memcpy ought to be ok, they are not lying; the Standard indeed makes memcpy undefined behavior in these cases.


In the particular case of std::pair, we can get some insight into what goes wrong:

int main()
{
    typedef std::pair<double,double> P;
    std::cout << "\nTC:  " << std::is_trivially_copyable<P>::value;
    std::cout << "\nTCC: " << std::is_trivially_copy_constructible<P>::value;
    std::cout << "\nTCv: " << std::is_trivially_constructible<P, const P&>::value;
    std::cout << "\n CC: " << std::is_copy_constructible<P>::value;
    std::cout << "\n MC: " << std::is_move_constructible<P>::value;
    std::cout << "\nTCA: " << std::is_trivially_copy_assignable<P>::value;
    std::cout << "\nTCvA:" << std::is_trivially_assignable<P, const P&>::value;
    std::cout << "\n CA: " << std::is_copy_assignable<P>::value;
    std::cout << "\n MA: " << std::is_move_assignable<P>::value;
    std::cout << "\nTD:  " << std::is_trivially_destructible<P>::value;
}

TC: 0 TCC: 1 TCv: 1 CC: 1 MC: 1 TCA: 0 TCvA:0 CA: 1 MA: 1 TD: 1

Evidently it isn't trivially copy assignable.1

The pair assignment operator is user-defined, so not trivial.


1I think that clang, gcc, and msvc are all wrong here, actually, but if it satisfied std::_is_trivially_copy_assignable it wouldn't help, because TriviallyCopyable requires that the copy constructor, if not deleted, is trivial, and not the TriviallyCopyAssignable trait. Yeah, they're different.

A copy/move assignment operator for class X is trivial if it is not user-provided and...

vs

is_assignable_v<T, const T&> is true and the assignment, as defined by is_assignable, is known to call no operation that is not trivial.

The operations called by pair<double, double>'s copy assignment operator are the assignments of individual doubles, which are trivial.

Unfortunately, the definition of trivially copyable relies on the first, which pair fails.

A trivially copyable class is a class:

  • where each copy constructor, move constructor, copy assignment operator, and move assignment operator is either deleted or trivial,
  • that has at least one non-deleted copy constructor, move constructor, copy assignment operator, or move assignment operator, and
  • that has a trivial, non-deleted destructor.
Ben Voigt
  • 277,958
  • 43
  • 419
  • 720
  • I understand that for tuple the case is complicated, but for pair I don't see why the copy assignment is not defaulted. Probably because assignment is templated wrt the source types and generates among other things a non-defaulted copy assignment. Sfinae should solve that and `pair` can be TC as it should. – alfC Mar 07 '18 at 02:19
  • @alfC: `std::pair` has a user-defined copy assignment, which is already enough to break trivially copyability. – Ben Voigt Mar 07 '18 at 02:33
  • Yes, but why. There are sure ways to avoid having a user defined copy assignment. – alfC Mar 07 '18 at 02:42
  • 1
    @alfC: Related: https://groups.google.com/a/isocpp.org/forum/#!topic/std-discussion/PUZ9WUr2AOU – Ben Voigt Mar 07 '18 at 03:25
  • 1
    @alfC: Here's a discussion where it seems the only reason it's user-defined is for backward compatibility: https://groups.google.com/a/isocpp.org/forum/#!msg/std-proposals/Zi2wriFMFvI/N1JlOE-FBQAJ – Ben Voigt Mar 07 '18 at 03:26
  • Ok, I think it is a defect of the implementation after all. Also, as a side note I could use is_trivially_destructible instead. I have a hard time imagining a trivially destructible class that is not memcopyable. – alfC Mar 07 '18 at 03:58
  • @alfC: Undefined behavior goes beyond "I'll take the chance of overwriting my memory"... if the compiler detects that your function causes undefined behavior, your code will no longer be optimization-stable. – Ben Voigt Mar 07 '18 at 04:48
  • Sure, I just wanted to say that trivially destructible apparently was harder to break that trivially copyable. I think memcpy IS the optimization here. (btw, memcpy is just an example what I really want is to use binary streaming when possible). – alfC Mar 07 '18 at 06:09
  • Arguably the standard requires user-defined assignment operators for `tuple` in [tuple.tuple] and the same for `pair` in [pair.pair] because they are not defaulted at the site of the declaration even conditionally. Keep in mind that a method that is declared without definition, and defaulted later is considered user-defined (after all, the `default` could not be visible from all TUs). It would make sense to change it if all elements are TC and not references, and it shouldn't hurt any sane legacy code IMO, but at the moment it will be hard to argue that this is an implementation defect. – Arne Vogel Mar 07 '18 at 12:45
  • @Arne: Agreed, the operator is required to be user-defined by the current text of the Standard. The implementation bug I see is in the implementation of the type trait `is_trivially_assignable`, because it doesn't reflect the rule given in the Standard (which is unfortunately different from the rule for a trivial assignment operator) – Ben Voigt Mar 07 '18 at 15:34
  • @BenVoigt, why do you mention `is_trivially_assignable`? what would be the bug in the trait? isn't also the trait implemented as a compiler hook? Do you mean that it is a bug in the compiler? – alfC Mar 08 '18 at 22:17
  • BTW, there is a case where everything works well: `std::complex`, `static_assert(std::is_trivially_copyable>{});` – alfC Mar 08 '18 at 22:17
  • 1
    @alfC: Yes, `std::complex` is trivially copyable (if the element type is), so is `std::array`. They don't have the extra-special sauce that supports `tie` usage. – Ben Voigt Mar 08 '18 at 23:12
4

This is only a partial answer to your question:

Type traits don't necessarily mean what their name says literally.

Specifically, let's take std::is_trivially_copyable. You were - rightly - surprised that a tuple of two double's is not trivially copyable. How could that be?!

Well, the trait definition says:

If T is a TriviallyCopyable type, provides the member constant value equal true. For any other type, value is false.

and the TriviallyCopyable concept has the following requirement in its definition:

  • Every copy constructor is trivial or deleted
  • Every move constructor is trivial or deleted
  • Every copy assignment operator is trivial or deleted
  • Every move assignment operator is trivial or deleted
  • At least one copy constructor, move constructor, copy assignment operator, or move assignment operator is non-deleted
  • Trivial non-deleted destructor

Not quite what you would expect, right?

With all in mind, it's not necessarily the case that any of the standard library traits would combine to fit the exact requirements of "constructible by memcpy()'ing".

einpoklum
  • 118,144
  • 57
  • 340
  • 684
  • 1
    Got it, trivial means that it does nothing not that is does something simple. I guess the only hope is is_pod and/or is_agregate. Which is unlikely to work. I guess I will implement recursively some custom trait based on the conviction that tuples of memcopyable stuff should be memcopyable in it self. – alfC Mar 05 '18 at 22:41
  • 1
    @alfC: I think `is_pod` [is going away in C++20](http://en.cppreference.com/w/cpp/types/is_pod). – einpoklum Mar 05 '18 at 23:13
  • Yes, https://stackoverflow.com/questions/48225673/why-is-stdis-pod-deprecated-in-c20. It is not clear with what it is replaced with. I hope the change derives in some kind of more powerful trait that includes the near podness of tuples. – alfC Mar 06 '18 at 03:10
  • I am confused now, your like actually says "Objects of trivially-copyable types are the only C++ objects that may be safely copied with std::memcpy". I guess the problem is that it doesn't work the other way around; there are obvious types that can be memcpy'd and but `std::is_trivially_copyable` is false. – alfC Mar 06 '18 at 03:16
  • @alfC: I think the words there may need a bit of work; probably they mean trivially-copyable in the colloquial sense, not in "standardese". – einpoklum Mar 06 '18 at 08:04
  • 2
    @einpoklum: Nope, `TriviallyCopyable` is the condition the Standard puts on use of a type with `memcpy`. The counter-intuitive thing is that `memcpy` can't be used with that pair, not any misbehavior in `std::is_trivially_copyable`. – Ben Voigt Mar 07 '18 at 01:06
  • @BenVoigt: In what sense can memcpy not be used with that pair? Also, in general, what you're saying can't be true, since one can artificially break `TriviallyCopyable` without actually changing anything about a class. – einpoklum Mar 07 '18 at 01:10
  • 1
    @einpoklum: In the sense that the Standard doesn't define the behavior of using `memcpy` on any type that isn't TriviallyCopyable. "For any trivially copyable type T, if two pointers to T point to distinct T objects obj1 and obj2, where neither obj1 nor obj2 is a base-class subobject, if the underlying bytes making up obj1 are copied into obj2, obj2 shall subsequently hold the same value as obj1." – Ben Voigt Mar 07 '18 at 01:16
  • and "For any object (other than a base-class subobject) of trivially copyable type T, whether or not the object holds a valid value of type T, the underlying bytes making up the object can be copied into an array of char, unsigned char, or std::byte. If the content of that array is copied back into the object, the object shall subsequently hold its original value." There's nothing else that defines the behavior of making a bytewise copy, and footnotes on those two rules call out `memcpy` as an example of this copy. – Ben Voigt Mar 07 '18 at 01:19
  • 1
    Also, I'm not sure which test you think `std::pair` should fail. TriviallyCopyable only restricts copy and move constructors, the "obscenely rich set" you mention are totally irrelevant. – Ben Voigt Mar 07 '18 at 01:26
  • @BenVoigt: I thought OP reported it fails. Anyway, will edit. About the non-definition in the standard - that's a bit puzzling, since if I take a `TriviallyCopyable` class, changed the default copy ctor into a custom copy ctor which does the same thing - you're saying the result of mencoy()ubg this undefibed – einpoklum Mar 07 '18 at 12:34
  • @einpoklum: It does fail, due to a user-defined assignment operator. So I'm confused why you are blaming the constructor. And yes, the Standard treats a user-provided constructor quite differently from a defaulted constructor that does the exact same thing. The guarantee that memcpy will work does not apply if any of the special member functions related to copying -- copy and move constructors, copy and move assignment, and destruction -- are user-defined (as a specific case of "non-trivial"). – Ben Voigt Mar 07 '18 at 15:31
4

To try and answer your question: std::memcpy() does not have any direct requirements but it does have these stipulations:

  • Copies count bytes from the object pointed to by src to the object pointed to by dest. Both objects are reinterpreted as arrays of unsigned char.
  • If the objects overlap, the behavior is undefined.
  • If either dest or src is a null pointer, the behavior is undefined, even if count is zero.
  • If the objects are not TriviallyCopyable, the behavior of memcpy is not specified and may be undefined.

Now to have the qualifications that an object is Trivially Copyable the following conditions or requirements must be met:

  • Every copy constructor is trivial or deleted
  • Every move constructor is trivial or deleted
  • Every copy assignment operator is trivial or deleted
  • Every move assignment operator is trivial or deleted
  • at least one copy constructor, move constructor, copy assignment operator, or move assignment operator is non-deleted
  • Trivial non-deleted destructor

This implies that the class has no virtual functions or virtual base classes.

Scalar types and arrays of TriviallyCopyable objects are TriviallyCopyable as well, as well as the const-qualified (but not volatile-qualified) versions of such types.


Which leads us to std::is_trivially_copyable

If T is a TriviallyCopyable type, provides the member constant value equal true. For any other type, value is false.

The only trivially copyable types are scalar types, trivially copyable classes, and arrays of such types/classes (possibly const-qualified, but not volatile-qualified).

The behavior is undefined if std::remove_all_extents_t is an incomplete type and not (possibly cv-qualified) void.

with this nice feature since c++17:

Helper variable template

template< class T >
inline constexpr bool is_trivially_copyable_v = is_trivially_copyable<T>::value; 

And you would like to try and use a type_trait to use std::tuple<> with std::memcpy().

But we need to ask ourselves if std::tuple is Trivially Copyable and why?

We can see the answer to that here: Stack-Q/A: std::tuple Trivially Copyable? and according to that answer; it is not because the standard does not require the copy/move assignment operators to be trivial.


So the answer that I would think that is valid would be this: No std::tuple is not Trivially Copyable but std::memcpy() doesn't require it to be but only states that if it isn't; it is UB. So can you use std::tuple with std::memcpy? I think so, but is it safe? That can vary and can produce UB.

So what can we do from here? Take a risk? Maybe. I found something else that is related but have not found anything out about it regarding if it is Trivially Copyable. It is not a type_trait, but it is something that might be able to be used in conjunction with std::tuple & std::memcpy and that is std::tuple_element. You might be able to use this to do the memcpy, but I'm not fully sure about it. I have searched to find out more about std::tuple_element to see if it is Trivially Copyable but haven't found much so all I can do is a test to see what Visual Studio 2017 says:

template<class... Args>
struct type_list {
    template<std::size_t N>
    using type = typename std::tuple_element<N, std::tuple<Args...>>::type;
};

int main() {
    std::cout << std::boolalpha;
    std::cout << std::is_trivially_copyable<type_list<int, float, float>>::value << '\n';
    std::cout << std::is_trivially_copyable<std::tuple<int, float, float>>::value << '\n';

    _getch(); // used to stop visual studio debugger from closing.
    return 0;
}

Output:

true
false

So it appears if we wrap std::tuple_element in a struct it is Trivially Copyable. Now the question is how do you integrate this with your std::tuple data sets to use them with std::memcpy() to be type safe. Not sure if we can since std::tuple_element will return the types of the elements within a tuple.

If we even tried to wrap a tuple in a struct as such:

template<class... Args>
struct wrapper {
    std::tuple<Args...> t;
};

And we can check it by:

{
    std::cout << std::is_trivially_copyable< wrapper<int, float, float> >::value << std::endl;
}

It is still false. However we have seen were std::tuple was already used in the first struct and the struct returned true. This may be of some help to you, to ensure you can safely use std::memcpy, but I can not guarantee it. It is just that the compiler seems to agree with it. So this might be the closest thing to a type_trait that might work.


NOTE: - All the references about memcpy, Trivially Copyable concepts, is_trivially_copyable, std::tuple & std::tuple_element were taken from cppreference and their relevant pages.

Inobelar
  • 59
  • 8
Francis Cugler
  • 7,788
  • 2
  • 28
  • 59
  • 3
    I don't get it. Tuple_element is used to get the type of an index , it doesn't hold any value, that is why is trivial. – alfC Mar 06 '18 at 07:34
  • @alfC yeah I'm not familiar with it. I didn't know if it could be used. – Francis Cugler Mar 06 '18 at 19:44
  • @alfC I think I worded the end of my answer wrong; as the OP was looking for a `type_trait` and this might help with that part of his question. – Francis Cugler Mar 06 '18 at 20:42