std::realloc with static_cast or placement new

Question

According to reinterpret_cast creating a trivially default-constructible object objects with trivial default constructors cannot be created by simply reinterpreting suitably aligned storage [] placement-new is required. This should apply equally to std::realloc. Is the static_cast in the cppreference example:

#include <cstdlib>
class MallocDynamicBuffer
{
    char* p;
public:
    void resize(std::size_t newSize) {
        if(void* mem = std::realloc(p, newSize)) {
            p = static_cast<char*>(mem);
        }
    }
};

incorrect, because it doesn't create a valid object?

Would it be correct to replace the line containing static_cast with p = new(mem) T[newSize], with T=char or any other trivially copyable and trivially constructible type? I'm unsure, because this would require that it valid to call placement new on data which was previously constructed and moved. And that calling the constructor of a trivially constructible type guarantees to leave the value unchanged and not indeterminate.

I copied a bit more of the cppreference example (which is linked). I hope this clarifies it. — Roland Schulz, Jul 15 '18 at 00:07
Because `char` is special, the sample code works by the standard. In practice, this will work for any [trivially copyable](https://en.cppreference.com/w/cpp/named_req/TriviallyCopyable) type, but technically I think it's UB for any non-`char` non-`std::byte` type. This is one of the very few cases where I'm inclined to say "It's technically UB but it works anyway"; I don't think the optimizer is yet smart enough to realize that `realloc` doesn't create a C++ object with placement `new` before copying the bytes, and I expect that the standard will be fixed before that happens. — Daniel H, Jul 15 '18 at 00:18
@NeilButterworth No, this question is perfectly clear without compilable code. It's asking if there is *any* standards-compliant use of `std::realloc`. The answer is "Yes", but I think it's technically "Yes, but only for `char`." — Daniel H, Jul 15 '18 at 00:20
The llvm optimizes the placement-new away. GCC does it too if the size is of type int and size<0 is checked or exceptions are disabled. See https://godbolt.org/g/XXKYBD. But is it guaranteed that that the placement new is valid? Is it guaranteed to not change the value and is it allowed to be called again on the moved value? — Roland Schulz, Jul 15 '18 at 00:41
@RolandSchulz If the type is trivially destructable, you're allowed to call placement `new` without destruction first. If the type is [trivially default constructible](https://en.cppreference.com/w/cpp/language/default_constructor#Trivial_default_constructor), the default constructor "takes no action". If the type is trivially copyable, then the actual copying of `realloc` is allowed. But I'm not sure if there's some formal reason to force the construction to happen before the byte copying instead of after. — Daniel H, Jul 15 '18 at 00:55
That is, with the right requirements on the type, each *individual piece* of calling `std::realloc` followed by placement `new` is allowed (at least if `newSize` is 1; I think placement `new` with array types might have further complications), but I'm not sure the entire operation works in that order. — Daniel H, Jul 15 '18 at 00:57
Why would it matter that the construction happens before the copying? Before the copying the objects have been constructed by the previous realloc (otherwise the ptr is null and no copying happens). Is it guaranteed that the default constructor takes no action? Isn't it performing default initialization? And doesn't default initialization result in indeterminate values for fundamental types? PS: if you put it as an answer I'll accept it. — Roland Schulz, Jul 15 '18 at 02:22
Default initialization means ["no initialization is performed"](https://timsong-cpp.github.io/cppwp/n4659/dcl.init#7.3) for non-class non-array types, and recursion bottoms out at a bunch of instances of that for all trivial types. However, it's also possible that leaves it at indeterminate value even though a value was copied in. I [created a question](https://stackoverflow.com/q/51344835/27302) to ask about that particular case, in more concrete terms and avoiding the other complications of `realloc`. Hopefully somebody will answer that or will find come here via that one and answer this. — Daniel H, Jul 15 '18 at 02:49
@DanielH The question may or may not be 'perfectly clear', but pulling people up for posting code that doesn't compile is (of course) Doing The Right Thing (and they should fix it). — Paul Sanders, Jul 15 '18 at 06:27
@PaulSanders The first version didn't post any code, which it probably should have (you shouldn't make people follow links). The second version posted a code fragment which showed what it needs to but, since it was a fragment, woudn't compile on its own. Since it wasn't asking for debugging help but instead used for illustrative purposes I think that's allowed. — Daniel H, Jul 15 '18 at 22:45
@Daniel Anything's allowed, but this question is still going to get closed. Trawling back through the edits, I think what Neil was actually getting at was that the OP didn't actually post anything _useful_ in response to his initial comment so he was was trying (again) to get him to do so. And he didn't have to do that, he's just that kind of guy. — Paul Sanders, Jul 16 '18 at 03:06

Daniel H · Accepted Answer · 2018-07-16T01:11:38.137

1

Practical answer: If realloc isn't allowed for trivially copyable types (that is, all copy and move constructors and assignment operators are trivial, there is at least one copy or move constructor or assignment operator, and the destructor is trivial), this seems like a bug in the standard I can't imagine a compiler writer causing issues here.

Language lawyer answer: I don't think this is allowed by the standard for any type other than char, unsigned char, and std::byte, and I'm no longer as certain about those as I was when I commented to the question (it would be fine if there were some underlying object created by realloc, but there isn't). These functions are specified to "have the semantics specified in the C standard library", which isn't particularly helpful because C doesn't have a semantic equivalent to the C++ concept of "object". The general understanding of malloc is that it does not create a C++ object, so you need to do that yourself with placement new; I would assume the same happens for realloc.

Your proposed solution of using placement new[] on the result of realloc would also not work. Because new[] needs to keep track of the number of elements allocated (for use by delete[]), it is allowed to add some overhead space, and then return a pointer later than the one it received so the first part is the overhead. Although this isn't necessary with placement new[], it is still allowed (which makes me think there is no valid way to use placement new[] at all because you don't know how much overhead will be requested, so can't be sure you have enough space for it).

Even if you ensured you only used non-array placement new (I can think of a couple workarounds where realloc still makes sense), it would indeed create a valid object there (for types with trivial default constructors, which are at least allowed to do no work when default-initialized), but it might have indeterminate value. Right now it seems to me that it probably works but the standard is somewhat ambiguous, but that is debatable and I may be wrong.

edited Jul 16 '18 at 01:11

answered Jul 15 '18 at 04:36

Daniel H

7,223
2
26
41

It's worth noting that placement `new[]` is needed here, to create a char array. And `new[]` can have a space overhead. So, this cannot be done in a strictly standard conformant way. – geza Jul 15 '18 at 06:25
@geza I got so carried away looking at at what I mentioned in the last paragraph that I forgot that to figure out how placement `new[]` differs from placement `new` in that regard. I'll look into that later tonight and update the answer. – Daniel H Jul 15 '18 at 22:43
@geza Is placement `new[]` *ever* legal (using the built-in placement functions, not your own ones)? You can't know what the overhead is so you can't know that the passed-in pointer has enough space (unless, I guess, you had a pointer to `SIZE_MAX` bytes of memory). – Daniel H Jul 15 '18 at 23:17
It's legal, but it might not do what you want. The standard is broken in this regard, IMO. There's no _need_ for any overhead, obviously, so why mandate that compilers are allowed to add one? It's just daft, I might raise a bug on github. – Paul Sanders Jul 16 '18 at 03:10
@PaulSanders Given that the normative text doesn't stop it *and* the example explicitly says it's possible, I'm not sure it qualifies as editorial so GitHub might not be the best place. Although apparently "Unclear writing, when the intention of the text in question is well-known to the committee, but the presentation could be clearer." is editorial, so the whole problem with `realloc` might count. – Daniel H Jul 16 '18 at 04:16
@Daniel I was thinking of raising it as an [issue](https://github.com/cplusplus/draft/issues). – Paul Sanders Jul 16 '18 at 04:19
@PaulSanders I thought even those were supposed to be for just editorial and anything else [should go on `std-discussion`](https://isocpp.org/std/submit-issue) first. – Daniel H Jul 16 '18 at 04:23
1

@Daniel Hmm, I guess so, but, rightly or wrongly, I did this: https://github.com/cplusplus/draft/issues/2264. I guess we'll see, I don't really care if they shoot me down. – Paul Sanders Jul 16 '18 at 04:33
@DanielH: according to the standard, `new[]` is pretty useless. I cannot imagine any scenario, where the "I can put as much as overhead I want" behavior is OK. Note, that this applies to own implementations of placement new as well, there is no difference in this regard. `new[]` can be useful anyway, because compilers usually don't add overhead for trivial types. – geza Jul 16 '18 at 06:57
@geza Non-placement `new[]` doesn't specify how much overhead it wants, but I imagine it's almost always `sizeof(std::size_t)` or the requested alignment, whichever is greater, for types which aren't trivially destructible. – Daniel H Jul 16 '18 at 07:00
@DanielH: placement and non-placement `new[]` don't differ in this regard. Both can use any overhead they want. – geza Jul 16 '18 at 07:03
@geza And `sizeof(size_t)` can be 1 while `CHAR_BIT` is 157. I'm fine with this being a QoI issue with the non-placement version. – Daniel H Jul 16 '18 at 07:04
@DanielH: absolutely. But, there must be a reason that the standard left this issue open. If this overhead could be circumvented in all cases, then why does the standard allow it? Even, why is the possibility of overhead highlighted: "This overhead may be applied in all array new-expressions, including those referencing the library function operator new[](std::size_t, void*) and other placement allocation functions" – geza Jul 16 '18 at 07:11
@DanielH: note, I've asked a question about this: https://stackoverflow.com/questions/51356882/overhead-of-placement-new – geza Jul 16 '18 at 09:00
@geza _there must be a reason that the standard left this issue open_ Perhaps, but I certainly think we should call them out on it. Posted on your other thread. – Paul Sanders Jul 16 '18 at 13:16
For future reference, from the GH issue: CWG has already seen the placement `new[]` issue and [said EWG should deal with it](http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_closed.html#476). EWG said [no, CWG should deal with it](https://cplusplus.github.io/EWG/ewg-closed.html#90). CWG hasn't responded to that. Mgithubeanwhile, there is [a proposal](http://wg21.link/p0593) which should make placement `new`of any sort unnecessary for many types and placement `new[]` workaroundable for the rest. – Daniel H Jul 16 '18 at 16:43
@DanielH: thanks for the info, I didn't know about this proposal, neither about this funny EWG/CWG ping-pong. C++ should be developed by one dedicated group, as a full time job. It is weird how slow it develops, and how much issues have left behind. – geza Jul 16 '18 at 19:23

std::realloc with static_cast or placement new

1 Answers1

Linked