3

I was looking an example of Modern OpenGL in c++ where a structure is used to hold data regarding vertex coordinates and a 3 coordinate color vector

struct VertexData{
    float x, y;
    float r, g, b;
}

std::vector<VertexData> myData = {
    {.x = 0, .y = 0, .r = 0, .g = 0, .b = 0},
    {.x = 0, .y = 0, .r = 0, .g = 0, .b = 0}
}

float *ptr = (float*) &myData[0];

To access the x value of the second element of the vector, I could acces ptr[5], technique used to pass a vertex buffer to OpenGL. My question is: Can I use this same technique with a more complex class? As a concrete example, if I have a Vertex class:

class Vertex {
public:
    float x, y, z;
    Vertex();
    // Other constructors here...
    Vertex operator+(const Vertex &other);
    // Overload of other operators like -,*,=, etc...
}
std::vector<Vertex> myData(100);

float *ptr = (float*) &myData[0];

would it still be safe (and a good idea...) to use the ptr variable to access all elements in this vector? I know this could be tricky in the presence of static variables, different types, inheritance and so on, but in this case were there are only plain variables and functions as members, would using a pointer be a reliable approach?

Malanche
  • 191
  • 8
  • Designated initializers, already? – StoryTeller - Unslander Monica Nov 25 '17 at 18:10
  • 7
    Even the first version isn't safe. The second is no different. – juanchopanza Nov 25 '17 at 18:10
  • Let the compiler optimize it for you -- much safer, much less brittle – Dave S Nov 25 '17 at 18:12
  • But structures will store the member values in a contiguous chunk of memory, and a vector would do the same, no? Why would it not be safe? – Malanche Nov 25 '17 at 18:12
  • *"But structures will store the member values in a contiguous chunk of memory"* - Where in the C++ standard did you find this guarantee? Please point us to it. – StoryTeller - Unslander Monica Nov 25 '17 at 18:13
  • 1
    Yes, this approach *can* work for a class, but only if you use 1-byte alignment for the class members (same goes for the `struct` example, for that matter) to avoid padding between members. However this is technically *undefined behavior* as it violates strict aliasing rules. So check your particular compiler to make sure it works as expected – Remy Lebeau Nov 25 '17 at 18:13
  • 1
    @Malanche There are no "structures" in C++. You get classes, which you can define with two different keywords. In terms of layout, there is no difference between your two examples, except that one has 5 floats and the other 3. – juanchopanza Nov 25 '17 at 18:16
  • @StoryTeller That is why I added a "No?" at the end of my statement, I was asking. I thought C++ was doing some kind of "malloc" of the total size of the member variables to store them, but I think this is not the case. – Malanche Nov 25 '17 at 18:19
  • @RemyLebeau Thank you for the answer! But then this becomes compiler-dependent, hence, I have no guarantee to obtain the same behavior across compilers, is it the case? – Malanche Nov 25 '17 at 18:21
  • 1
    @Malanche: I've been using a technique where I put a `float c[3];` and `struct { float x, y, z };` into a union (so I can access an element with `c[1]`, or `y`) for more than 10 years, with a lot of different compilers. This technique works with all of them. According to the standard, this is UB. According to practice... well, it works (I can break any time, but it is very unlikely it would). – geza Nov 25 '17 at 18:25
  • 3
    @geza I hope you put a scary warning in your header files reminding you (and more important, anyone else using the code) that any change to the data members (like adding a new one) will break code in exciting and fun ways :) – Dave S Nov 25 '17 at 18:31
  • @DaveS: these classes have very well defined semantics, no-one should modify their data. For example, `Matrix3x4r_float`. It means that it stores a 3x4 matrix, float elements, row major order. I think, in this case, if you want to have the convenience to refer to a member as `tx` (translation x), or `c[3]`, then there is no other solution. I'd rather write code conveniently, than using awkward techniques, just because the standard doesn't guarantee something, which is absolutely logical, and there is absolutely no reason to ever break. – geza Nov 25 '17 at 18:36
  • Oh - pardon me I did not see your `float *ptr = (float*) &myData[0];` This will not be working - only DIRECT pointers to properties. In memory - them not just floats, one-by-one – Muritiku Nov 25 '17 at 18:54
  • My C++ compiler doesn't support the C99 syntax for the named initialized fields. – Eljay Nov 25 '17 at 19:24

2 Answers2

-1

According to the C++ Standard, this technique has undefined behavior. The compiler is free to add padding between members. So, according to the principles of the standard, this is a bad idea.

However.

All compilers I know (I've worked with 8 different C++ compilers in the past 15 years), don't add padding in this case. Why would they? So, if you dare to enter undefined-behavior land, you can use this technique. Even in complex cases. Inheritance, virtual functions, etc. All the compilers I know, do the logical thing, and put float members next to each other in memory in your case, and their optimizers don't fail for this case. Note: the standard allows compilers reorder members between access specifiers, so if you want to be on the safe side (which you've already left), put all members under the same access specifier (still, I've yet to see a compiler, which actually reorders members between access specifiers.).

So. Is it reliable? According to the standard: No. According to the current practice: in my experience, yes. I've yet to see a compiler, for which such a code fail. I encourage anyone who knows such a compiler, to not be silent, and add it as an answer.

Note: this is the current status. It may break any time. I'd only recommend using this technique if you know what you're doing, and you're aware of the possible consequences. And don't forget to comment this fact in your code.

geza
  • 28,403
  • 6
  • 61
  • 135
  • 1
    No, it is not reliable according to current practice. The fact that a C++ implementation does not insert padding in this situation does not mean its optimizer will not recognize the undefined behavior and transform the undefined code to something completely different. – Eric Postpischil Nov 25 '17 at 19:23
  • 1
    @EricPostpischil: please give an actual example. I've been using this technique for 15 years, 8 different compilers, it worked with all of them. – geza Nov 25 '17 at 19:24
  • 1
    Reliability comes from assurance by some specification that the behavior will be as stated. Unless either the C++ standard or the implementation documentation specifies that the code will behave in the desired way, reliability cannot exist. You can test if you want, but you cannot know that tomorrow’s version of the compiler, or some subtle change in your source code, will not produce a different result. – Eric Postpischil Nov 25 '17 at 19:27
  • @EricPostpischil: Agreed. That's why I say: "**current** practice: Yes." But again, feel free to give an example, which proves the opposite. I'm not saying it's impossible, I've just never seen any. – geza Nov 25 '17 at 19:31
  • 1
    Current practice includes changes in the source code. You cannot know. You only know the situations you have tested in have worked (or at least that errors have gone undiscovered). Without assurance from a specification, you cannot know that other cases that you have not tested will also work. I do not have one at hand, but I know that I cannot rely on behavior that is not specified, and I know that the reasoning that because there is no padding, this behavior can be safely used is faulty reasoning. – Eric Postpischil Nov 25 '17 at 19:32
  • 1
    As examples of things that **could** go wrong, just to show you that optimizers are tricky: It might be that with code such as `ptr[i]`, the optimizer does not know at compile-time what `i` might be, so it generates run-time code to do the access, and that works. But if a constant were used, such as `ptr[2]`, the optimizer could recognize it is undefined and eliminate the code entirely. So a small change can break code like this. – Eric Postpischil Nov 25 '17 at 19:39
  • 1
    Another example: Perhaps `ptr[i]` is used twice in some code. So the optimizer decides to fetch it once, keep a copy in a register, and use it twice. But between the two uses, `myData[0].y` is set to a new value. When defined code is used, the optimizer recognizes that the value may change, and it generates a second load (or whatever code is necessary) to get the changed value. But, because the access is through an improperly aliased pointer, the optimizer may fail to recognize there is a change, so it does not generate a load and uses the old copy of the value. – Eric Postpischil Nov 25 '17 at 19:41
  • @EricPostpischil: if there's no padding, then `ptr[2]` isn't undefined, as `ptr+2` points to a proper `float`. For your second case: the optimizer is broken. `ptr[i]` points to `myData[0].y`, yet it doesn't load the value again. I myself could give a lot of theory, why could it break. Here, the question is not "what could go wrong", but "what does go wrong". Again (last time): if no example can be given that it breaks, then it's reliable. If anyone can give a counter example, I'll delete my answer (or leave it here as a memento). – geza Nov 25 '17 at 19:50
  • `ptr[2]` is undefined because no statement in the C++ standard defines what happens if you convert the address of a `VertexData` to a pointer to `float`, add 2, and dereference the result. The C++ does not directly define memory layouts in actual hardware by which you can assume objects are there. It defines behavior inside an abstract machine, and nothing in the abstract machine is guaranteed to behave the way you visualize actual machines unless it is specified by the C++ standard (or specified by a particular implementation you are using). – Eric Postpischil Nov 25 '17 at 19:53
  • @EricPostpischil: can you please differentiate between the C++ standard, and practice? I'm not talking about the standard. I'm talking about the current compilers, and practice. I've said in my answer, that it is UB. According to the standard, anything can happen. But according practice, it just works. If you say, that it is not reliable in practice, then give a counter example (not theory, what could go wrong, etc., an actual example!). – geza Nov 25 '17 at 19:59
  • 2
    Can you please differentiate between “I have not seen this break” and “This is reliable”? – Eric Postpischil Nov 25 '17 at 20:01
  • @EricPostpischil: yes, they could differ. But until a counter example is given, for me, they are the same. (I've written in my answer, why I think it reliable. I didn't just say "reliable, you can use it, no problem") – geza Nov 25 '17 at 20:06
  • @EricPostpischil: related question: https://stackoverflow.com/questions/47490264/is-moving-a-pointer-to-past-a-struct-member-ub-and-accessing-it – geza Nov 25 '17 at 20:36
  • @EricPostpischil: I've edited my answer, hopefully it's better (fine?) now. – geza Nov 26 '17 at 01:04
-1

As others already mentioned, this method of yours is not reliable, because the compiler may choose to pad your struct. For example, if you are on a 64 bit platform, the compiler may choose to pad the struct to 8-byte align.

But the idea of using a pointer to access members of class still can work. You can use sizeof to get the actual size of the struct (in bytes).

With your first example, it would look like this:

float *ptr = (float*) &myData[0].x;

and access the next x value like this:

ptr = (float*)(((char*)ptr) + sizeof(VertexData));

or, alternatively, if you are sure that sizeof(VertextData) is a multiple of sizeof(float), it could be:

ptr += sizeof(VertexData) / sizeof(float);

This is a bit complicated, but is reliable. Also, you can wrap things into macros, if you wish.

Whether this method is practical depends on your particular situation. I'd say in certain situations it could be useful.


EDIT:

There is, nevertheless, some danger in using this method.

The most important thing to remember is that, if you insert some elements into your vector, the whole storage could be reallocated, hence your previous pointer will no longer be valid.


EDIT:

There is also another way, avoiding the use of sizeof. To access the next x value, you can use:

ptr = (float*) (((VertexData*) ptr) + 1);

This should compile to the same result.

WhatsUp
  • 1,618
  • 11
  • 21
  • The fact that a C++ implementation does not insert padding in this situation does not mean its optimizer will not recognize the undefined behavior and transform the undefined code to something completely different. This is not reliable. – Eric Postpischil Nov 25 '17 at 19:43
  • @EricPostpischil I don't understand why this is not reliable. I think `sizeof` already takes into account the potential paddings at compile time... – WhatsUp Nov 25 '17 at 19:45
  • Compilers are no longer simple programs that write obvious code. As a compiler parses your code, it constructs an abstract description of your program as it should behave inside a virtual machine described by the language standard. Then it runs an optimizer that analyzes the abstract code and transforms it in various ways. The resulting transformations are designed so that defined behavior stays the same (while performance and other features improve), but they do not provide guarantees about undefined behavior. Anything not defined by the rules can be completely changed. – Eric Postpischil Nov 25 '17 at 19:49
  • @EricPostpischil I absolutely agree with you, but `std::vector` guarentees to store data contiguously, so this should not be undefined. The distance between two consecutive `VertexData` should be exactly `sizeof(VertexData)`. – WhatsUp Nov 25 '17 at 19:53
  • Where the elements are stored is irrelevant. If you break the rules, the optimizer is allowed to break your code. Period. When you write C++ code, you are **not** writing instructions about accessing memory in your hardware. `ptr[2]` is **not** an instruction to take the value of `ptr`, add 2 elements to it, and use the address to access memory. It is an instruction to reference the third member (index 2) of the array of `float` whose first member is pointed to by `ptr`. If `ptr` is a pointer converted from `&myData[0]`, it is not pointing to an array of `float`. – Eric Postpischil Nov 25 '17 at 19:58
  • To expand on that last part, if you take the address of `myData[0]`, which is not a `float`, and you convert it to a pointer to `float`, **you do not have a pointer to a float object** as defined by the C++ standard. You have a pointer that has been converted in some incompletely specified way. If you use it as a pointer to a float object even though it is not, the behavior is undefined **regardless of what is in memory**. – Eric Postpischil Nov 25 '17 at 19:59
  • (As an aside, for completeness, in C, you can convert a pointer to a structure to a pointer to the type of its initial member, and the result is a valid pointer to the initial member. There is a special rule for that in C, but I do not think it is in C++. So `ptr[0]` is defined in C. However, using that pointer to access additional members as if they were an array is not defined even in C. `ptr[1]` is undefined.) – Eric Postpischil Nov 25 '17 at 20:04
  • @EricPostpischil Please read my post more carefully before your criticism. I took the address of `myData[0].x`, instead of `myData[0]`. If you insist that my code has undefined behavior, then what does it mean that `std::vector` guarantees to store objects contiguously??? – WhatsUp Nov 25 '17 at 20:05
  • Setting `float *ptr` from either `&myData[0].x` or `&myData[0]` and then using `ptr[i]` for anything other than `i` equal to 0 is undefined behavior in both cases. – Eric Postpischil Nov 25 '17 at 20:06
  • The fact that `std::vector` stores its objects contiguously means that the objects in a `std::vector` form an array of `VertexData`. You could take the address of one of them, assign it to an object of type `VertexData *`, and use it with subscripts to access the various `VertexData` objects within the vector. That has nothing to do with accessing the contents within a `VertexData` structure using improperly derived pointers. – Eric Postpischil Nov 25 '17 at 20:08
  • @EricPostpischil I didn't use anything like `ptr[i]`. I manually add `sizeof(VertexData)` to the value of `ptr`. Please, do read my post carefully before saying anything... – WhatsUp Nov 25 '17 at 20:09
  • You are correct, I have not been looking at your answer correctly. However, it is still wrong. When you take the address of a scalar object, such as `&myData[0].x`, pretty much the only thing you can use it for is accessing that object. (Some other legal things may include comparing it to other pointers or adding 1 to it but no more.) If you try to add various things to it and use it as a pointer to some other object, even one you think is laid out in a defined location in memory, the behavior is undefined. – Eric Postpischil Nov 25 '17 at 20:12
  • @EricPostpischil If you are really academical, yes, you could also complain that using `memcpy` to copy structures is undefined behavior ... and the current problem is no different from `memcpy(dst, src, sizeof(VectorData))`. – WhatsUp Nov 25 '17 at 20:25
  • `memcpy` is defined by the C++ standard (by reference to the C standard), so the behavior of using it is defined. And it is not academic; if you want to write commercial software that is reliable, you have to follow the rules. – Eric Postpischil Nov 25 '17 at 20:28
  • @EricPostpischil I'm not saying that `memcpy` has undefined behavior... I'm saying that, if you think my method has undefined behavior, then by the same logic, you would say that `memcpy(dst, src, sizeof(VectorData))` doesn't guarantee to correctly copy `src` to `dst`. – WhatsUp Nov 25 '17 at 20:30
  • C++ clause 6.9 paragraph 3 specifies that the bytes of an object can be copied, and that the effect is to copy the object’s value, and shows an example using `memcpy`. Copying bytes in C and C++ is special, including by `memcpy`, as defined by the standard. It is not some hack somebody made up; it is in the rules. As long as you follow the rules, the behavior is defined. If you do not follow the rules, the behavior is not defined. – Eric Postpischil Nov 25 '17 at 20:36
  • @EricPostpischil If you admit that all instances of a `struct` has the same arrangements in memory, and that all objects in a `vector` are stored contiguously in memory, then I don't understand why `ptr += sizeof(VectorData)` doesn't point to the next `x` value. – WhatsUp Nov 25 '17 at 20:44
  • The rules in the C++ and C standards originate in part from machines that did not have the flat address space that many of today’s machines do. In those machines, pointers are not just addresses that you can add some number of bytes to and get another address. They were composed of segment descriptors of some sort and offsets. To adjust a pointer, you had to do a variety of manipulations, possibly including table lookups. The C standard defined pointer arithmetic so that the compiler had to do this for you, as long as you used pointers in legal ways (and the C++ standard inherited).… – Eric Postpischil Nov 25 '17 at 20:48
  • … In consequence, if you violated those rules, the manipulations the compiler had to do broke. So it did not matter if things were laid out contiguously in memory; the hardware addressing mechanisms would break. The “address” would not point to where you wanted it to. Of course, many modern machines use a flat address space and do not have this addressing problem. But C and C++ implementations have taken advantage of the rules, and additional layers of semantics, including strong typing, have been added. The optimizers take advantage of these rules to figure out things like `x` and `y` point… – Eric Postpischil Nov 25 '17 at 20:50
  • … to different types. Therefore, if somebody writes to `*x`, it cannot possibly affect the value in `*y`. Therefore, I can optimize reads from `*y` by caching a copy of the value in a register, and I do not need to update it when somebody modifies `*x`. This optimization works as long as people obey the rules. But if you violate the rules, writing to `*x` might change the thing in memory that `y` points to, but the optimizer does not know it, so it continues to use the old copy it saved. And your program breaks. This is true even if `x` has the same type as `y` but was improperly derived. – Eric Postpischil Nov 25 '17 at 20:52
  • **Anytime you break a rule of the standard, you may break something the optimizer does, even if it is not obvious from the underlying hardware.** – Eric Postpischil Nov 25 '17 at 20:53
  • @EricPostpischil I am quite familiar with 16 bit assembly, in particular the `offset, segment` pair. But this has nothing to do with the memory model in C language. The pointer arithmetic already takes care of this problem. – WhatsUp Nov 25 '17 at 20:55
  • As to your example of optimization, it would be hard to imagine that an optimizer would think that `*(x + something)` only affects members of `*x`... this would just be a VERY STUPID optimizer. You should, in that case, change your optimizer, instead of your code. – WhatsUp Nov 25 '17 at 20:57
  • The pointer arithmetic in C takes care of this problem **if you obey the rules**. And it is not stupid for an optimizer to do whatever it wants with undefined behavior. There are in fact good reasons for it, because it allows valid useful optimizations that are impossible otherwise. – Eric Postpischil Nov 25 '17 at 21:00
  • @EricPostpischil Yes, I'm saying that I am obeying the rule. Which rule am I breaking? Adding some thing to a pointer? Casting a pointer to a different type of pointer? All these are typical in C language, which should be compatible with C++. Of course, in C you should be careful that your pointer does point to somewhere, but this is different from breaking a rule. – WhatsUp Nov 25 '17 at 21:04
  • Given that `ptr` was assigned the address of `x` via `(float*) &myData[0].x`, it is a pointer to a single object, `x`. Per C++ clause 8.7, paragraph 4: “When an expression that has integral type is added to or subtracted from a pointer, the result has the type of the pointer operand. If the expression P points to element x[i] of an array object x with n elements, the expressions P + J and J + P (where J has the value j) point to the (possibly-hypothetical) element x[i + j] if 0 ≤ i + j ≤ n; **otherwise, the behavior is undefined.**” A footnote adds that, for this purpose,… – Eric Postpischil Nov 25 '17 at 21:14
  • … a single object acts like an array of one element. So, you have an expression of integral type (`sizeof(VertexData) / sizeof(float)`), you are adding it to a pointer. That pointer points to element 0 of an array with 1 element (as specified by the footnote), but the result is not a pointer to element 0 or what would be element 1. It is beyond the bounds. Therefore, the “otherwise” clause applies: the behavior is undefined. – Eric Postpischil Nov 25 '17 at 21:16
  • @EricPostpischil Hence by the same argument, you would say that after `char* s = (char*) 10; s ++;` the value of `s` is undefined. But I would say it's definitely equal to `11`. – WhatsUp Nov 25 '17 at 21:23
  • I have seen a platform where (a) segmented memory existed, (b) arrays couldn't span segments, (c) size_t was 16bit, (d) pointers were 32bit (implemented like `struct pointer {uint16_t segment, offset};`), (e) pointer arithmetics affected only offset. Doing `while (a < b) ++a;` would lead to infinite loop if a and b were in different segments. And it was standart comliant! – Revolver_Ocelot Nov 25 '17 at 21:26
  • @WhatsUp: After `char *s = (char *) 10; s++;`, not only is the value of `s` unspecified, but the entire behavior of the program is undefined. You might get 11 if you try it, but that is not informative in any way about whether the behavior is defined or not. The standard is clear, the behavior is undefined. – Eric Postpischil Nov 25 '17 at 21:28
  • And to char example, if you do `char* s = (char*) 0xAFFFF; s++` on platform I mentioned, value of s would be `0xA0000`. Notice that both examples are straight _"dumb compiler"_ examples without fancy optimizers tricks and UB-driven optimisations. So things could break even without optimisers! – Revolver_Ocelot Nov 25 '17 at 21:31
  • @EricPostpischil Alright, then we don't have anything to argue. – WhatsUp Nov 25 '17 at 21:33
  • @Revolver_Ocelot What is the platform you are talking about? If arrays couldn't span segments, the problem will not even exist: the whole `vector` should be in one segment. Besides, I used `(char*) 10`, such a small value, just to avoid this problem. – WhatsUp Nov 25 '17 at 21:38
  • It was some unnamed DSP I struggled^W worked with in university. Most of standard library didn't exist on that embedded enviroment. My task was to implement class representing a big (>64KB) array (in the end I made something closely resembling std::deque). My comments were mostly abstract anecdotical evidence, that unsupported pointer arithmetics can easily go wrong. – Revolver_Ocelot Nov 25 '17 at 21:56