22

I have seen the following macro being used in OpenGL VBO implementations:

#define BUFFER_OFFSET(i) ((char *)NULL + (i))
//...
glNormalPointer(GL_FLOAT, 32, BUFFER_OFFSET(x));

Could you provide a little detail on how this macro works? Can it be replaced with a function? More exactly, what is the result of incrementing a NULL pointer?

Dan Nestor
  • 2,441
  • 1
  • 24
  • 45

4 Answers4

36

Let's take a trip back through the sordid history of OpenGL. Once upon a time, there was OpenGL 1.0. You used glBegin and glEnd to do drawing, and that was all. If you wanted fast drawing, you stuck things in a display list.

Then, somebody had the bright idea to be able to just take arrays of objects to render with. And thus was born OpenGL 1.1, which brought us such functions as glVertexPointer. You might notice that this function ends in the word "Pointer". That's because it takes pointers to actual memory, which will be accessed when one of the glDraw* suite of functions is called.

Fast-forward a few more years. Now, graphics cards have the ability to perform vertex T&L on their own (up until this point, fixed-function T&L was done by the CPU). The most efficient way to do that would be to put vertex data in GPU memory, but display lists are not ideal for that. Those are too hidden, and there's no way to know whether you'll get good performance with them. Enter buffer objects.

However, because the ARB had an absolute policy of making everything as backwards compatible as possible (no matter how silly it made the API look), they decided that the best way to implement this was to just use the same functions again. Only now, there's a global switch that changes glVertexPointer's behavior from "takes a pointer" to "takes a byte offset from a buffer object." That switch being whether or not a buffer object is bound to GL_ARRAY_BUFFER.

Of course, as far as C/C++ is concerned, the function still takes a pointer. And the rules of C/C++ do not allow you to pass an integer as a pointer. Not without a cast. Which is why macros like BUFFER_OBJECT exist. It's one way to convert your integer byte offset into a pointer.

The (char *)NULL part simply takes the NULL pointer (which is usually a void* in C and the literal 0 in C++) and turns it into a char*. The + i just does pointer arithmetic on the char*. Because the null pointer usually has a zero address, adding i to it will increment the byte offset by i, thus generating a pointer who's value is the byte offset you passed in.

Of course, the C++ specification lists the results of BUFFER_OBJECT as undefined behavior. By using it, you're really relying on the compiler to do something reasonable. After all, NULL does not have to be zero; all the specification says is that it is an implementation-defined null pointer constant. It doesn't have to have the value of zero at all. On most real systems, it will. But it doesn't have to.

That's why I just use a cast.

glVertexAttribPointer(1, 4, GL_FLOAT, GL_FALSE, 0, (void*)48);

It's not guaranteed behavior either way (int->ptr->int conversions are conditionally supported, not required). But it's also shorter than typing "BUFFER_OFFSET". GCC and Visual Studio seem to find it reasonable. And it doesn't rely on the value of the NULL macro.

Personally, if I were more C++ pedantic, I'd use a reinterpret_cast<void*> on it. But I'm not.

Or you can ditch the old API and use glVertexAttribFormat et. al., which is better in every way.

Nicol Bolas
  • 449,505
  • 63
  • 781
  • 982
  • 1
    If the implementation of glNormalPointer subtracts (char*)NULL to get the integer offset, and NULL isn't represented by 0, then (void*)offset won't work. The safest, clearest, and most principled course is to follow the API (no matter how demented it is). – Jim Balter Nov 27 '11 at 07:38
  • 8
    @JimBalter: `BUFFER_OFFSET` is *not* part of the API. You will find this macro *nowhere* in actual OpenGL headers. Just because it's a fairly common thing to see online doesn't make it part of OpenGL. The OpenGL specification clearly states that the `gl*Pointer` calls are required to convert the `void*` into an integer value, which represents a byte-offset from the beginning of the buffer. Subtracting `(char*)NULL` from this value is *not* part of the specification. So such an implementation would be in violation of the OpenGL Specification. – Nicol Bolas Nov 27 '11 at 07:42
  • NULL is always 0 value, at least as far as C and C++ are concerned. Read the language specs, if you don't believe me. – datenwolf Nov 27 '11 at 10:02
  • 2
    @datenwolf `NULL` may be `0` (or `(42-42)`), but `(char*)NULL` may have a non-zero representation... in theory. In practice, there are just too many programs that just call `memset` or `bzero` to zero-initialise everything (int, float, pointer) to have a non-zero representation of null pointers. – curiousguy Nov 27 '11 at 12:19
  • @curiousguy: On the C side a null pointer ALWAYS is zero. On the machine side things may look different. But the C standard mandates that `(char*)0 == (int*)0 == (anytype*)0 == (void*)0` – this is a common misconception, that null pointers may have special representation on the C side. I explained that in length in my answer below. If you want do to it really cleanly, you shoud cast the function signature, not the offset to a pointer. – datenwolf Nov 27 '11 at 12:56
  • @datenwolf I am not sure what you mean by "C side". – curiousguy Nov 27 '11 at 13:06
  • 1
    @curiousguy He probably means (roughly) "at compile time", because the C standard requires `NULL` to be a literal integer expression with the value zero, possibly cast to `void *`. The actual machine, however, is free to use any other representation of null pointers, possibly varying by type, but a conforming _compiler_ cannot define `NULL` differently (or neglect to treat `(char *)0` as equivalent to `(char *)NULL` _when compiling_). The difference _can_ appear at runtime, i.e., if a memory location is treated as a pointer and has all bits zero, it doesn't necessarily work as a null pointer. – Arkku Nov 27 '11 at 13:40
  • 2
    @datenwolf: The macro `NULL` does not *have* to be the number 0. It simply needs to be a null pointer. The C++ specification at no time states that the null pointer *is* zero. It simply says that the integer literal 0, when converted to a pointer, is the null pointer. But that's different from saying that the null pointer has the value of zero. Or that doing pointer arithmetic on a null `char*` means incrementing from zero. – Nicol Bolas Nov 27 '11 at 20:08
  • @NicolBolas: It's not just 0 integer literals that shall yield a null pointer. It's any R-values of 0. – datenwolf Nov 27 '11 at 20:44
  • Seeing how `glVertexAttribPointer` must be loaded as function pointer (and cast to function-pointer-type) anyway, there's the temptation to do without macros and casts and instead just use a less braindead function pointer declaration, something like `typedef void (APIENTRYP PFNGLVERTEXATTRIBPOINTERPROC) (GLuint, GLint, GLenum, GLboolean, GLsizei, unsigned int);`... :-) – Damon Jul 19 '12 at 15:54
29
#define BUFFER_OFFSET(i) ((char *)NULL + (i))

Technically the result of this operation is undefined, and the macro actually wrong. Let me explain:

C defines (and C++ follows it), that pointers can be casted to integers, namely of type uintptr_t, and that if the integer obtained that way, casted back into the original pointer type it came from, would yield the original pointer.

Then there's pointer arithmetic, which means if I have two pointers pointing so the same object I can take the difference of them, resulting in a integer (of type ptrdiff_t), and that integer added or subtracted to either of the original pointers, will yield the other. It is also defines, that by adding 1 to a pointer, the pointer to the next element of an indexed object is yielded. Also the difference of two uintptr_t, divided by sizeof(type pointed to) of pointers of the same object must be equal to the pointers themself being subtracted. And last but not least, the uintptr_t values may be anything. They could be opaque handles as well. They're not required to be the addresses (though most implementations do it that way, because it makes sense).

Now we can look at the infamous null pointer. C defines the pointer which is casted to for from type uintptr_u value 0 as the invalid pointer. Note that this is always 0 in your source code. On the backend side, in the compiled program, the binary value used for actually representing it to the machine may be something entirely different! Usually it is not, but it may be. C++ is the same, but C++ doesn't allow for as much implicit casting than C, so one must cast 0 explicitly to void*. Also because the null pointer does not refer to an object and therefore has no dereferenced size pointer arithmetic is undefined for the null pointer. The null pointer referring to no object also means, there is no definition for sensibly casting it to a typed pointer.

So if this is all undefined, why does this macro work after all? Because most implementations (means compilers) are extremely gullible and compiler coders lazy to the highest degree. The integer value of a pointer in the majority of implementations is just the value of the pointer itself on the backend side. So the null pointer is actually 0. And although pointer arithmetic on the null pointer is not checked for, most compilers will silently accept it, if the pointer got some type assigned, even if it makes no sense. char is the "unit sized" type of C if you want to say so. So then pointer arithmetic on cast is like artihmetic on the addresses on the backend side.

To make a long story short, it simply makes no sense to try doing pointer magic with the intended result to be a offset on the C language side, it just doesn't work that way.

Let's step back for a moment and remember, what we're actually trying to do: The original problem was, that the gl…Pointer functions take a pointer as their data parameter, but for Vertex Buffer Objects we actually want to specify a byte based offset into our data, which is a number. To the C compiler the function takes a pointer (a opaque thing as we learned). The correct solution would have been the introduction of new functions especially for the use with VBOs (say gl…Offset – I think I'm going to ralley for their introduction). Instead what was defined by OpenGL is a exploit of how compilers work. Pointers and their integer equivalent are implemented as the same binary representation by most compilers. So what we have to do, it making the compiler call those gl…Pointer functions with our number instead of a pointer.

So technically the only thing we need to do is telling to compiler "yes, I know you think this variable a is a integer, and you are right, and that function glVertexPointer only takes a void* for it's data parameter. But guess what: That integer was yielded from a void*", by casting it to (void*) and then holding thumbs, that the compiler is actually so stupid to pass the integer value as it is to glVertexPointer.

So this all comes down to somehow circumventing the old function signature. Casting the pointer is the IMHO dirty method. I'd do it a bit different: I'd mess with the function signature:

typedef void (*TFPTR_VertexOffset)(GLint, GLenum, GLsizei, uintptr_t);
TFPTR_VertexOffset myglVertexOffset = (TFPTR_VertexOffset)glVertexPointer;

Now you can use myglVertexOffset without doing any silly casts, and the offset parameter will be passed to the function, without any danger, that the compiler may mess with it. This is also the very method I use in my programs.

datenwolf
  • 159,371
  • 13
  • 185
  • 298
  • 3
    I'd give a +50 if I could for actually exposing this little ridicule in OpenGL standard. Client-side vertex arrays have been out for a while and yet they still haven't cleaned up this archaism even in OpenGL 4. – Kos Jan 01 '12 at 13:59
  • @Kos: Give it time, as soon as GPUs start needing an address space larger than 32-bit for vertices, then using `void*` could be inadequate in certain build environments. But generally the size of the memory the language gives for data pointers is large enough to reference any location that can be indexed within a VBO and definitely not worth the effort of defining completely new variants of the vertex pointer functions. – Andon M. Coleman Apr 21 '14 at 17:06
  • 1
    @AndonM.Coleman: Well, in the meantime since I wrote this answer OpenGL introduced a set of new functions to specify offests into VBOs (glVertexAttribFormat and friends). That should take care of this. – datenwolf Apr 21 '14 at 21:26
  • @datenwolf: That is a good point, but it still uses `GLuint` (32-bit). They'll need another set of functions to expand the buffer object address space beyond 32-bit. Something like `glVertexAttrib{L|I}Format64 (...)`. Things like `void*` and `GLintptr` have no guaranteed size and are determined by the client. Limiting the buffer object's address space to the size of the client's pointer like most of the buffer object API does is kind of funny; that is an artificial limitation just because they chose to reuse `void*`. If 16-bit clients were common, they would have changed that long ago. – Andon M. Coleman Apr 21 '14 at 22:16
2

That's not "NULL+int", that's a "NULL cast to the type 'pointer to char'", and then increments that pointer by i.

And yes, that could be replaced by a function - but if you don't know what it does, then why do you care about that? First understand what it does, then consider if it would be better as a function.

Arafangion
  • 11,517
  • 1
  • 40
  • 72
  • Thank you, understanding is indeed my purpose here. My question then becomes: what is the result of incrementing a NULL pointer? – Dan Nestor Nov 27 '11 at 04:56
  • Same as the result of incrementing a pointer of any other value? Actually you do have a point, I'm not sure if, strictly speaking, incrementing a NULL pointer is defined behavior in C. – Arafangion Nov 27 '11 at 05:03
  • Whatever the result, I don't imagine it'd be an address you're allowed to access. – derekerdmann Nov 27 '11 at 05:09
  • 1
    @derekerdmann I presume not, this is meant to be passed as an argument to a OpenGL function which expects a void* offset. – Dan Nestor Nov 27 '11 at 05:11
  • 1
    (Very) strictly speaking, performing arithmetic with a null pointer operand yields undefined behavior (at least in any case where the result is not also a null pointer), as a null pointer is not a pointer to an object. – James McNellis Nov 27 '11 at 05:16
  • 'That's not "NULL+int"' -- but it is (char*)NULL+int. 'that's "pointer to char, initialised to NULL' -- well, no; it's "NULL cast to pointer to char"; there's no 'initialis'ation here. 'and you then increment that pointer by i' -- no, 'you' don't do anything and there's no 'increment'ing going on -- there is simply an *expression* which is the *sum* of two values ... and no add or 'increment' instruction appears in the generated code. 'First understand what it does' -- with which you provided no assistance. – Jim Balter Nov 27 '11 at 07:51
  • @Jim Balter: Apologies - that's a terminology point, and you are quite right. (I spend more time in C++), will update so that the response is clearer, however I do believe you are overly harsh with your comment. Lets argue about the semantics of "you" elsewhere, perhaps in linguistics. – Arafangion Nov 28 '11 at 06:13
2

openGL vertex attribute data is assigned through the same function (glVertexAttribPointer) as either pointers in memory or offsets located within a Vertex Buffer Object depending on context.

the BUFFER_OFFSET() macro appears to convert an integer byte offset into a pointer simply to allow the compiler to pass it as a pointer argument safely. The "(char*)NULL+i" expresses this conversion through pointer-arithmetic; the result should be the same bit-pattern assuming sizeof(char)==1, without which, this macro would fail.

it would also be possible through simple re-casting, but the macro might make it stylistically clearer what is being passed; it would also be a convenient place to trap overflows for 32/64bit safety/futureproofing

struct MyVertex { float pos[3]; u8 color[4]; }
// general purpose Macro to find the byte offset of a structure member as an 'int'

#define OFFSET(TYPE, MEMBER) ( (int)&((TYPE*)0)->MEMBER)

// assuming a VBO holding an array of 'MyVertex', 
// specify that color data is held at an offset 12 bytes from the VBO start, for every vertex.
glVertexAttribPointer(
  colorIndex,4, GL_UNSIGNED_BYTE, GL_TRUE,
  sizeof(MyVertex), 
  (GLvoid*) OFFSET(MyVertex, color) // recast offset as pointer 
);
centaurian_slug
  • 3,129
  • 1
  • 17
  • 16
  • 1
    also, it has the same effect as `(char *) ( sizeof(char)*i )` if that helps anybody with understanding it – PeterT Nov 27 '11 at 04:57
  • 2
    @PeterT: That is not necessarily the same: the null pointer is not necessarily represented by an integer value of zero, and the conversion from integer to pointer is implementation-defined. – James McNellis Nov 27 '11 at 05:13
  • So the OpenGL function internally adds this pointer to the pointer to the VBO? If the VBO contains float values, shouldn't the macro compute an offset using float* ? – Dan Nestor Nov 27 '11 at 05:15
  • @JamesMcNellis: Although the C standard does not require that the value of a `NULL` pointer be zero (even though `NULL == 0`), the systems for which this is not true are fairly arcane. – Dietrich Epp Nov 27 '11 at 05:16
  • @Dan Nestor - yes it adds the pointer to it's own VBO address which might be in a different space (the graphics card). But you wouldn't use 'float*', it wants the exact unchanged integer value merely CAST as a pointer. - personally I would just re-cast the integer argument directly, without the macro. (see edit) – centaurian_slug Nov 27 '11 at 05:33
  • "assuming sizeof(char)==1" -- no assuming necessary; the C standard requires it. And even if it didn't, the assumption would not be necessary, as the integer value can be retrieved by subtracting (char*)NULL (if, say, sizeof(char) were 2, that would divide the doubled offset by 2). – Jim Balter Nov 27 '11 at 07:56
  • "also, it has the same effect as (char *) ( sizeof(char)*i ) if that helps anybody with understanding it" -- I can't imagine how it could. First, it has different semantics. Second, sizeof(char) is required to be 1 in C. – Jim Balter Nov 27 '11 at 08:00