Pointer address span on various platforms

Question

A common situation while coding in C is to be writing functions which return pointers. In case some error occurred within the written function during runtime, NULL may be returned to indicate an error. NULL is just the special memory address 0x0, which is never used for anything but to indicate the occurrence of a special condition.

My question is, are there any other special memory addresses which never will be used for userland application data?

The reason I want to know this is because it could effectively be used for error handling. Consider this:

#include <stdlib.h>
#include <stdio.h>

#define ERROR_NULL 0x0
#define ERROR_ZERO 0x1

int *example(int *a) {
    if (*a < 0)
        return ERROR_NULL;
    if (*a == 0)
        return (void *) ERROR_ZERO;
    return a;
}

int main(int argc, char **argv) {
    if (argc != 2) return -1;
    int *result;
    int a = atoi(argv[1]);
    switch ((int) (result = example(&a))) {
        case ERROR_NULL:
            printf("Below zero!\n");
            break;

        case ERROR_ZERO:
            printf("Is zero!\n");
            break;

        default:
            printf("Is %d!\n", *result);
            break;
    }
    return 0;
}

Knowing some special span of addresses which never will be used by userland applications could effectively be utilized for more efficient and cleaner condition handling. If you know about this, for which platforms does it apply?

I guess spans would be operating system specific. I'm mostly interested in Linux, but it would be nice to know for OS X, Windows, Android and other systems as well.

score 5 · Answer 1 · answered Mar 06 '13 at 00:07

NULL is just the special memory address 0x0, which is never used for anything but to indicate the occurrence of a special condition.

That is not exactly right: there are computers where NULL pointer is not a zero internally (link).

are there any other special memory addresses which never will be used for userland applications?

Even NULL is not universal; there are no other universally unused memory addresses, which is not surprising, considering the number of different platforms programmable in C.

However, nobody stops you from defining your own special address in memory, setting it in a global variable, and treating it as your error indicator. This will work on all platforms, and would not require a special address location.

In the header:

extern void* ERROR_ADDRESS;

In a C file:

static int UNUSED;
void *ERROR_ADDRESS = &UNUSED;

At this point, ERROR_ADDRESS points to a globally unique location (i.e. the location of UNUSED, which is local to the compilation unit where it is defined), which you can use in testing pointers for equality.

Woa! That was a cleaver one! Didn't think about registering them myself. It meets all my criteria. Amazing! — Emanuel, Mar 06 '13 at 10:51

score 1 · Answer 2 · answered Mar 06 '13 at 00:08

It completely depends on both the computer and the operating system. For example, on a computer with memory-mapped IO like the Game Boy Advance, you probably don't want to confuse the address for "what color is the upper left pixel" with userland data:

http://www.coranac.com/tonc/text/hardware.htm#sec-memory

score 1 · Answer 3 · answered Mar 06 '13 at 00:09

You should not be worrying about addresses as a programmer, because it's different on different platforms and between actual hardware addresses and your application you have quite some layers. There's the physical to virtual translation being one of the big ones, and the virtual address space is mapped into memory, and each process has it's own address space, protected at hardware level from other processes, on most modern operating systems.

What you are specifying here are just hexadecimal values, they aren't interpreted as addresses. A pointer set to NULL is essentially saying it doesn't point to anything, not even address zero. It's just NULL. Whatever the value of that may be, depends on platform, compiler and a lot of other things.

Setting a pointer to any other value is not defined. A pointer is a variable that stores the address of another, what you're trying to do is give this pointer some other value than what is valid.

score 1 · Accepted Answer · edited May 23 '17 at 12:03

The answer depends a lot on your C compiler and on your CPU and OS, where your compiled C program is going to run.

Your userland applications typically will never be able to access data or code through pointers pointing to the OS kernel data and code. And the OS usually does not return such pointers to applications.

Typically they will also never get a pointer pointing to a location that's not backed up by physical memory. You can only get such pointers through an error (a code bug) or by purposefully constructing such a pointer.

The C standard does not anyhow define what a valid range for pointers is and isn't. In C valid pointers are either NULL pointers or pointers to objects whose lifetime hasn't ended yet and those can be your global and local variables and those created in malloc()'d memory and functions. The OS may extend this range by returning:

pointers to code or data objects not explicitly defined in your C program at its source code level (the OS may let apps access some of its code or data directly, but this is uncommon, or the OS may let apps access some of their parts that are either created by the OS when the app loads or created by the compiler when the app was compiled, one example would be Windows letting apps examine their executable PE image, you can ask Windows where the image starts in the memory)
pointers to data buffers allocated by the OS for/on behalf of apps (here, usually, the OS would use its own APIs and not your app's malloc()/free(), and you'd be required to use the appropriate OS-specific function to release this memory)
OS-specific pointers that can't be dereferenced and only serve as error indicators (e.g. you could have more than just one undereferenceable pointer like NULL and your ERROR_ZERO is a possible candidate)

I would generally discourage use of hard-coded and magic pointers in programs.

If for some reason, a pointer is the only way to communicate error conditions and there are more than one of them, you could do this:

char ErrorVars[5] = { 0 };
void* ErrorPointer1 = &ErrorVars[0];
void* ErrorPointer2 = &ErrorVars[1];
...
void* ErrorPointer5 = &ErrorVars[4];

You can then return ErrorPointer1 through ErrorPointer1 on different error conditions and then compare the returned value against them. There' a caveat here, though. You cannot legally compare a returned pointer with an arbitrary pointer using >, >=, <, <=. That's only legal when both pointers point to or into the same object. So, if you wanted a quick check like this:

if ((char*)(p = myFunction()) >= (char*)ErrorPointer1 &&
    (char*)p <= (char*)ErrorPointer5)
{
  // handle the error
}
else
{
  // success, do something else
}

it would only be legal if p equals one of those 5 error pointers. If it's not, your program can legally behave in any imaginable and unimaginable way (this is because the C standard says so). To avoid this situation you'll have to compare the pointer against each error pointer individually:

if ((p = myFunction()) == ErrorPointer1)
  HandleError1();
else if (p == ErrorPointer2)
  HandleError2();
else if (p == ErrorPointer3)
  HandleError3();
...
else if (p == ErrorPointer5)
  HandleError5();
else
  DoSomethingElse();

Again, what a pointer is and what its representation is, is compiler- and OS/CPU-specific. The C standard itself does not mandate any specific representation or range of valid and invalid pointers, so long as those pointers function as prescribed by the C standard (e.g. pointer arithmetic works with them). There's a good question on the topic.

So, if your goal is to write portable C code, don't use hard-coded and "magic" pointers and prefer using something else to communicate error conditions.

LihO · Answer 5 · 2013-03-06T00:26:39.770

This code:

#define ERROR_NULL 0x0
#define ERROR_ZERO 0x1

int *example(int *a) {
    if (*a < 0)
        return ERROR_NULL;
    if (*a == 0)
        return (void *) ERROR_ZERO;
    return a;
}

defines a function example that takes input parameter a and returns the output as a pointer to int. At the same time, when the error occurs, this function abuses cast to void* to return the error code to the caller in the same way it returns the correct output data. This approach is wrong, because the caller must know that sometimes valid output is received, but it doesn't actually contain the desired output but the error code instead.

are there any other special memory addresses which never will be used ... ?
... it could effectively be used for error handling

Don't make any assumptions about the possible address that might be returned. When you need to pass a return code to the caller, you should do it in more straightforward way. You could take the pointer to the output data as a parameter and return the error code that identifies success or failure:

#define SUCCESS     0x0
#define ERROR_NULL  0x1
#define ERROR_ZERO  0x2

int example(int *a, int** out) {
    if (...)
        return ERROR_NULL;
    if (...)
        return ERROR_ZERO;
    *out = a;
    return SUCCESS;
}
...
int* out = NULL;
int retVal = example(..., &out);
if (retVal != SUCCESS)
    ...

score 0 · Answer 6 · edited May 23 '17 at 11:49

Actually NULL(0) is a valid address. But it's not an address that you can typically write to.

From memory, NULL could be a different value on some old VAX hardware with some very old c compiler. Maybe someone can confirm that. It will always be 0 now as the C standard defines it - see this question Is NULL always false?

Typically the way errors are returned from functions is to set errno. You could piggy back on this if the error codes makes sense in the particular situation. However, if you need your own errors then you could do the same thing as the errno method.

Personally I prefer to not return void* but make the function take a void** and return the result there. Then you can return an error code directly where 0 = success.

e.g.

int posix_memalign(void **memptr, size_t alignment, size_t size);

Note the allocated memory is returned in memptr. The result code is returned by the function call. Unlike malloc.

void *malloc(size_t size)

score 0 · Answer 7 · edited Sep 18 '19 at 10:43

On Linux, on 64-bit and when using the x86_64 architecture (either from Intel or AMD) only 48 bits of the total 64-bit address space are used (hardware limitation AFAIK). Basically, any address after 2⁴⁷ until 2⁶² can be used now as it will not be allocated.

For some background, the virtual address space of a Linux process is made of a user and kernel space. On the above mention architecture, the first 47 bits (128 TB) are used for the user space. The kernel space is used at the end of the spectrum, so the last 128 TB at the end of a full 64-bit address space. In between is terra incognita. Although that could change any time in the future and this is not portable.

But I could think of many other way to return an error than your method, so I do not see the advantage of using such an hack.

phuclv · Answer 8 · 2020-10-09T10:14:56.897

TL;DR:

Use -1 if you want just one more error condition beside NULL
For more special conditions just set the least significant bit(s), because the returned value from malloc() family or new is guaranteed to be aligned for any fundamental alignment and will have the low bits always zero, so they're free for use (like in a tagged pointer)

If allocation succeeds, returns a pointer that is suitably aligned for any object type with fundamental alignment.

https://en.cppreference.com/w/c/memory/malloc

Pointers to types wider than char are also always aligned. If you point to a char or a char array on stack then just align as necessary with alignas
For even more conditions you can limit the range of allocated addresses. This needs platform-specific code and there won't be a portable solution

As others said, it highly depends. However if you're on a platform with dynamic allocation then -1 is (extremely likely) a safe value.

That's because the memory allocator gives out memory in BIG BLOCKS instead of just single bytes^§. Therefore the last address that can be returned would be -block_size. For example if block_size is 4 then the last block will span across the addresses { -4, -3, -2, -1 }, and the last possible address will be -4 = 0xFFFF...FFFC. As a result, -1 will never be returned by the malloc() family

Various system functions on Linux also return -1 for an invalid pointer instead of NULL, for example mmap() and shmat(). Win32 APIs that return a handle can also return NULL (0) or INVALID_HANDLE_VALUE (-1) for a failure case or an ill-formed handle. They have to do that because sometimes NULL is a valid memory address. In fact if you're on a Harvard architecture then location zero in the data space is quite usable. And even on von Neumann architectures then what you said

"NULL is just the special memory address 0x0, which is never used for anything but to indicate the occurrence of a special condition"

is still wrong, because the address 0 is also valid. It's just that most modern OSes map the page zero somehow to make it trap when user space code dereferences it. Yet the page is accessible from within kernel code. There were some exploits related to NULL pointer dereference bug in Linux kernel

In fact, quite contrary to the zero page's original preferential use, some modern operating systems such as FreeBSD, Linux and Microsoft Windows actually make the zero page inaccessible to trap uses of NULL pointers. This is useful, as NULL pointers are the method used to represent the value of a reference that points to nothing

https://en.wikipedia.org/wiki/Zero_page

In MSVC and GCC, a NULL pointer to member is also represented as the bit pattern 0xFFFFFFFF on a 32-bit machine. And in AMD GCN NULL pointer also has a value of -1

You can go even further and return a lot more error codes by exploiting the fact that pointers are normally aligned. For example malloc always "aligns memory suitable for any object type (which, in practice, means that it is aligned to alignof(max_align_t))"

Nowadays the default alignment for malloc is 8 or 16 bytes depending on whether you're on a 32 or 64-bit OS, which means you'll have at least 3 bits available for error reporting or any purposes of yours. And if you use a pointer to a type wider than char then it's always aligned. So generally there's nothing to worry about unless you want to return a char pointer that's not output from malloc (in which case you can align easily). Just check the least significant bit to see whether it's a valid pointer or not

int* result = func();
if ((uintptr_t)result & 1)
    error_happened(); // now the high bits can be examined to check the error condition

In case of 16-byte alignment then the last 4 bits of a valid address are always 0s, and the total number of valid addresses is only ¹⁄₁₆ the total number of bit patterns, which means you can return at most ¹⁵⁄₁₆×2⁶⁴ error codes with a 64-bit pointer. Then there's aligned_alloc if you want more least significant bits.

That trick has been used for storing some information in the pointer itself. On many 64-bit platforms you can also use the high bits to store more data. See Using the extra 16 bits in 64-bit pointers

You can even go to the far extreme by limiting the range of the allocated pointers with some help from the OS. For example if you specify that the pointers must be allocated in the range 2-3GB then any addresses below 2GB and above 3GB will be available for you to indicate an error condition. On how to do that see:

Pointer address span on various platforms

8 Answers8

TL;DR:

See also

Linked