3

I am using tricks to store extra information in pointers, At the moment some bits are not used in pointers(the highest 16 bits), but this will change in the future. I would like to have a way to detect if we are compiling or running on a platform that will use more than 48 bits for pointers.

related things: Why can't OS use entire 64-bits for addressing? Why only the 48-bits? http://developer.amd.com/wordpress/media/2012/10/24593_APM_v2.pdf

The solution is needed for x86-64, Windows, C/C++, preferably something that can be done compile-time. Solutions for other platforms are also of interest but will not marked as correct answer.

Community
  • 1
  • 1
Jim Hansson
  • 132
  • 7
  • At the moment I have asserts in debug mode that checks that pointers return from malloc is not using more than 48 bits but those asserts are turned of in release. I would like to have a solution that does one check at startup or compiletime instead. – Jim Hansson Oct 01 '14 at 11:18
  • 1
    If you want compile-time tests for processor-feature, use cpuid 80000008H, see the intel manual. – Deduplicator Oct 01 '14 at 11:25
  • How much memory does your application need? Less than 2GB? – Deduplicator Oct 01 '14 at 11:31
  • 1
    "preferably something that can be done compile-time" - so you know you'll *only* run on the computer you compile on? From [wikipedia](http://en.wikipedia.org/wiki/X86-64#Architectural_features) "48 bits are used in current implementations" / "architecture definition allows this limit to be raised in future implementations to the full 64 bits" so any hack you use may break on later hardware/OS. Is it just `new`-ed memory you care about, or must you support pointers to local and global variables, shared memory, `malloc`-ed memory etc.? Anyway, which *compiler* you're using is essential info. – Tony Delroy Oct 01 '14 at 11:35
  • @Deduplicator I need a lot more than 2GB(but less than 256TB at the moment). Time check out intels documentation on cpuid instruction. – Jim Hansson Oct 01 '14 at 11:41
  • @TonyD did not think about that, You are right it need to be a runtime check, at startup. I only have this problem with malloc:ed pointers. Today we are using Visual studio:s but we are looking at others. – Jim Hansson Oct 01 '14 at 11:44
  • *Never* trick pointers by storing extra information in them. If absolutely needed, request huge range of addresses from the OS and handle them differently. – Basile Starynkevitch Oct 02 '14 at 14:41
  • @BasileStarynkevitch _never_say_never_, This trick was needed to compress the datasets I am working on, into something that fits into the available RAM(for speed), Are you telling me to segment the memory into different areas and use the base address for the area instead of type-tagging the pointers? – Jim Hansson Oct 03 '14 at 00:18
  • A little more background info, I am using a union to store either a double, int(with a optional null flag), or a few different types of pointers to other types of data into a 64-bits area. and instead of having a enum on the side to tell me what this union represent, I use this trick to get that all into those 64bits. the first version used a struct+enum instead of the union, and resulted in only being able to load a little part of the datasets, so we have tried simplier solutions before. But segmenting the memory based on type has not been tried yet. and is something I shall look more into. – Jim Hansson Oct 03 '14 at 00:36

1 Answers1

4

Windows has exactly one switch for 32bit and 64bit programs to determine the top of their virtual-address-space:

IMAGE_FILE_LARGE_ADDRESS_AWARE

For both types, omitting it limits the program to the lower 2 GB of address-space, severely reducing the memory an application can map and thus also reducing effectiveness of Address-Space-Layout-Randomization (ASLR, an attack mitigation mechanism).

There is one upside to it though, and just what you seem to want: Only the lower 31 bits of a pointer can be set, so pointers can be safely round-tripped through int (32 bit integer, sign- or zero-extension).

At run-time the situation is slightly better:

Just use the cpuid-instruction from intel, function EAX=80000008H, and read the maximum number of used address bits for virtual addresses from bits 8-15.

The OS cannot use more than the CPU supports, remember that intel insists on canonical addresses (sign-extended).

See here for how to use cpuid from C++: CPUID implementations in C++

Community
  • 1
  • 1
Deduplicator
  • 44,692
  • 7
  • 66
  • 118
  • The first claim contradicts [this post](http://stackoverflow.com/questions/15510715/setpeflags-image-file-large-address-aware-in-64-bit-delphi-application) which claims 64 bits processes are by definition large address aware. – MSalters Oct 01 '14 at 11:45
  • @MSalters: It's not by definition, but by default, according to MS. Added source. – Deduplicator Oct 01 '14 at 11:46
  • Thanks, that's indeed a subtle but relevant difference. – MSalters Oct 01 '14 at 11:50