0

I have a 64bit OS and I am building a 64bit program that uses rip-relative addressing. My problem is that I cannot allocate using .lcomm directive more than 2GB of data. Is there a way to allocate more than 2GB in gas or do I have to use a different directive?

.lcomm array,2*1024*1024*1024 
#>=2GB doesn't work
#error:additional relocation overflows omitted from the output
user0042
  • 7,917
  • 3
  • 24
  • 39
BinaryBurst
  • 139
  • 5
  • Does `.lcomm` fail by itself, or only when you try to use the label from something that's linked after the buffer? – Peter Cordes Nov 01 '17 at 10:45
  • 2
    Architectural limitation, no doubt, x86_64 displacements are limited to a 32-bit signed value. So there is simply no way to address that much data with RIP relative addressing. – Hans Passant Nov 01 '17 at 10:52
  • 1
    Consider allocating the array at runtime using either the `mmap` system call or the `malloc` function from libc. – fuz Nov 01 '17 at 11:06
  • The problem isn't strictly speaking the large array, but the all instructions that generate relocation overflows. You can solve your problem by changing those instructions so they're not using 32-bit relocations (ie. don't use RIP-relative addressing for these instructions), or changing where the array is located memory so the 32-bit relocations don't refer to things on the opposite side of the 2G array. The later is what is Peter Cordes is suggesting in his answer, but a simpler way to do this is to dynamically allocate the array at runtime as fuz suggests above. – Ross Ridge Nov 01 '17 at 18:24

1 Answers1

1

RIP-relative displacements are signed 32-bit. The default code model is "small", where all static code/data is in the low 2GB of virtual address space, so a RIP-relative addressing mode or a rel32 branch / call can reach any symbol from anywhere. (Or for small PIC, everything is within 2GB at most but you let ASLR put it anywhere in memory.)

Check the x86-64 System V ABI for how to use code models other than the default "small". Where is the x86-64 System V ABI documented?.

If it's only just this one giant array, you should probably use some kind of linker script or something to arrange for it to be the only thing that extends outside the low 2G, so everything else can still assume the small code model. i.e. put it at the end of the BSS above everything else, so the array symbol is RIP-relative addressable from code anywhere in the low 2G, along with all other labels.

But the end of the array can be out of reach without causing a problem, because you get the base address into a register and index it. You'll only get a linker error if you write something like array[1ULL<<32] = 1; in C and the compiler is using the small code model so it will emit asm like movb $1, array+1<<32 (%rip) which of course won't work.

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
  • So i use malloc inside c++ and then send the pointer to assembly. Using rip relative addressing mode works at least to 10 gb – BinaryBurst Nov 02 '17 at 05:03
  • 1
    @BinaryBurst: huh, yes, of course you can pass 64-bit pointers to dynamically allocated memory. What does RIP-relative addressing have to do with that, though? Are you passing the pointer by storing it in a static variable instead of as a function arg? – Peter Cordes Nov 02 '17 at 05:25
  • no, no, I just thought that the code model wouldn't let me address more than 2GB but it seems that it works, somehow :) I just described what I was doing so that people know why I chose this answer :) – BinaryBurst Nov 02 '17 at 05:29
  • 1
    @BinaryBurst: but using rip-relative for *what*? If you're using RIP-relative to access a single pointer value, then you're totally missing the point. RIP-relative only has to reach the memory location where the pointer is stored (`&ptr`, not `ptr` in C). The pointer itself is 64 bits and can address the entire virtual address space; storing it in static storage does nothing. Or do you mean that you did what my answer suggested and put your array last so you can use a RIP-relative `lea` to get a pointer to the start, and then 64-bit indexing into it? – Peter Cordes Nov 02 '17 at 05:32
  • And BTW, yes the small code only restricts the size of code + *static* data, not stack or dynamic allocation. You still use 64-bit pointers for that. When you don't even need 64-bit pointers, you can use the `x32` aka ILP32 ABI where pointers are 32-bit. This reduces cache footprint for pointer-heavy data structures. – Peter Cordes Nov 02 '17 at 05:38
  • So i get a pointer from malloc inside the C++ code. I store it in a 64 bit variable from the assembly code. From within the assembly code I use rip relative addressing mode as follows: `lea r8,[rip+asm_array_pointer] ; mov r8,[r8]` – BinaryBurst Nov 02 '17 at 05:41
  • 1
    @BinaryBurst: using `malloc` isn't what my answer describes. If you arrange your binary the right way, all your labels are in the low 2GB but the end of your static array in the BSS can extend beyond that. I think this works; but it would break if anything wants the address of the end of the BSS and uses a RIP-relative LEA or a `mov` imm32 to get it. – Peter Cordes Nov 02 '17 at 05:41
  • Anyway, I'm glad my answer helped and got you to think of a workaround that does work, even if you didn't want to use my solution to the problem. The extra level of indirection is not optimal, but probably not a measurable difference unless you need to reload the pointer often, but not often enough that it stays hot in cache. – Peter Cordes Nov 02 '17 at 05:43