Linker error on relocating a program above 2GB in x86_64 linux?

Question

I have a user program which normally compiles to have an entry point at 0x400460 which I have to relocate to have an entry point starting at within 2GB of the shared libraries loaded in Linux. e.g

linux-vdso.so.1 =>  (0x00007fff109cd000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fcd195e6000)
lib64/ld-linux-x86-64.so.2 (0x00007fcd199af000)

I am using gcc command line argument -Wl,-Ttext=0x80000000 to specify the start address for the .text segemnt.

The issue is that when I am giving an address above 2GB in this argument I am getting a linker error which is:

gcc test.c -ggdb -Wl,-Ttext=0x80000000 -o test1
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 0 has invalid symbol index 10
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 1 has invalid symbol index 11
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 2 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 3 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 4 has invalid symbol index 10
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 5 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 6 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 7 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 8 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 9 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 10 has invalid symbol index 11
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 11 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 12 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 13 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 14 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 15 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 16 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 17 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 18 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 19 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 20 has invalid symbol index 20
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_line): relocation 0 has invalid symbol index 2
/usr/lib/x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x12): relocation truncated to fit: R_X86_64_32S against symbol `__libc_csu_fini'   defined in .text section in /usr/lib/x86_64-linux-gnu/libc_nonshared.a(elf-init.oS)
/usr/lib/x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x19): relocation truncated to fit: R_X86_64_32S against symbol `__libc_csu_init' defined in .text section in /usr/lib/x86_64-linux-gnu/libc_nonshared.a(elf-init.oS)
/usr/lib/x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x20): relocation truncated to fit: R_X86_64_32S against symbol `main' defined in    .text section in /tmp/ccFshK69.o
/var/services/homes/adabral/elider/gc/a1/lib/gcc/x86_64-unknown-linux-  gnu/4.8.2/crtbegin.o: In function `deregister_tm_clones':
crtstuff.c:(.text+0x8): relocation truncated to fit: R_X86_64_32S against  `.tm_clone_table'
 /var/services/homes/adabral/elider/gc/a1/lib/gcc/x86_64-unknown-linux- gnu/4.8.2/crtbegin.o: In function `register_tm_clones':
 crtstuff.c:(.text+0x38): relocation truncated to fit: R_X86_64_32S against  `.tm_clone_table'
 collect2: error: ld returned 1 exit status

: The reason for this as far as I can make out after going through several SO questions and forums can be that some of the sections are still mapped to low 2GB address space.

This is the output of the readelf -a for a binary compiled with text segment just below 2GB (at 0x79990000).

Dynamic section at offset 0x190310 contains 24 entries:
Tag        Type                         Name/Value
0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
0x000000000000000c (INIT)               0x4003f0
0x000000000000000d (FINI)               0x79990204
0x0000000000000019 (INIT_ARRAY)         0x79b902f8
0x000000000000001b (INIT_ARRAYSZ)       8 (bytes)
0x000000000000001a (FINI_ARRAY)         0x79b90300
0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
0x0000000000000004 (HASH)               0x400278
0x0000000000000005 (STRTAB)             0x400318
0x0000000000000006 (SYMTAB)             0x4002a0
0x000000000000000a (STRSZ)              72 (bytes)
0x000000000000000b (SYMENT)             24 (bytes)
0x0000000000000015 (DEBUG)              0x0
0x0000000000000003 (PLTGOT)             0x79b904e8
0x0000000000000002 (PLTRELSZ)           72 (bytes)
0x0000000000000014 (PLTREL)             RELA
0x0000000000000017 (JMPREL)             0x4003a8
0x0000000000000007 (RELA)               0x400390
0x0000000000000008 (RELASZ)             24 (bytes)
0x0000000000000009 (RELAENT)            24 (bytes)
0x000000006ffffffe (VERNEED)            0x400370
0x000000006fffffff (VERNEEDNUM)         1
0x000000006ffffff0 (VERSYM)             0x400360
0x0000000000000000 (NULL)               0x0

You can see that the INIT and some other sections still starts at the low 2GB address space. So the dynamic linker can not offset the relocation address at run time since the relocation type is R_X86_64_32.

So I tried compiling my code with gcc -mcmodel=large flag but I am still getting the same linker error. Using the large model should have rectified this error but it is not.
I am stuck at this point, any help is highly appreciated.

I am working on x86_64 ubuntu. gcc version 4.8.2

"I have to relocate to have an entry point starting at within 2GB of the shared libraries loaded in Linux. e.g" -- there is *absolutely* no guarantee that shared libraries will be loaded where you expect them to load (I expect the load addresses to change drastically after `echo 1 > /proc/sys/vm/legacy_va_layout`). Besides, why on earth would you want to be within 2GB of the shared library? — Employed Russian, Jan 16 '14 at 04:21
@EmployedRussian I am working on some academic project which requires this. — abhi, Jan 16 '14 at 05:09

score 8 · Accepted Answer · answered Jan 15 '14 at 22:49

8

You should first understand that the ABI for x86_64 has several different "models": small, kernel, medium, and large. These are described under the GCC -mcmodel option here: http://gcc.gnu.org/onlinedocs/gcc/i386-and-x86-64-Options.html

What you're experiencing is that crt1.o, the startup code that gets linked into every program that's responsible for taking the initial ELF register/stack state and passing them into the libc startup code which eventually calls main, seems to be using the small model. You can see here:

/usr/lib/x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x20): relocation truncated to fit: R_X86_64_32S against symbol `main' defined in    .text section in /tmp/ccFshK69.o

What's happening is that crt1.o has a relocation for the address of main that only allows for a 32-bit address to be filled in. (Note: Even if main were defined in a shared library rather than the executable, there would be a PLT entry in the executable, and the address of this PLT entry would be the official address of main to which the relocation would be resolved.)

To solve this problem, you would need a crt1.o that can handle full 64-bit addresses. One way to get this might be by using Scrt1.o, which is normally only used for PIE executables, instead of crt1.o. You could probably achieve this with -nostartfiles and manually specifying all the start files on the link command line. It might be worth filing a bug report against glibc requesting that the x86_64 crt1.o be converted to "large model" so that it works with main programs not linked in the 32-bit range.

Note that you'll probably also need -mcmodel=large (or perhaps -fPIE would work) for all of your own code to make it link and work correctly at high addresses. This could make it considerably larger and slower. You might want to rethink why you're doing this and whether you really need to.

answered Jan 15 '14 at 22:49

R.. GitHub STOP HELPING ICE

208,859
35
376
711

Thanks a lot for the answer. Is it possible to do the other way round i.e can I force the shared libraries like libc.so and other to load in the initial 2Gb address space while using the small model? Thanks again – abhi Jan 16 '14 at 21:09
Not easily. What is the motivation? I may have an alternative solution depending on what you need. – R.. GitHub STOP HELPING ICE Jan 17 '14 at 00:27
I am actually trying to access the shared library function in a different way then it is currently being done(using indirection through PLT and GOT table). – abhi Jan 18 '14 at 00:57
You want to be able to make a direct call to it via a relocation in the code (text) segment? This might work if you use the large model with a full 64-bit field for the relocation. – R.. GitHub STOP HELPING ICE Jan 18 '14 at 02:18
Yes I am making the text section writable and updating the resolved address directly at the call ins. I want to do it with the small model that is why I am trying to move the program near the shared libraries. – abhi Jan 18 '14 at 02:28
This is for a research project I am working on, we are trying to see if doing this would improve upon the no of instruction executed – abhi Jan 22 '14 at 00:50
2

@abhi: Just for the record, in the years since this comment thread, modern gcc has `gcc -fno-plt` to use `call [func@GOTPCREL]` with a RIP-relative addressing mode, which isn't *quite* as good as `call rel32`, but is definitely better than `mov r64, imm64` / `call reg` you'd get from `-mcmodel=large`! See the bottom of [32-bit absolute addresses no longer allowed in x86-64 Linux?](https://stackoverflow.com/a/46493456) for a mention of `-fno-plt` (most of the answer is about PIE executable.) – Peter Cordes Mar 06 '21 at 02:51

Linker error on relocating a program above 2GB in x86_64 linux?

1 Answers1

Linked