Android linker relocating C functions for ARM assembly, but not for x86?

Question

I successfully changed the ARM library of an Android project into PIC (position independent code) because I wanted to make some unrelated fixes and Android only supports PIC libraries since Lollipop. (My fork's latest source is http://github.com/sleekweasel/Beebdroid)

Now I want to get x86 working too before I tidy up and cook a pull request (because it's in the original project) but my x86 is rather weaker than my ARM knowledge, and Android assembly articles only seem to handle ARM.

The two instructions I'm having problems with are LEA and CALL: they cause ld to emit

warning: shared library text segment is not shareable

If I comment them out then ./gradlew build happily links, but obviously the code doesn't work quite as well.

Here are snippets from the project - maybe they'll be clearer than my English description. I think this is complete context, since it's only the instructions and their interaction with the linker that's at issue:

app/src/main/jni/6502asm_x86.S:

.intel_syntax noprefix
.text
.global exec6502
.global acpu

exec6502:
    pusha
    # Keep CPU* in EBP
    lea  ebp,acpu  // Causes PIC to fail.
    // ... code removed ...
    lea   ebx, fns_asm // Causes PIC to fail.
    // ... code removed ...
    call do_poll_C // Causes PIC to fail.
    // ... code removed ...
    popa
    ret

// Lots of op-code implementations here - they have no effect on linking

// .section .rodata
.balign 4

fns_asm:
    .long 0                 // 0x00 BRK
    .long 0 - fns_asm + opasm_ora_indzx     // 0x01 ORA (,x)
    .long 0 - fns_asm + opasm_undef

app/src/main/jni/main.h

typedef struct M6502_struct M6502;

struct M6502_struct { ... };

void exec6502(M6502* cpu);
extern void do_poll_C(M6502*, int c);

app/src/main/jni/6502.c

M6502 acpu;
M6502* the_cpu = &acpu;

void do_poll_C(M6502* cpu, int c) {
...
}

app/src/main/jni/main.c

JNIEXPORT jint JNICALL Java_com_littlefluffytoys_beebdroid_Beebdroid_bbcRun(JNIEnv * env, jobject  obj)
{
  // Position independent code, hopefully!
  the_cpu->c_fns = &fns; // +40
  exec6502(the_cpu);
  return the_cpu->pc_trigger_hit;
}

app/src/main/jni/Android.mk

# ... stuff ...
ifeq ($(TARGET_ARCH),arm)
    LOCAL_CFLAGS +=  -march=armv6t2 -O9 -D_ARM_ -fPIC
    LOCAL_SRC_FILES := 6502asm_arm.S
endif
ifeq ($(TARGET_ARCH),x86)
    LOCAL_CFLAGS += -m32 -fPIC
    LOCAL_SRC_FILES := 6502asm_x86.S
endif
# ... stuff ...

app/src/main/jni/Application.mk

# The ARMv7 is significanly faster due to the use of the hardware FPU
APP_ABI := armeabi-v7a x86
APP_PLATFORM := android-16

ifneq ($(APP_OPTIM),debug)
  APP_CFLAGS += -O3 -fPIC
endif
APP_CFLAGS += -fPIC

LOCAL_SRC_FILES += \
    6502.c \
    main.c \
    and_more_files.c

I recall seeing somewhere that I need to rewrite LEA into a CALL to the next instruction, pop the return value off the stack, and then add the difference between the address of the CALL instruction and the target memory offset (also in the text segment): because only computed offsets within the text segment are needed, the linker is circumvented. (I only have one LEA to rewrite: to access a jump table of offsets within the assembly code - the other two will turn into an input parameter and a pointer in a block pointed to by that parameter, as I've done for ARM.)

I'm more confused by CALL to C functions not being handled by the linker, because ARM's loader is happy to relocate BL instructions. I already have -fPIC in the C compile flags. Adding .global do_poll_C makes no difference for that function and presumably the others. ARM doesn't need .global declarations for the external C functions.

I'm aware that I could pass in a block initialised with C function pointers - I started doing that for the ARM library but then found that the loader made that unnecessary. (I presume I could even add a C pointer to the assembly language table fns_asm, since C presumably finds the assembly exec6502 symbol.)

Do I need a function pointer block for x86, or is there some magic incantation I can use to ask the loader to handle my x86 CALL instructions, the way BL 'just works' with ARM?

Thanks for any help.

Answer 1

Like I suspected, you're calling across source file (from asm to C), but not across shared-object boundaries, so it shouldn't need a runtime relocation. Use __attribute__ ((visibility ("hidden"))) so the linker can resolve call do_poll_C at static link time, not having it participate in symbol interposition at dynamic-link time.

Or more simply, use -fvisibility=hidden -fvisibility-inlines-hidden to change the default, and only export the few symbols you need exported using attributes. https://gcc.gnu.org/wiki/Visibility

I'm assuming this works the same for Android as for GNU/Linux. If that's not the case, much of this answer will be useless!

---

IDK why ARM would be different; maybe Android/x86 isn't willing to do runtime relocations at all.

Under GNU/Linux (not Android), dynamic linking will do runtime fixups of 64-bit absolute addresses (or I think in 32-bit mode, 32-bit absolute or relative).

In 64-bit mode, call rel32 can't reach arbitrary 64-bit targets so the linker won't let you do that for symbols that need runtime relocation. But that doesn't apply to you; you are (for some reason) making obsolete 32-bit code with -m32.

---

lea ebp,acpu // Causes PIC to fail Nothing to do with LEA specifically, everything to do with the symbol reference as a 32-bit absolute address.

(Also, IDK why you'd use LEA there in the first place since you can't use x86-64 RIP-relative addressing; mov reg, imm32 is a more efficient way to put a 32-bit absolute address into a register: 1 byte shorter.).

Look at clang -m32 -fPIC code-gen for putting an address in a register: yes it uses call / pop to read EIP into a register and adds and offset to get a pointer to the GOT. Then it can address "private" static data relative to that. Check compiler output, or google, for the right syntax like lea ebp, [ebx + acpu@GOTOFF] or something, given ebx as your GOT pointer.

Difference between GOT and GOTOFF shows an example in AT&T syntax: leal .LC0@GOTOFF(%eax), %edx gets the address of the label .LC0 into a register, given EAX as a pointer to the GOT. It also includes gcc's code-gen for getting EAX pointing to the GOT.

It's a lot easier if you use x86-64 where you can just use RIP-relative LEA; then you don't need to jump through hoops to make PIC code. lea rbp, [RIP + acpu] references the symbol relative to the current address.