577

I have read about GCC's Options for Code Generation Conventions, but could not understand what "Generate position-independent code (PIC)" does. Please give an example to explain me what does it mean.

Zoe
  • 27,060
  • 21
  • 118
  • 148
Narek
  • 38,779
  • 79
  • 233
  • 389

5 Answers5

701

Position Independent Code means that the generated machine code is not dependent on being located at a specific address in order to work.

E.g. jumps would be generated as relative rather than absolute.

Pseudo-assembly:

PIC: This would work whether the code was at address 100 or 1000

100: COMPARE REG1, REG2
101: JUMP_IF_EQUAL CURRENT+10
...
111: NOP

Non-PIC: This will only work if the code is at address 100

100: COMPARE REG1, REG2
101: JUMP_IF_EQUAL 111
...
111: NOP

EDIT: In response to comment.

If your code is compiled with -fPIC, it's suitable for inclusion in a library - the library must be able to be relocated from its preferred location in memory to another address, there could be another already loaded library at the address your library prefers.

Drew Dormann
  • 59,987
  • 13
  • 123
  • 180
Erik
  • 88,732
  • 13
  • 198
  • 189
  • 52
    This example is clear, but as a user what will be the difference if I create a shared labrary (.so) file without the option? Are there some cases that without -fPIC my lib will be invalid? – Narek Mar 15 '11 at 12:25
  • 29
    Yes, building a shared library that isn't PIC could be an error. – John Zwinck Mar 15 '11 at 12:31
  • 115
    To be more specific, the shared library is supposed to be shared between processes, but it may not always be possible to load the library at the same address in both. If the code were not position independent, then each process would require its own copy. – Simon Richter Mar 15 '11 at 12:36
  • 5
    The problem is that, as I know, all the addresses written in they example above are virtual addresses and the will be other when this library will be loaded in a memory. So one process can call the same function from A address and the second process can call the same function from the same A address, or can copy the lib into a memory once again and call the function from B address. What is the problem? Where the error should occur without -fPIC? – Narek Mar 15 '11 at 12:51
  • 2
    I mean that as the addresses in above example are virtual and they should be loaded into the real memory, then they can be loaded anywhere and they will work properly, isn't it? – Narek Mar 15 '11 at 13:09
  • 21
    @Narek: the error occurs if one process wants to load more than one shared library at the same virtual address. Since libraries cannot predict what other libraries could be loaded, this problem is unavoidable with the traditional shared library concept. Virtual address space doesn't help here. – Philipp Mar 15 '11 at 13:32
  • 5
    Sorry to bring up an old thread, but @SimonRichter... So do you mean that if my shared lib is not compiled with -fPIC it will simply be a memory inefficiency since it will need to be loaded by each process? Or will there be some cases where it doesn't work. I recently took on a Linux development project at work that has been somewhat ignored/shelved for awhile. We haven't seen any problems using shared libraries without -fPIC, but want to know if we could in the future. The shared lib is currently used by only one process on the system. – Kyle Preiksa Aug 27 '15 at 23:08
  • 8
    @KylePreiksa, on i386 it will merely be inefficient (because the dynamic linker can relocate the library, but the modified memory image needs to be tracked by the kernel (unmodified pages can be discarded when memory is tight, and reloaded from the original file), so you are wasting memory. On other architectures, linking the shared library fails because there is no valid way to express how the relocation would have to be performed. For example, PowerPC only has a 24 bit relative branch instruction that cannot be used between different DLLs, and -fPIC uses two instructions instead. – Simon Richter Aug 28 '15 at 00:05
  • @SimonRichter thank you for the explanation, especially why it works on some architectures and not others. We are x86 exclusively so I won't worry too much, but will try to optimize in a future version! – Kyle Preiksa Aug 28 '15 at 00:20
  • 3
    @SimonRichter I will ask in different way. When *not* to use -fPIC? – fider Jan 21 '16 at 13:23
  • 15
    You can omit `-fPIC` when compiling a program or a static library, because only one main program will exist in a process, so no runtime relocation is ever necessary. On some systems, programs are still made [position independent](https://en.wikipedia.org/wiki/Position-independent_code#Position-independent_executables) for enhanced security. – Simon Richter Jan 22 '16 at 15:22
  • @SimonRichter sorry to wake up this thread again ... Even if are sure there is only one dynamic library loaded, -fPIC is necessary because of Address Layout Randomization mecanism, right ? – Hugo Mar 03 '18 at 18:17
  • @Hugo, yes, also I'm not sure how the linker would react to a library that cannot be relocated. – Simon Richter Mar 03 '18 at 19:25
  • Would the second example (non-PIC) not work because the process linking this lib might be using those **virtual addresses**? is that what you meant in your examples? – Tony Tannous Oct 09 '18 at 23:17
  • @SimonRichter Thanks for when `-fPIC` *can* be omitted. When *shall* it be omitted? – alx - recommends codidact Aug 04 '19 at 11:44
  • 3
    @CacahueteFrito, on some machines, position independent code comes with a performance penalty. Intel CPUs before the 686 are notorious for this: no PC-relative addressing modes, and few registers, so programs would use a relative subroutine call to push the current program counter onto the stack, pop it into a register and address data items relative to that. A variant of that is calling a `pop; push; ret` sequence (slower locally, but doesn't confuse the branch predictor as much). – Simon Richter Aug 05 '19 at 12:51
  • 3
    Modern CPUs generally have PC-relative addressing and also lots of registers, so the performance impact is a lot less pronounced. – Simon Richter Aug 05 '19 at 12:52
83

I'll try to explain what has already been said in a simpler way.

Whenever a shared lib is loaded, the loader (the code on the OS which load any program you run) changes some addresses in the code depending on where the object was loaded to.

In the above example, the "111" in the non-PIC code is written by the loader the first time it was loaded.

For not shared objects, you may want it to be like that because the compiler can make some optimizations on that code.

For shared object, if another process will want to "link" to that code it must read it to the same virtual addresses or the "111" will make no sense. But that virtual-space may already be in use in the second process.

Błażej Michalik
  • 4,474
  • 40
  • 55
Roee Gavirel
  • 18,955
  • 12
  • 67
  • 94
  • 2
    `Whenever a shared lib is loaded, the loader changes some addresses in the code depending on where the object was loaded to.` I think this is not correct if compiled with -fpic and the reason why -fpic exists i.e. for performance reasons or because you have a loader that's not able to relocate or because you need multiple copies in different locations or for many more reasons. – robsn Apr 15 '20 at 14:16
  • 9
    Why not always use -fpic? – Jay Apr 21 '20 at 06:22
  • 12
    @Jay - because it will required one more calculation (the function address) for each function call. So performance-wise, if not needed it is better not to use it. – Roee Gavirel Apr 21 '20 at 06:41
59

Code that is built into shared libraries should normally be position-independent code, so that the shared library can readily be loaded at (more or less) any address in memory. The -fPIC option ensures that GCC produces such code.

Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278
  • 1
    Why wouldn't a shared library be loaded at any address in memory without having the `-fPIC` flag on? is it not linked to the program? when the program is running, the operating system uploads it to memory. Am I missing something? – Tony Tannous Oct 09 '18 at 23:13
  • 1
    Is the `-fPIC` flag used, to ensure this lib can be loaded to any **virtual address** in the process that is linking it? sorry for double comments 5 minutes elapsed can't edit previous one. – Tony Tannous Oct 09 '18 at 23:20
  • 2
    Distinguish between building the shared library (creating `libwotnot.so`) and linking with it (`-lwotnot`). While linking, you don't need to fuss about `-fPIC`. It used to be the case that when building the shared library, you needed to ensure `-fPIC` was used for all the object files to be built into the shared library. The rules may have changed because compilers build with PIC code by default, these days. So, what was critical 20 years ago, and might have been important 7 years ago, is less important these days, I believe. Addresses outside the o/s kernel are 'always' virtual addresses'. – Jonathan Leffler Oct 09 '18 at 23:27
  • So previously you had to add the `-fPIC`. Without passing this flag, the generated code when building the .so needs to be loaded to specific virtual addresses that might be in use? – Tony Tannous Oct 09 '18 at 23:34
  • 1
    Yes, because if you didn't use the PIC flag, the code wasn't reliably relocatable. Things like ASLR (address space layout randomization) aren't possible if the code is not PIC (or, at least, are so hard to achieve that they are effectively impossible). – Jonathan Leffler Oct 09 '18 at 23:37
27

Adding further...

Every process has same virtual address space (If randomization of virtual address is stopped by using a flag in linux OS) (For more details Disable and re-enable address space layout randomization only for myself)

So if its one exe with no shared linking (Hypothetical scenario), then we can always give same virtual address to same asm instruction without any harm.

But when we want to link shared object to the exe, then we are not sure of the start address assigned to shared object as it will depend upon the order the shared objects were linked.That being said, asm instruction inside .so will always have different virtual address depending upon the process its linking to.

So one process can give start address to .so as 0x45678910 in its own virtual space and other process at the same time can give start address of 0x12131415 and if they do not use relative addressing, .so will not work at all.

So they always have to use the relative addressing mode and hence fpic option.

Community
  • 1
  • 1
Ritesh
  • 1,809
  • 1
  • 14
  • 16
  • 1
    Thanks for the virtual addr explanation. – Hot.PxL Dec 22 '14 at 07:59
  • 2
    Can anyone explain how this is not a problem with a static library, why you don't have to use -fPIC on a static library? I understand that the linking is done in compile time (or right after actually), but if you have 2 static libraries with position dependent code, how are they going to be linked? – Michael P Apr 20 '16 at 23:12
  • 3
    @MichaelP object file has a table of position depended labels and when particular obj file is linked all labels are updated accordingly. This cannot be done to shared library. – Slava Jan 18 '17 at 17:54
16

A minor addition to the answers already posted: object files not compiled to be position independent are relocatable; they contain relocation table entries.

These entries allow the loader (that bit of code that loads a program into memory) to rewrite the absolute addresses to adjust for the actual load address in the virtual address space.

An operating system will try to share a single copy of a "shared object library" loaded into memory with all the programs that are linked to that same shared object library.

Since the code address space (unlike sections of the data space) need not be contiguous, and because most programs that link to a specific library have a fairly fixed library dependency tree, this succeeds most of the time. In those rare cases where there is a discrepancy, yes, it may be necessary to have two or more copies of a shared object library in memory.

Obviously, any attempt to randomize the load address of a library between programs and/or program instances (so as to reduce the possibility of creating an exploitable pattern) will make such cases common, not rare, so where a system has enabled this capability, one should make every attempt to compile all shared object libraries to be position independent.

Since calls into these libraries from the body of the main program will also be made relocatable, this makes it much less likely that a shared library will have to be copied.

user1016759
  • 169
  • 1
  • 3