13

Is the O_LARGEFILE flag needed if all that I want to do is write a large file (O_WRONLY) or append to a large file (O_APPEND | O_WRONLY)?

From a thread that I read titled "Cannot write >2gb index file" on the CLucene-dev mailing list, it appears that O_LARGEFILE might be needed to write large files, but participants in that discussion are using O_RDWR, not O_WRONLY, so I am not sure.

Vadim Kotov
  • 8,084
  • 8
  • 48
  • 62
Daniel Trebbien
  • 38,421
  • 18
  • 121
  • 193

2 Answers2

18

O_LARGEFILE should never be used directly by applications. It's to be used internally by the 64-bit-offset-compatible version of open in libc when it makes the syscall to the kernel (Linux, or possibly another kernel with this 64-bit-offset-mode-is-a-second-class-citizen nonsense). Just make sure to always include -D_FILE_OFFSET_BITS=64 in your CFLAGS and you'll never have to worry about anything.

R.. GitHub STOP HELPING ICE
  • 208,859
  • 35
  • 376
  • 711
  • 2
    Do you have any source for that O_LARGEFILE should not be used? – dmeister Jan 12 '12 at 12:20
  • The best source is the fact that grepping POSIX for it turns up zero results. – R.. GitHub STOP HELPING ICE Jan 12 '12 at 13:38
  • So you mean, O_LARGEFILE is something like a cap ? It must be some reason why this flag exists. – kirugan Jun 29 '13 at 05:41
  • 1
    On 32-bit systems, the version of `open` used with `-D_FILE_OFFSET_BITS=64` passes `O_LARGEFILE` to the kernel transparently. The version used with 32-bit `off_t` does not. This in turn determines "the offset maximum established in the open file description", in the language of POSIX, which causes a number of functions to report errors when they would result in a file offset that can't be represented in the 32-bit `off_t`. The maximum is associated with the open file description rather than the process since open file descriptions can be shared between processes. – R.. GitHub STOP HELPING ICE Jun 29 '13 at 05:59
  • In any case, there's absolutely no reason to poke at `O_LARGEFILE` yourself. If you're using 64-bit `off_t` on a 32-bit system, it will be set automatically for you, and otherwise (on 64-bit systems, or on 32-bit systems with 32-bit `off_t`), it should not be set. – R.. GitHub STOP HELPING ICE Jun 29 '13 at 06:00
  • Does POSIX mention `_FILE_OFFSET_BITS`? Can you link to it? – Ciro Santilli OurBigBook.com Aug 14 '17 at 07:37
  • No, it's a glibc thing. Sane systems always use 64-bit offsets. POSIX allows multiple environments and the getconf command can tell you the CFLAGS for a particular one. – R.. GitHub STOP HELPING ICE Aug 14 '17 at 10:53
12

IIRC if you do

#define _LARGEFILE_SOURCE
#define _FILE_OFFSET_BITS 64

before all other includes you do not need to pass this flag.

additionally see

caf
  • 233,326
  • 40
  • 323
  • 462
codymanix
  • 28,510
  • 21
  • 92
  • 151