21

I want a minimal o-damn-malloc-just-failed handler, which writes some info to a file (probably just standard error). I would prefer to use fprintf() rather than write(), but this will fail badly if fprintf() itself tries to malloc().

Is there some guarantee, either in the C standard, or even just in glibc that fprintf won't do this?

Adrian Ratnapala
  • 5,485
  • 2
  • 29
  • 39
  • IMPORTANT!!! In the HANDLER of malloc things, to stay safe with anything, you should do: a) re-register malloc handlers (all 4) to standard ones b) do your things about logging etc. c) do the real memory re/de/allocation as needed by the caller etc. d) restore YOUR handlers e) then continue, well, with exitting. This has an important advantage that you can still do whatever you need during the processing and don't have to worry about whether any of them is using malloc or not. I have once been doing it. File is too slow, I was using a socket. :) – Ethouris Jan 21 '15 at 16:12
  • Just to clarify, I have not actually registered any malloc handlers. I simply have some low-level functions which call `malloc()`, when the allocation fails, I want that function to print a log message. – Adrian Ratnapala Jan 21 '15 at 18:36
  • Ah! First, how are you able to continue the program if `malloc()` failed? I guess the only way for you to continue is to print a predefined message in a text and send it using `write()`. Generally, allocation failures in today software are rather qualifying as "call `abort()` on allocation failure". If this is for debugging/diagnostics, the coredump will tell you much more than any error message (unless you have demolished the stack, of course). – Ethouris Jan 22 '15 at 11:00
  • That's the exact motivation of my question: I want to reliably get a message out before exiting. I wanted to know if I could use `printf()` for that, whether I have to stick with `write()`. – Adrian Ratnapala Jan 23 '15 at 14:31
  • So, you should prepare for the worst - that is, `printf()` is likely to call malloc(). – Ethouris Jan 26 '15 at 08:12
  • Do not forget that malloc usually does not fail even if there is no memory left if overcommiting is on (which it usually is for unix/linux systems). – lalala Sep 22 '20 at 10:19

3 Answers3

27

No, there's no guarantee that it won't. However, most implementations I've seen tend to use a fixed size buffer for creating the formatted output string (a).

In terms of glibc (source here), there are calls to malloc within stdio-common/vfprintf.c, which a lot of the printf family use at the lower end, so I wouldn't rely on it if I were you. Even the string-buffer output calls like sprintf, which you may think wouldn't need it, seem to resolve down to that call, after setting up some tricky FILE-like string handles - see libio/iovsprintf.c.

My advice is to then write your own code for doing the output so as to ensure no memory allocations are done under the hood (and hope, of course, that write itself doesn't do this (unlikelier than *printf doing it)). Since you're probably not going to be outputting much converted stuff anyway (probably just "Dang, I done run outta memory!"), the need for formatted output should be questionable anyway.

(a) The C99 environmental considerations gives an indication that (at least) some early implementations had a buffering limit. From my memory of the Turbo C stuff, I thought 4K was about the limit and indeed, C99 states (in 7.19.6.1 fprintf):

The number of characters that can be produced by any single conversion shall be at least 4095.

(the mandate for C89 was to codify existing practice, not create a new language, and that's one reason why some of these mimimum maxima were put in the standard - they were carried forward to later iterations of the standard).

paxdiablo
  • 854,327
  • 234
  • 1,573
  • 1,953
  • Thanks for that, even if it's the answer I didn't want. And, stdio-common/vprintif.c was entertaining. IMHO, there's quite some impressive obfuscation going on there. – Adrian Ratnapala Jul 19 '11 at 06:50
  • "Most implementations I've seen tend to use a fixed size buffer for creating the formatted output string" - one wonders why. There is absolutely no need for using a buffer (other than the one of the file stream itself) in any of the `printf()` functions. (I've done it without, so I know what I'm talking about.) But the hint that one doesn't need much conversion here is a good one: Check if `puts()` does the trick for you. – DevSolar Jul 19 '11 at 07:06
  • @DevSolar, you generally need _some_ sort of buffer, such as for expanding ints and floats to their character representation. It doesn't have to be big, and it can be used piecemeal, once per %-item rather than once for the whole output (which would require a bigger buffer). The environmental limits of C99 indicates that there were implementations that had to be accounted for - see the update. – paxdiablo Jul 19 '11 at 08:35
  • 1
    @paxdiablo: Not necessarily. I [implemented](http://pdclib.rootdirectory.de) `printf()` using recursion, without any buffer. – DevSolar Jul 19 '11 at 08:44
  • Why am I not surprised to see you at the top of this question? :) Awesome answer! – Tim Post Jul 19 '11 at 10:14
  • @DevSolar: I've not seen your implementation, but I do point out that recursion is using a buffer -- that buffer is the stack. You'll still run into implementation limits. – Billy ONeal Jul 19 '11 at 14:44
  • @Billy ONeal: Since the recursion is per conversion specifier, the maximum recursion depth is well-defined (max. number of digits to be printed). The advantage over a static buffer is that the stack is only used when necessary. The advantage over `malloc()` is that it doesn't call `malloc()`. Advantage over both is that you don't run into problems because you've miscalculated the max. digits by one or two. There are disadvantages, sure, but implementation limits aren't one of them. If your machine can't recurse 20-deep, you're into trouble no matter what. ;-) – DevSolar Jul 19 '11 at 17:17
  • 1
    @Dev: I don't see how that differs to putting a 20 character wide chunk on the stack. I'm not saying that your solution is a bad one. I'm just saying there's a buffer that's got to be used somewhere. Recursion doesn't make that go away, it just makes it more insidious. – Billy ONeal Jul 19 '11 at 17:19
  • @Billy ONeal: I *do* invite you to have a look at my implementation. Main worker for integer conversions [here](http://pdclib.rootdirectory.de/trac.fcgi/browser/trunk/functions/_PDCLIB/print.c#L139) (putting a `intmax_t` and a pointer-to-struct on the stack, not a "20-character wide chunk"), and the structure definition [here](http://pdclib.rootdirectory.de/trac.fcgi/browser/trunk/internals/_PDCLIB_int.h#L334). I'm open to criticism. Mail is solar@. – DevSolar Jul 20 '11 at 07:48
  • 1
    @DevSolar: I'm not criticizing at all. I'm just saying there is a buffer in use. Your implementation places at least one `intmax_t` and one pointer on the stack for every digit in the integer. That is a buffer. Hiding it with recursion does not make the buffer go away. (And really, for integers greater than two digits, a plain `char` buffer would have used less space) – Billy ONeal Jul 20 '11 at 16:18
8

The C standard doesn't guarantee that fprintf won't call malloc under the hood. Indeed, it doesn't guarantee anything about what happens when you override malloc. You should refer to the documentation for your specific C library, or simply write your own fprintf-like function which makes direct syscalls, avoiding any possibility of heap allocation.

bdonlan
  • 224,562
  • 31
  • 268
  • 324
  • Thanks for that. Actually I am not overriding malloc(), just wrapping it. But the principle of your answer is the same. I will have to do a plain-old write(). – Adrian Ratnapala Jul 19 '11 at 06:52
4

The only functions you can be reasonably sure will not call malloc are those marked async-signal-safe by POSIX. Since malloc is not required to be async-signal-safe (and since it's essentially impossible to make it async-signal-safe without making it unusably inefficient), async-signal-safe functions normally cannot call it.

With that said, I'm nearly sure glibc's printf functions (including fprintf and even snprintf) can and will use malloc for some (all?) format strings.

R.. GitHub STOP HELPING ICE
  • 208,859
  • 35
  • 376
  • 711
  • Yes, but overriding/wrapping `malloc` is *always* illegal (UB), of course... :-) – R.. GitHub STOP HELPING ICE Jul 19 '11 at 15:57
  • R - what do you mean wrapping malloc is illegal? Perhaps you mean "wrapping malloc in another function called malloc". By wrapping I think of providing a new function that returns malloc'ed() memory, provides some other functionality too. – Adrian Ratnapala Jul 21 '11 at 11:28