14

If do the next:

int* array = malloc(10 * sizeof(int));

and them I use realloc:

array = realloc(array, 5 * sizeof(int));

On the second line (and only it), can it return NULL?

Novak
  • 2,760
  • 9
  • 42
  • 63
  • somewhat related: http://stackoverflow.com/questions/1736433/can-realloc-move-pointer-if-new-size-smaller?rq=1 –  Aug 25 '12 at 20:26

7 Answers7

15

Yes, it can. There are no implementation guarantees on realloc(), and it can return a different pointer even when shrinking.

For example, if a particular implementation uses different pools for different object sizes, realloc() may actually allocate a new block in the pool for smaller objects and free the block in the pool for larger objects. Thus, if the pool for smaller objects is full, it will fail and return NULL.


Or it may simply decide it's better to move the block

I just used the following program to get size of actually allocated memory with glibc:

#include <stdlib.h>                                                          
#include <stdio.h>                                                           

int main()                                                                   
{                                                                            
    int n;                                                                   

    for (n = 0; n <= 10; ++n)                                                
    {                                                                        
        void* array = malloc(n * sizeof(int));                               
        size_t* a2 = (size_t*) array;                                        

        printf("%d -> %zu\n", n, a2[-1]);                                    
    }                                                                        
}

and for n <= 6, it allocates 32 bytes, and for 7-10 it is 48.

So, if it shrank int[10] to int[5], the allocated size would shrink from 48 to 32, effectively giving 16 free bytes. Since (as it just has been noted) it won't allocate anything less than 32 bytes, those 16 bytes are lost.

If it moved the block elsewhere, the whole 48 bytes will be freed, and something could actually be put in there. Of course, that's just a science-fiction story and not a real implementation ;).


The most relevant quote from the C99 standard (7.20.3.4 The realloc function):

Returns

4 The realloc function returns a pointer to the new object (which may have the same value as a pointer to the old object), or a null pointer if the new object could not be allocated.

'May' is the key-word here. It doesn't mention any specific circumstances when that can happen, so you can't rely on any of them, even if they sound obvious at a first glance.


By the way, I think you could consider realloc() somewhat deprecated. If you'd take a look at C++, the newer memory allocation interfaces (new / delete and allocators) don't even support such a thing. They always expect you to allocate a new block. But that's just a loose comment.

Greenbeard
  • 508
  • 5
  • 14
Michał Górny
  • 18,713
  • 5
  • 53
  • 76
  • 8
    I must object to calling `realloc` deprecated just because C++ doesn't have an analogue in the new/delete regime. C++ is a very different language from C, and in particular, support for moving objects in C++ would require some way for the implementation to notify an object that it's being relocated and allow it to update its own internal references. C on the other hand doesn't automate or encapsulate any of this, so it's up to the caller (and thus perfectly fine) to be responsible for whether the object's contents need to be changed after `realloc`. – R.. GitHub STOP HELPING ICE Aug 25 '12 at 21:47
  • 13
    Generally, I find it a bit strange to reply to a C question with C++ code and C++ thoughts about deprecation. – Jens Gustedt Aug 25 '12 at 22:16
  • 1
    Indeed, I hadn't even bothered to read the code... That should really be fixed since this question is about C, not C++. – R.. GitHub STOP HELPING ICE Aug 25 '12 at 23:30
  • 1
    I'd prefer [malloc_usable_size](http://man7.org/linux/man-pages/man3/malloc_usable_size.3.html) over a negative index. – Alcaro Feb 11 '19 at 15:06
  • This looks incorrect to me, because there is memory available, which is the current allocation. If realloc() fails just because it wants to move memory somewhere else, then it's incorrect for it to return ENOMEM. It's not lack of memory, it's something else. – hdante Jun 05 '20 at 19:23
7

The other answers have already nailed the question, but assuming you know the realloc call is a "trimming", you can wrap it with:

void *safe_trim(void *p, size_t n) {
    void *p2 = realloc(p, n);
    return p2 ? p2 : p;
}

and the return value will always point to an object of size n.

In any case, since the implementation of realloc knows the size of the object and can therefore determine that it's "trimming", it would be pathologically bad from a quality-of-implementation standpoint not to perform the above logic internally. But since realloc is not required to do this, you should do it yourself, either with the above wrapper or with analogous inline logic when you call realloc.

R.. GitHub STOP HELPING ICE
  • 208,859
  • 35
  • 376
  • 711
  • Out of curiosity, is it practically useful? I'd consider that if `realloc()` just failed, there will be soon a random failing `malloc()`… – Michał Górny Aug 25 '12 at 20:54
  • 3
    Yes, I believe it is. Code that's trimming storage for an existing result may not be able to "back out" its progress on failure and report failure in a meaningful way to the higher-level code. Thus, it's very valuable to be able to write the code in a way that it *cannot fail*. Even if the next call to `malloc` is going to fail somewhere else, that will (in a robust program, at least) be at a point where the program can handle the failure case, back out any partial work, and report the error. – R.. GitHub STOP HELPING ICE Aug 25 '12 at 21:08
  • Is the old pointer still valid if realloc fails? I was under the impression that it frees it first, but I could be wrong. – mk12 Aug 25 '12 at 23:13
  • 3
    Yes, of course it is. If it weren't, `realloc` would be utterly useless in robust programs. This is actually an extremely common form of memory leak (i.e. `p=realloc(p,newsize);` which loses the old memory if `realloc` fails). – R.. GitHub STOP HELPING ICE Aug 25 '12 at 23:28
  • @R..: Yes, you're right—I see that now after reading the man page for `realloc(3)`. I've just fixed a leak in my code where I didn't free the old memory when realloc fails (although it's not extremely important since I call `exit` right after). – mk12 Aug 27 '12 at 03:42
  • 1
    @R..: Is there reason why an even-remotely-decent-quality implementation that would be unable to usefully reduce the size of an allocation shouldn't simply ignore the request? The Standard makes no attempt to forbid "conforming" implementations that are of such poor quality as to be useless, but I see no reason programmers targeting good implementations should cater to the quirks of abysmal ones. – supercat Jul 16 '18 at 17:55
  • 1
    @supercat: Considering it again, if the only way an implementation could reduce the size is by leaving the remainder permanently unusable, or unusable for the lifetime of the object, it would arguably be a higher-quality behavior to report failure to the caller and let it know it can still use the full original size rather than hiding the extra space. – R.. GitHub STOP HELPING ICE Jul 16 '18 at 18:29
  • @R..: The one *realistic* situation in which a quality allocator should fail to shrink an object would occur when the amount of contiguous space that would be freed up would be insufficient to hold a new allocation. If code repeatedly tries to shrink an allocation by one byte, some operations would succeed and some would fail, but the final size of the object would likely be within 16 or maybe 32 bytes of the last requested size. – supercat Jul 16 '18 at 19:28
  • 1
    @supercat: That's not a realistic situation, it's a nonsensical one. If allocation granularity is, for example, 32 bytes, trying to resize an object of size 32 down to size 31 should not fail. There is no benefit whatsoever to having it do so. On the other hand, if an object has size 10000000 and you want to resize it down to size 10, but no memory is available in zones that support partitioning into small sizes, there is value in reporting failure rather than succeeding just by leaving the caller an object that wastes ~10MB of unusable space. – R.. GitHub STOP HELPING ICE Jul 16 '18 at 20:34
  • @R..: Some platforms may have an allocation granularity which is smaller than the size of an allocation header. A *quality* implementation which is asked to resize a 10000000 byte object to 10 bytes *should always succeed*. If an implementation that has a granularity of 4 bytes but uses 8-byte headers is asked to shrink a 10000000-byte allocation to 9999996 bytes, it would be reasonable for it to leave the allocation at 100000000 bytes. Even if asked to shrink it to 9999992 bytes, it might not be unreasonable for it to decide that the value of creating space for a zero-sized object... – supercat Jul 16 '18 at 21:05
  • ...would be unlikely to be sufficient to justify the effort. If the following object gets freed before the object that got realloc'ed, the zero-sized allocation might get consolidated into the resulting free space, and there may be some scenarios where that could end up being useful, but for the most part it would be better not to add such tiny chunks to the free list. – supercat Jul 16 '18 at 21:08
  • @supercat: You're not making sense, and mixing up implementation details which are not visible to the application with properties that have severe effects on the application. – R.. GitHub STOP HELPING ICE Jul 16 '18 at 23:12
  • @R..: On a quality implementation, following a `realloc()` that shrinks a block, the block should be left with a size close to what's requested. If the block isn't shrunk very much, the application shouldn't care about whether the block was actually shrunk. – supercat Jul 17 '18 at 02:46
  • Return `NULL` does not always mean failure. If `n==0`, it is possible that the old memory is freed and no longer valid. – 12431234123412341234123 Nov 27 '20 at 13:40
  • @12431234123412341234123: Per the spec that's false, though many erroneous implementations did/do it. – R.. GitHub STOP HELPING ICE Nov 27 '20 at 15:03
  • @R..GitHubSTOPHELPINGICE Can you quote the part of the standard that forbids that? I found this in 7.22.3 Memory management functions - 1: _If the size of the space requested is zero, the behavior is implementation-defined: either a null pointer is returned, or the behavior is as if the size were some nonzero value, except that the returned pointer shall not be used to access an object._ – 12431234123412341234123 Nov 27 '20 at 15:07
  • @12431234123412341234123: 7.22.3.5 ¶4: "If memory for the new object cannot be allocated, the old object is not deallocated and its value is unchanged." However this was a topic of much controversy; see [DR400](http://www.open-std.org/jtc1/sc22/wg14/www/docs/summary.htm#dr_400) and [glibc issue 12547](https://sourceware.org/bugzilla/show_bug.cgi?id=12547). Note that the result of DR 400 allows the wrong behavior for future and deprecates `realloc` with size 0. – R.. GitHub STOP HELPING ICE Nov 27 '20 at 16:41
  • Note further that current POSIX (see https://pubs.opengroup.org/onlinepubs/9699919799/functions/realloc.html) does not allow the wrong behavior; I'm not sure if future is allowing it. – R.. GitHub STOP HELPING ICE Nov 27 '20 at 16:41
3

The language (and library) specification makes no such guarantee, just like it does not guarantee that a "trimming" realloc will preserve the pointer value.

An implementation might decide to implement realloc in the most "primitive" way: by doing an unconditional malloc for a new memory block, copying the data and free-ing the old block. Obviously, such implementation can fail in low-memory situations.

AnT stands with Russia
  • 312,472
  • 42
  • 525
  • 765
3

Don't count on it. The standard makes no such provision; it merely states "or a null pointer if the new object could not be allocated".

You'd be hard-pressed to find such an implementation, but according to the standard it would still be compliant.

cnicutar
  • 178,505
  • 25
  • 365
  • 392
  • I believe you shouldn't be calling such an implementation *braindead*. It may be actually *more optimal*. – Michał Górny Aug 25 '12 at 20:31
  • @MichałGórny In my language "more optimal" is considered a pleonasm, so I would avoid saying it. But yes, I edited :-) – cnicutar Aug 25 '12 at 20:32
  • @cnicutar: For some reason, some people who write implementations seem to regard "clever" and "stupid" as antonyms. – supercat Jul 17 '18 at 03:20
2

I suspect there may be a theoretical possibility for failure in the scenario you describe.

Depending on the heap implementation, there may be no such a thing as trimming an existing allocation block. Instead a smaller block is allocated first, then the data is copied from the old one, and then it's freed.

For instance this may be the case with bucket-heap strategy (used by some popular heaps, such as tcmalloc).

valdo
  • 12,632
  • 2
  • 37
  • 67
  • It would still be *valid* to just return the original pointer in this case. Whether that's more helpful, I'm not sure. Reporting the error is more informative and allows the caller to make the choice to use the existing oversized allocation, but it also has a high chance of breaking bad code that assumes a "trimming" `realloc` never fails. – R.. GitHub STOP HELPING ICE Aug 25 '12 at 21:44
  • 1
    tcmalloc can fail if it shrinks, check the source `tcmalloc.cc` function `do_realloc()` which is used in `tc_realloc()`, (https://github.com/gperftools/gperftools/blob/master/src/tcmalloc.cc#L1446) – 12431234123412341234123 Aug 21 '17 at 15:19
  • @R..: The Standard really should define some standard macros to indicate how an implementation will behave in various corner cases, and allow code to refuse to run on wacky implementations, rather than requiring extra code to handle cases that shouldn't arise on quality implementations. Even better might be to have a more generalized allocation-control function with a parameter to indicate whether an allocation is *expected* to grow or shrink, and indicate whether relocation is acceptable. Implementations would not be allowed to ignore information about expectations, and have requests... – supercat Jul 16 '18 at 21:32
  • ...to expand a block if possible without relocation always fail, but quality implementations that optimize based upon information and requests they receive may be able to outperform those that don't. In any case, programmers targeting good implementations shouldn't have to bend over backward to accommodate bottom-of-the-barrel ones. – supercat Jul 16 '18 at 21:34
2

A bit late, but there is at least one popular implementation which realloc() with a smaler size can fail: TCMalloc. (At least as far as i understand the code)

If you read the file tcmalloc.cc, in the function do_realloc_with_callback(), you will see that if you shrink enough (50% of alloced memory, otherwise it will be ignored), TCMalloc will alloc the new memory first (and possible fail) and then copy it and remove the old memory.

I do not copy the source code, because i am not sure if the copyrights (of TCMalloc and Stackoverflow) will allow that, but here is a link to the source (revision as at May 17, 2019).

rici
  • 234,347
  • 28
  • 237
  • 341
-1

realloc will not fails in shrinking the existing memory, so it will not return NULL. It can return NULL only if fails during expansion.

But shrinking can fail in some architecture, where realloc can be implemented in a different manner like allocating a smaller size memory separately and freeing the old memory to avoid fragmentation. In that case shrinking memory can return NULL. But its very rare implementation.

But its better to be in a safer side, to keep NULL checks after shrinking the memory also.

rashok
  • 12,790
  • 16
  • 88
  • 100
  • Is this implementation guaranteed? Or can an implementation still try and *move* the allocated memory (e.g. a "free" and "malloc") on a realloc and thus fail? –  Aug 25 '12 at 20:18
  • 2
    So then the statement *"will not fail"* is incorrect/misleading :) –  Aug 25 '12 at 20:23
  • In some RTOS architecutre, realloc can be implemented by free and malloc(smallersize) for avoiding fragmentation. – rashok Aug 25 '12 at 20:23
  • 2
    (I'm just pointing out that your first two sentences and the rest of the answer disagree. This is why it does not have any up-votes .. it either *can* fail or *will never* fail. Pick one.) –  Aug 25 '12 at 20:25
  • If an architecture tries to shrink then it will not fail, but if it do malloc(smallersize) and free(oldblock) then it can fail(but this type of implementation is very rare). – rashok Aug 25 '12 at 20:26
  • What if you shrink it to zero? Then it will return NULL. – Trevor Hickey Feb 28 '17 at 16:45