How to expand allocated memory in place

Question

Is there something like std::realloc but do nothing if it fails to expand the existing memory (of course it will let you know it failed)?

I mean truly expand in place. Not copy to a newly created memory chunk.

When std::vector fills to current capacity, it automatically expands capacity (typically doubling it). I'd say 'expanding' capacity is 'built in'. So, even if there was something like realloc, why would you use it? — 2785528, Dec 05 '17 at 02:31
@DOUGLASO.MOEN I can imagine a vector that tries to expand in place. Saves a lot of copies and some management. But then that only requires `std::realloc`. — Passer By, Dec 05 '17 at 02:49
What do you want to do that `std::realloc` doesn't achieve? Can you provide a use case? This sounds awfully like a [XY problem](https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem) — Passer By, Dec 05 '17 at 02:49
@PasserBy `std::realloc` might do a copy. It's a `C` function so it's not reliable to copy non-pod C++ type I think. — W.H, Dec 05 '17 at 02:59
It's not just that there's no C function for it, dynamic memory allocation simply can't guarantee that another object hasn't been allocated right being the target to be grown. You have to copy one of them in that situation. — lockcmpxchg8b, Dec 05 '17 at 06:20
The closest thing I can think of is to use mmap to allocate and http://man7.org/linux/man-pages/man2/mremap.2.html to reallocate. — technosaurus, Dec 05 '17 at 07:12

score 6 · Accepted Answer · answered Dec 05 '17 at 07:15

Contrary to what other answers say, the function you need could be provided by the C standard ibrary. It isn't for some reason or other.

It was considered for an adition to the C standard, see e.g. http://www.open-std.org/JTC1/SC22/wg14/www/docs/n1527.pdf (search for try_realloc). I don't know if the C committee is still pursuing this.

Bottom line, there's no such function as of now.

gsamaras · Answer 2 · 2017-12-05T06:17:49.503

2

No, there is no such function.

From your comments, I suppose that the following might be of interest to you: `std::realloc` for non-POD types and What is C++ version of realloc(), to allocate the new buffer and copy the contents from the old one?

_{PS: Re-approach your problem, since the existing methology is most usually enough.}

edited Dec 05 '17 at 06:17

answered Dec 05 '17 at 06:00

gsamaras

71,951
46
188
305

Basile Starynkevitch · Answer 3 · 2017-12-05T06:47:12.910

truly expand in place. Not copy to a newly created memory chunk.

This is impossible because of how virtual address space works on direct-addressing (non-segmented) instruction set architectures like x86-64 (or ARM, MIPS, RISCV, etc....).

^{(so that strong limitation is mostly not from software, but from your hardware, in the previous century some segmented architectures like iAPX432 or Rekursiv have been proposed, but failed)}

Let's suppose you want to realloc a pointer zone of one megabytes to get four megabytes of more memory. But the virtual address space of your process could contain some other data -for some other malloc-ed zone or something else like a code segment, a call stack, etc...- used two megabytes ahead. So you need to move your (growing) zone -by asking your OS for some fresh 4 megabyte zone- and copy from the previous one.

On Linux, look into proc(5) (e.g. type man 5 proc in a terminal) and into mmap(2). Run the cat /proc/self/maps and cat /proc/$$/maps commands in some terminal to get a better intuition about virtual address space. Be aware that all memory management is done above such mmap and related (sometimes, old sbrk) system calls. So your malloc or operator new is (sometimes) using them (most of the time, reusing previously free-d zone is preferable). Use strace(1) to explore the system calls done by existing programs.

BTW, on Linux, both your C standard library and your C++ standard library are free software. You could study their source code.

Read also Operating Systems: Three Easy Pieces to understand more how OSes work. Consider reading some book on Computer Architecture (or at least some slides on the simplified Y86 architecture). You've got some deep misunderstanding about how your computer works.

It is all very interesting but hardly relevant. The normal `realloc` *either* (1) reallocates the block without changing its address, grabbing and adjoining an unallocated chunk right after the block, *or* (2) moves the block to a new address if there happens to be not enough unallocated bytes right after the block. OP asks for a function that either does (1) or returns a null pointer and leaves the block alone. The C standard could very well provide this function, there is nothing theoretically impossible about it. Every implementation does (1) if it can. — n. m. could be an AI, Dec 05 '17 at 07:24

How to expand allocated memory in place

3 Answers3