0

I have a very large (~1E9) array of objects that I malloc, realloc, and free on iterations of a single thread program.

Specifically,

//(Individual *ind)

//malloc, old implementation
ind->obj = (double *)malloc(sizeof(double)*acb->in.nobj); 

//new implementation
ind->obj = (double *)a_allocate(acb->in.nobj*sizeof(double));

void *a_allocate (int siz){
   void *buf;

   buf = calloc(1,siz);
   acb->totmemMalloc+=siz;

   if (buf==NULL){
      a_throw2("a_allocate...failed to allocate buf...<%d>",siz);
   }

   return buf;
}

...

//realloc
ind->obj = (double *)a_realloc(ind->obj, acb->in.nobj*sizeof(double));

void *a_realloc (void *bufIn,int siz)
{
   void *buf   = bufIn;

   if (buf==NULL){
      a_throw2("a_realloc called with null bufIn...");
   }

   buf = realloc(buf,siz);

   return buf;
}

...

//deallocate
free(ind->obj);

The other three dozen properties are processed similarly.

However, every few test runs, the code fails a heap validation on the deallocation of only this object property (the free() statement). At the time of failure, the ind->obj property is not null and has some valid value.

Is there any obvious problem with what I'm doing?

I'm very new to C and am not entirely sure I'm perform the memory operations correctly.

Thanks!

EDIT: using _CRTLDBG_REPORT_FLAG

HEAP[DEMO.exe]: Heap block at 010172B0 modified at 010172E4 past requested size of 2c
Dan
  • 93
  • 8
  • 1
    You're probably dealing with memory fragmentation. http://www.design-reuse.com/articles/25090/dynamic-memory-allocation-fragmentation-c.html – xaxxon Feb 15 '16 at 06:53
  • Could you please make it a minimal, complete and verifiable example. – Martin Zabel Feb 15 '16 at 06:53
  • 1
    First of all, in C you [should not cast the result of `malloc` et. al.](http://stackoverflow.com/questions/605845/do-i-cast-the-result-of-malloc) (or other functions returning `void *`). Then you *do* know that it's okay to call [`realloc`](http://en.cppreference.com/w/c/memory/realloc) with a `NULL` pointer (it's equivalent to calling `malloc`)? – Some programmer dude Feb 15 '16 at 06:53
  • 1
    Your `a_realloc` function probably should check against failure of `realloc` – Basile Starynkevitch Feb 15 '16 at 06:54
  • It might be operating system specific. Perhaps port your program to Linux and use [valgrind](http://valgrind.org/) – Basile Starynkevitch Feb 15 '16 at 07:01
  • @xaxxon Is this possible when the program only uses around 50MB of memory with 60GB free? – Dan Feb 15 '16 at 07:01
  • 1e9 * sizeof(double) = 8 gigabytes. – xaxxon Feb 15 '16 at 07:03
  • @xaxxon sorry, I wasn't clear. I've reduced the test case to only 50 elements and the problem persists. – Dan Feb 15 '16 at 07:05
  • @BasileStarynkevitch wouldn't realloc return NULL only if buf is NULL? – Dan Feb 15 '16 at 07:08
  • @JoachimPileborg Yes to your question. I've changed my implementation. – Dan Feb 15 '16 at 07:11
  • No, [realloc](http://en.cppreference.com/w/c/memory/realloc) can fail (and probably would when you reach your memory resource limits). On failure, `realloc` gives `NULL` and sets `errno` – Basile Starynkevitch Feb 15 '16 at 07:17

1 Answers1

2

Heap validation is a delayed metric. The Visual Studio debug heap can be used (debug build) with more frequent checks Microsoft : Debug Heap flags.

Alternative using application verifier and turning on heap checking, will help find the point which is causing this.

           +-----+----------+-----+  +----+-------------+-----+
           | chk | memory   |chk2 |  | chk| different m | chk2|
           +-----+----------+-----+  +----+-------------+-----+

When the system allocates memory, it puts meta- information about the memory before the returned pointer (or maybe after). When these memory pieces get overwritten, then that causes the heap failure.

This may be the memory you are freeing, or the memory which was directly before hand.

Edit - to address comments

A message such as "HEAP[DEMO.exe]: Heap block at 010172B0 modified at 010172E4 past requested size of 2c"

Implies that the memory at 01017280 wrote beyond the end of the allocated memory.

This could be because the amount malloced/realloced was too small, or an error in your loops.

+---+-----------------+----+--------------------------+
|chk|d0|d1|d2|d3|d4|d5|chk2| memory                   |
+---+-----------------+----+--------------------------+

So if you tried to write into d6 above, that would cause 'chk2' to be overwritten, which is being detected. In this case the difference is small - requested size is 0x2c and the difference = E4 - B0 = 0x34

Turning on these debug checks should change your code to be more crashing and predictable. If there is no randomness in your data, then turn off ASLR (only for debugging) and the addresses being used will be predictable, you can put a breakpoint in the malloc/realloc for a given memory address.

mksteve
  • 12,614
  • 3
  • 28
  • 50