Error Encountered while Dynamic Memory allocation in C

Question

In the below code, line[] array contains names of all image files contained in a folder. We are just reading the names dynamically and sending the file names one by one to a function function_foo as given in the code. When there are 8 images the code is executing as expected. But as soon as we increase the number of image files in the folder to any number above 8, the process gets terminated and returns -1. Please suggest.

FILE **fp;
struct Structure_Name data[100];
fp = malloc( sizeof(FILE *) * 10000);

for(i=0;i<total; i++)
{
  sprintf(fName,"/home/souvik/Images/%s",line[i]);
  printf("%s\n",fName);
  fp[i] = fopen(fName, "rb");
  data[i]=function_foo(fp[i],(data[i]));
}

show us the complete code (mini-version in case of large code), and more importantly, what have you tried? (something like google about gdb, valgrind ) — abasu, Mar 09 '16 at 08:41
just reading multiple image file from a folder using gcc linux. The function_foo returns the rows and column data of an image. Structure_Name is a structure for holding the image data. — Souvik Kundu, Mar 09 '16 at 08:47
Is there are reason why you aren't using an array? `FILE* fp_arr = malloc( sizeof(FILE*[n]) );` — Lundin, Mar 09 '16 at 09:38
What is the value of 'total' etc.? Not what you think it is, what it actually is - from your debugger.:) — Martin James, Mar 09 '16 at 09:38
You are not giving us enough - read again what @abasu wrote above. — Martin James, Mar 09 '16 at 09:44
Hi James, total = number of files in a directory; line[ ] contains the names of the files that are there in the directory. — Souvik Kundu, Mar 09 '16 at 09:47
yes the file names are correct and NULL check has also been done, see the Update given by iila below.. — Souvik Kundu, Mar 09 '16 at 10:53

score 2 · Answer 1 · edited May 23 '17 at 12:15

2

It is a really bad idea to open 10000 files at the same time. I'd recommend you to change this logic: to read content of these files one by one, and to send this content to your function function_foo. But currently your code tries to fopen very big number of files without closing of it. I expect it is the main problem (i.e. it seems that in your environment maximal number of files that can by opened is 8).

See also:

Update:

Since you don't need an array of all pointers, you can try following version. Do you see the same problem in this case?

FILE *fp;
struct Structure_Name data[100];

for(i=0; i<total; i++)
{
  sprintf(fName,"/home/souvik/Images/%s", line[i]);
  printf("%d of %d, %s\n", i, total, fName);
  fp = fopen(fName, "rb");
  if (fp != NULL)
  {
    data[i]=function_foo(fp,(data[i]));
    fclose(fp);
  } else
  {
    printf("A null pointer is returned by fopen()");
  }
}

edited May 23 '17 at 12:15

Community

1
1

answered Mar 09 '16 at 08:48

Ilya

4,583
4
26
51

Now I have included fclose(fp[i]) , after calling the function function_foo inside the loop. That means now the files are closed after reading is done. But the same problem persists. Again, by reducing the value 10000 to 10 the program is able to open only 2 files. Any suggestion ? – Souvik Kundu Mar 09 '16 at 09:23
Ok, I updated the answer (and added an information about `total`, requested by Martin James). Please tell us, do you see the same problem. – Ilya Mar 09 '16 at 10:17
The same error : total=11 0 of 11, /home/souvik/Images/LPF_ChangeR_-4.pgm 1 of 11, /home/souvik/Images/LPF_ChangeR_3.pgm 2 of 11, /home/souvik/Images/LPF_ChangeR_-5.pgm 3 of 11, /home/souvik/Images/LPF_ChangeR_5.pgm 4 of 11, /home/souvik/Images/LPF_ChangeR_-3.pgm 5 of 11, /home/souvik/Images/LPF_ChangeR_1.pgm 6 of 11, /home/souvik/Images/LPF_ChangeR_4.pgm 7 of 11, /home/souvik/Images/LPF_ChangeR_-1.pgm 8 of 11, /home/souvik/Images/LPF_ChangeR_0.pgm 9 of 11, /home/souvik/Images/LPF_ChangeR_2.pgm Process returned -1 (0xFFFFFFFF) execution time : 42.209 s Press ENTER to continue. – Souvik Kundu Mar 09 '16 at 10:29
2

I'll suggest use valgrind and check, if it is running for 42.209 s, there's some heavyweight things being done, have you tried some simple things like, instead of function_foo, do a fseek and print the size, see, if you are getting the data or not, try to triangulate the problem – abasu Mar 09 '16 at 10:32
all the data upto data[9] are obtained and the the program returns -1 while fetching the 10th file name (i.e. when i = 10 in the loop ). Any suggestion ? – Souvik Kundu Mar 09 '16 at 11:01
@SouvikKundu in your code the array `data` contains only 100 elements, but the array `fp` contains 10000 elements. And after it you call `function_foo(fp[i],(data[i]))` (i.e. with the same index `i`). I expect that you have similar mistakes somewhere else in this program (and it might be the reason of breaking of something unexpected in the memory). – Ilya Mar 09 '16 at 11:11

score 0 · Answer 2 · answered Mar 17 '16 at 10:45

@ALL, Thanks for your active help.

Now the problem is solved. The root cause of the problem was that, while the program was looping through all the images it was storing the image data in data[i] , i.e. data[1], data[2], data[3], data[4]......... the data is temporarily stored in \var folder during program execution. after reading 9 image files the total size of 9 images (data[1]+data[2]+data[3]+..... data[9]) exceeds the size of \var in our machines. We simply de-allocated the space allocated for the structure variable data[i] after each iteration. This solved the problem.

Error Encountered while Dynamic Memory allocation in C

2 Answers2