Diagnosing Slow Write Speed Issue

Question

So I hope someone can help me with this question which might seem obvious to anyone with any amount of programming experience but not to me! Please excuse my poor descriptions since I don't know the right terminology for a lot of things.

Basically I've inherited a project using an old(ish) computer which has a framegrabber card installed (Matrox Meteor II). My basic aim is to simply capture live video in real time from a camera and write it uncompressed to the hard disk of the computer. Programming for the card is done in C and makes use of a library which came bundled with it called the Matrox Imaging Library (MIL).

Currently I am just writing all my images in raw as a bit array and using imageJ to view them. I've written a basic program which uses double buffering, so an image is captured to a buffer then written to the text file while the next image is captured to another buffer. I will put a code snippet below, all of the M-prefix commands are from the MIL library. You don't actually need to understand them to help me, however.

I just use the following:

/* Put the digitizer in asynchronous mode. */
MdigControl(MilDigitizer, M_GRAB_MODE, M_ASYNCHRONOUS);

/* Grab the first buffer. */
MdigGrab(MilDigitizer, MilImage[0]);

/*open my file where I will write the images*/
pfile = fopen("myfile.txt", "wb");

/* Process one buffer while grabbing the other. */
while( !kbhit() )
  {

  /* Grab second buffer while processing first buffer. */
  MdigGrab(MilDigitizer, MilImage[1]);

  /* Synchronize and start the timer. */
  if (NbProc == 0)
     {
     MappTimer(M_TIMER_RESET, M_NULL);
     }

  /* Process the first buffer already grabbed.  */

  /*copy the image to a display buffer*/
  #if COPY_IMAGE 
  MbufCopy(MilImage[0], MilImageDisp);
  #endif      

  /*Here we copy the buffer into a user supplied array*/
  MbufGet(MilImage[0], my_array);

  /* Then we write the array into the text file using fwrite*/
  fwrite(my_array,sizeof(char),sizeof(my_array),pfile);

    /*Output the amount of time it took to process that image*/
    MappTimer(M_TIMER_READ, &Time);
Capture_Time = Time - Time_Holder;
printf("IMG %s took %.7f .\n", text, Capture_Time);
Time_Holder = Time;
    /* Count processed buffers.  */
     NbProc++;         


  /* Grab first buffer while processing second buffer. */
  MdigGrab(MilDigitizer, MilImage[0]);

  /* Process the second buffer already grabbed.  */

  #if COPY_IMAGE 
  MbufCopy(MilImage[1], MilImageDisp);
  #endif      




  /*Here we copy the buffer into our array*/
  MbufGet(MilImage[1], my_array);

  /* Then we write the array into the text file using fwrite*/
  fwrite(my_array,sizeof(char),sizeof(my_array),pfile);

  MappTimer(M_TIMER_READ, &Time);
Capture_Time = Time - Time_Holder;
printf("IMG %s took %.7f .\n", text, Capture_Time);
Time_Holder = Time;


  /* Count processed buffers. */
  NbProc++;         
  }
  getchar();   
   /* Wait last grab end and stop timer. */
   MdigGrabWait(MilDigitizer, M_GRAB_END);

   /*Close our file*/
   fclose(pfile);

My problem is that I would like to be doing the above at a frame rate of 24fps, but I find that the process of capturing images and writing them to the hard disk is slightly slower than that sometimes (the slowdown is inconsistent but the program seems to occasionally hang on one frame for ~500ms).

I was wondering if anyone knew of a way to diagnose what causes these slowdowns, and whether it might be the write speed of my hard disk.

Your program hangs when the file system cache is filled to capacity and needs to wait for the slow disk to catch up. You can only get ahead by compressing the video with an encoder or getting a faster disk. Encoding video is of course entirely normal. You'll need to go shopping. — Hans Passant, May 28 '14 at 12:06
Thanks so much for your reply, the disk I currently have is a 7200 RPM SATA with an 8GB Cache. From looking online it seems that the write speed should be much greater than the 10.008 MB/S I think that I need. I do absolutely believe that this is the issue but do you know a way of confirming for sure before I go and start spending money? — afid_difa, May 28 '14 at 12:36
You might be able to make the process stall for shorter time by forcing a flush to disk after each image is captured. — stark, May 28 '14 at 13:15
The cache doesn't mean anything, you'll quickly fill it up. The rotational speed sets a hard upper limit on how fast you can write. But the degree to which the file data on the drive is fragmented is the dominating factor. It forces head seeks and those completely kill throughput. A heavy randomizer of course, it never gets better over time unless you babysit the drive. — Hans Passant, May 28 '14 at 15:31

Nikita Chopra · Answer 1 · 2016-02-25T22:11:39.770

This will help you fix the occasional hanging at single frame by enabling Disk Write Caching . You can go to C: drive-> Properties->Hardware->Select Drive-> Drive Properties -> Policies-> Enable write caching on the device.

Other modification that can be done on the code is corresponding to first line of your code where you put the digitizer in ASYNCHRONOUS mode instead of that use MdigControl(milDigitizer,M_GRAB_MODE,M_ASYNCHRONOUS_QUEUED); This will continue after each grab is queued and doesn't wait for Mil grab function to exit .

Diagnosing Slow Write Speed Issue

1 Answers1