0

I am making an data logger project. In my data logger, I have 5 sensors like: temperature, humidity, accelerometer, light and barometer. Here, I am storing the data in to SD card in binary format. I'm storing accelerometer @3200HZ and all other sensors at 1HZ.

While writing, I'm adding a special character like $,& to each sensor measurements to identify the data while reading. I am not adding any spl character for accelerometer data. Accelerometer data is huge data with 3200HZ so I'm not adding spl character as it consumes some memory also it making delay while writing leading to miss out some samples and able to achive only 2600HZ for 3200HZ.

My reading data code looks like below:

while((nr = fgetc(logFile)) != EOF)
{
    if (nr == '$')
    {      // read temp data if nr = $
        fread(&temp_read,sizeof(float),1,logFile);
        pc.printf("\r\n %f",temp_read);
    }
    if (nr == '&')
    {    // read humidity data if nr = &
        fread(&Humidity_read,sizeof(float),1,logFile);
        pc.printf("\r\n %f",Humidity_read);
    }
    else
    {           // if nr is not a spl character
        data1 = nr;// send nr to a variable.
        nr = fgetc(logFile);// read next byte and save it in nr
        data2 = nr;// send nr to another variable
        int16_t temp = (data1 | (data2 << 8));// club both bytes to form                   int16_t data 
        pc.printf("\r\n %i",temp);// print the one axis 
    }
}

Unfortunately this program giving wrong data. There is huge difference. Is there any another way to solve this problem. Where I'm making errors. Thank you in advance.

LPs
  • 16,045
  • 8
  • 30
  • 61
Ghani
  • 51
  • 1
  • 7
  • 3
    You know, that these *special* characters are not really special is represented in binary? If you want read binary data, first of all get read of the text-oriented functions like `fgetc`. Next think of a way to properly delimit your data. Hint: Fixed width/byte stuffing/textual encoding. – Eugene Sh. Jan 30 '17 at 15:26
  • 2
    My first question is: are you sure the SPI is reading and writing correctly to the SD card? Have you tried a very low speed read and write of dummy data to make sure even that is working? – DiBosco Jan 30 '17 at 15:26
  • 1
    First of all: please indent your code. – Jabberwocky Jan 30 '17 at 15:53
  • 2
    Please show the code that opens the file: (`logfile = fopen(...);`). – Jabberwocky Jan 30 '17 at 15:54
  • See also @Gandhi history of the project: [Gandhi's Arduino Data Logger](https://www.google.com/search?q=stackoverflow+c%2B%2B+gandhi+sensor+&ie=utf-8&oe=utf-8) – Thomas Matthews Jan 30 '17 at 15:57
  • Be aware that [`fgetc`](http://en.cppreference.com/w/cpp/io/c/fgetc) returns an `int`, which allows it return 16-bits. I recommend using `fread` instead. – Thomas Matthews Jan 30 '17 at 16:00
  • Are all the write synchronized? If writing of data are not synchronized then there is a possibility of data being corrupted while writing. – army007 Jan 30 '17 at 16:47
  • 2
    Eugene above is correct. If you write accelerometer data using binary, you WILL WRITE all kind of characters to the SD, included $s and &s. Then when you read those data back, you can no more separate things. Playing with bits of accelerometer data, if 16 are more than enough, could help. – linuxfan says Reinstate Monica Jan 30 '17 at 17:04
  • @ThomasMatthews `fgets()` returns an `int` in the range of `unsigned char` or `EOF`. Unclear about your concern with "it return 16-bits.". – chux - Reinstate Monica Jan 30 '17 at 17:27
  • The problem is likely that data is not _written_ clearly. Post that code too. – chux - Reinstate Monica Jan 30 '17 at 17:28
  • Smart people without English as a first language might struggle with your unnecessary abbreviations. Smart people who might otherwise be able to help you. – Clifford Jan 30 '17 at 18:58
  • Sampling at 3200Hz would suggest an unusually high bandwidth (1.6KHz) for a typical MEMS accelerometer. What is the device you are using? Sampling >2x the bandwidth gains you nothing. – Clifford Jan 30 '17 at 19:05
  • This follows a [previous question](http://stackoverflow.com/questions/41809849/binary-reading-and-writing-in-c). – Weather Vane Jan 30 '17 at 19:35

1 Answers1

2

You are probably sweating the small stuff with your attempts to capture the high-speed samples consistently through modifying the data format. The problem is almost certainly associated with block-latency in the SD card (extra time taken to write when a flash page or sector boundary is crossed).

You may need to take a number of approaches in combination:

  • Use an RTOS with the sampling performed in a high priority task and the logging deferred to a low priority task
  • Send the samples to the logging task in a queue of sufficient length to cover the number of samples acquired during maximum SD latency.
  • Carefully select your SD card; especially if using an SPI rather then SDIO interface. I have found cards with sustained write speeds on between 32k and 800k bytes per second, and block latency exceeding 400ms.

See How can I use an SD card for logging 16-bit data at 48 ksamples/s?

Community
  • 1
  • 1
Clifford
  • 88,407
  • 13
  • 85
  • 165