0

I am working on a project, it reads data(24bits per sample) from binary files, decompress it to 32bits and convert it signed int. I have this function for the decompress and convert tasks, and it causes wasting a lot of time in the project for huge data.

signed int convertedSample(QByteArray samp){
QString sample = QString(samp.toHex() + "00");
bool ok;
signed int decimalSample = sample.toInt(&ok, 16);
return decimalSample;
}

To read data from files and store it, I do:

while(!file.atEnd()){
QByteArray chunkSamples = file.read(chunkSize);// chunkSize here is the size of 1 million of samples
for (int i = 0; i < chunkSamples.size() ; i++){
QByteArray samp = chunkSamples.mid(i * 3, 3); // 24bits
buffer.append(convertedSample(samp)); // buffer is a QVector of int
}

Any help to make it faster ?

A.A.
  • 125
  • 1
  • 11
  • 5
    If you want to read three bytes, and store them in a 32-bit `int` value. Just read the three bytes, and use bitwise shifting and OR to combine the individual bytes into an (*unsigned*) `int`. Don't go through strings, that will always be slow. – Some programmer dude Feb 07 '23 at 11:30
  • 1
    With that said, reading from files isn't very often realtime-sensitive so why do you need to "optimize" it? What are your requirements and limitations in regards to this? And just as [`while (!stream.eof())` is always wrong](https://stackoverflow.com/questions/5605125/why-is-iostreameof-inside-a-loop-condition-i-e-while-stream-eof-cons), so is `while(!file.atEnd())`. – Some programmer dude Feb 07 '23 at 11:38
  • thanks @Someprogrammerdude , so using strings causes the slow here. how I can do the shifting and combine here from `QBytearray` to `unsigned int` ? – A.A. Feb 07 '23 at 11:47
  • 1
    Also avoid the `QByteArray::mid` call. It will create a useless temporary object. Access directly the bytes in `chunkSamples`. – prapin Feb 07 '23 at 11:51
  • 1
    Please take out a pen and some paper. Draw a box, divided into four parts. That represents the resulting 32-bit integer. Add labels on the divisions and ends, like `0`, `8`, `16`, `24` and `32`. Now to get one byte into the first part, from `0` to `7`, how much do you need to shift it? Then from `8` to `15`, how much do you need to shift it? And so on. The answer is already in the numbers you have written down. :) – Some programmer dude Feb 07 '23 at 11:57
  • 1
    Oh and remember that bits are numbered in the reverse order, so `32` should be on the left side, and `0` on the right. That will also tell you the shift-direction you need to use. – Some programmer dude Feb 07 '23 at 11:58
  • @Someprogrammerdude thanks, I tried this `signed int test = samp.toHex().toInt(&ok, 16) << 8;` that's a correct I guess ? – A.A. Feb 07 '23 at 13:03
  • That is still a useless round-trip to and from a string. You can call `toInt` directly on the QByteArray. – Botje Feb 07 '23 at 13:07
  • 1
    That's still using strings as an intermediate. And don't make much sense. Instead use `samp[0] | (samp[1] << 8) | (samp[2] << 16)` (depending on *byte order*). Assuming you did the pen-and-paper exercise I mentioned above, it should be very easy to see how it fits. – Some programmer dude Feb 07 '23 at 14:06
  • yes thank you @Someprogrammerdude, I got it and now it works faster ;) – A.A. Feb 07 '23 at 14:26

1 Answers1

1

Thanks to the guys in the comments, I've made some improvements over time. I avoided using strings for the conversion because it seems that using strings is always slow and replaced it with using bitwise shift and OR to combine the individual bytes into int.

I replaced this code:

QString sample = QString(samp.toHex() + "00");
bool ok;
int decimalSample = sample.toInt(&ok, 16);

with :

int32_t decimalSample = ((samp[0] << 24) | ((samp[1] << 16) & 0xFF0000) | ((samp[2] << 8) & 0xFF00));
     if(samp[0] & 0x80){
          decimalSample = - ((~decimalSample + 1) & 0xFFFFFFFF);
     }
A.A.
  • 125
  • 1
  • 11