Processing large files with libsndfile

Question

Currently I've written a program (in Visual Studio C++) based on the available example code of "libsndfile" in which I've just added a subtraction-function to subtract one (channel of a) WAV-file with another.

The program worked perfectly for files with a limit of 4 channels.

The problem is that as soon as I upload a file with as many as 5 channels or more, the program refuses to execute the function. This is independent of the frame-size as I've managed to process 4-channel files which were many times larger than some 5/6-channel-files which I have used.

Has anyone encountered this or a similar problem before? If requested I will provide the source code.

My apologies for the bad structure, I don't really optimize until I get the whole deal working.

Code:

#include <sndfile.hh>
#include <tinyxml2.h>
#include <iostream>
#include <fstream>

#define BLOCK_SIZE 32

using TiXmlDocument = tinyxml2::XMLDocument;
using CStdStringA = std::string;

int main(int argc, char* argv[])

{
    SNDFILE *infile  = NULL;
    SNDFILE *infile2 = NULL;
    SNDFILE *outfile = NULL;
    SF_INFO    sfinfo;
    SF_INFO    sfinfo2;
    SF_INFO    sfinfoOut;
    sf_count_t readcount;
    //int filetype = SF_FORMAT_WAV | SF_FORMAT_PCM_24;                  // 24-bit Wav-Files
    TiXmlDocument iDoc;                                               // Init tinyXML
    int i;                                                            // Iteration-var;
    short BufferIn[BLOCK_SIZE];                                       // Buffer1
    short BufferIn2[BLOCK_SIZE];                                      // Buffer2
    short BufferOut[BLOCK_SIZE];
    CStdStringA FileLoc, FileName, FileLocWav, FileLocTxt,FileLocRaw; // OutputFile locationstring
    CStdStringA WavName, WavName2, FileIn, FileIn2;
    FileLoc = "C:\\Testfiles\\";
    //TextTitle(FileLoc);                                             // Print console-header
    printf("Name Wavefile 1:\n");
    std::cin >> WavName;
    FileIn = FileLoc + WavName + ".wav";
    if((infile = sf_open(FileIn.c_str(),SFM_READ,&sfinfo)) == NULL)   // Open Wav-file 2 instance
    {
        printf("Not able to open input file 1\n");
        printf("\n\nPress any key to exit.");                         // Exit-text if file not present
        puts(sf_strerror(NULL)) ;
        return 1;
    }
    std::cout << "Wav file 1 succesfully opened." <<  std::endl;
    std::cout << "\nChannels: " << sfinfo.channels << "\nFrames: " << sfinfo.frames << std::endl; // Print Channels & Frames
    std::cout << "Samplerate: " << sfinfo.samplerate << "\n\n" <<  std::endl;// Print Samplerate
    printf("Name Wavefile 2:\n");
    std::cin >> WavName2;
    FileIn2 = FileLoc + WavName2 + ".wav";
    if((infile2 = sf_open(FileIn2.c_str(),SFM_READ,&sfinfo2)) == NULL) // Open Wav-file 2 instance
    {
        printf("Not able to open input file 2\n");
        printf("\n\nPress any key to exit.");                          // Exit-text if file not present
        puts(sf_strerror(NULL)) ;
        return 1;
    }
    std::cout << "Wav file 2 succesfully opened." <<  std::endl;
    std::cout << "\nChannels: " << sfinfo2.channels << "\nFrames: " << sfinfo2.frames  << std::endl; // Print Channels & Frames
    std::cout << "Samplerate: " << sfinfo2.samplerate << "\n\n" <<  std::endl;// Print Samplerate
//{
    //std::cout << "Format differs from eachother, abort program.";
    //printf("\n\nPress any key to exit.");                     // Exit-text
    //return 1;
//}
    if(sfinfo.channels != sfinfo2.channels)                                // Abort if channels not the same
    {
        std::cout << "Channelammount differs from eachother, abort program.";
        printf("\n\nPress any key to exit.");                     // Exit-text
        return 1;
    }
    if(sfinfo.samplerate != sfinfo2.samplerate)                   // Abort if samplerate not the same
    {
        std::cout << "Samplerate differs from eachother, abort program.";
        printf("\n\nPress any key to exit.");                     // Exit-text
        return 1;
    }
    std::cout << "Give a filename for Txt- & Wav-file: ";
    std::cin >> FileName;
    FileLoc += FileName;                                          // Fuse Filelocation with given name
    FileLocTxt = FileLoc + ".txt";
    FileLocWav = FileLoc + ".wav";
    FileLocRaw = FileLoc + "Raw.txt";
    sfinfoOut.channels = sfinfo.channels;
    sfinfoOut.format = sfinfo.format;
    sfinfoOut.samplerate = sfinfo.samplerate;
    if((outfile = sf_open(FileLocWav.c_str(),SFM_WRITE,&sfinfoOut)) == NULL)      // Open Wav-file 2 instance
    {
        printf("Not able to create output file \n");
        printf("\n\nPress any key to exit.");                      // Exit-text if file not present
        puts(sf_strerror(NULL)) ;
        return 1;
    }
    std::ofstream myfile;
    myfile.open(FileLocTxt.c_str(),std::ios::app);
    std::ofstream myfileRaw;
    myfileRaw.open(FileLocRaw.c_str(),std::ios::app);
    while((readcount = sf_read_short(infile, BufferIn, BLOCK_SIZE)))     // While there are still frames to be processed
    {
        //process_data (data, readcount, sfinfo.channels) ;
        auto readcount2 = sf_read_short(infile2, BufferIn2, BLOCK_SIZE);
        for(i = 0; i < BLOCK_SIZE; i++)                                  // BLOCK_SIZE decides the chunk-size
        {
            BufferOut[i] = BufferIn[i] - BufferIn2[i];
            myfileRaw << BufferOut[i];
        }
        sf_write_short(outfile, BufferOut, BLOCK_SIZE) ;                 // Write the data to a new file
    }
    sf_close(infile);                                                    // Close Wav-file handlers
    sf_close(infile2);
    sf_close(outfile);
    myfile.close();                                                      // Close text-file handlers
    printf("\n\nPress any key to exit.");                                // Exit-text
    return 0;
}

made it selfcontained and compiling on linux/gcc with `g++ -std=c++0x -Wall -pedantic -g -O0 test.cpp -o test -lsndfile -ltinyxml2` — sehe, Oct 15 '13 at 10:19
Just guessing after a quick read of the documentation: `sf_read_short(infile, BufferIn, BLOCK_SIZE)` seems problematic to start with: sf_read_short is going to read BLOCK_SIZE * #channels so you are writing out of array bounds. There's some stuff on the stack after BufferIn so probably you could get away wih 4 channels because you're lucky, then the UB really kicked in. (sidenote: *please* declare variables as close as possible to where you're going to use them. It makes more sense and it makes code much easier to read. This is not C.) — stijn, Oct 15 '13 at 10:28
Right, does make sense indeed. Problem is that I'm native to writing C for microcontroller-applications and this is a whole new environment for me. Is there any way to give the Buffer-arrays a dynamic size? Where the size represents BLOCK_SIZE * # Channels? — ItWillDo, Oct 15 '13 at 10:33
use `std::vector< short >`, it provides continuous storage with the same semantics as a plain array. It's size can be set in the contructor, or with it's `resize()` method. It has a member function `data()` which gives you a pointer to the beginning of the sequence, that's wht you'd pass to sf_read/sf_write — stijn, Oct 15 '13 at 10:38
@stijn I don't believe your reading of the documentation is accurate and there shouldn't be an overrun. — sehe, Oct 15 '13 at 10:42
Alright thanks. I'll give it a shot and report if it works as supposed to. One thing that did come to mind is the fact that as far as I remember 'sf_read_short()' reads a number of items and sf_readf_short()' reads a number of frames. So I think that only in the 2nd case should the allocation be large enough to hold #items * #channels. EDIT: Just for testing purposes I only made array-size into BLOCK_SIZE*10. The problem still holds so I doubt it's the allocation. — ItWillDo, Oct 15 '13 at 10:46
I do believe I should explain the result a bit better though. Normally when processing the calculation, it takes about 1 second to 1,5 minutes depending on the frame size and number of channels. Afterward it just displays the 'Press any key to exit.'-message and the program ends. When trying to process channelsizes >= 5, the program finishes up in about half a second and displays the last message as if the calculation went as planned. — ItWillDo, Oct 15 '13 at 10:52
@sehe but it says "Care must be taken to ensure that there is enough space in the array pointed to by ptr, to take (frames * channels) number of items (shorts, ints, floats or doubles)." — stijn, Oct 15 '13 at 10:55
@stijn interesting. Do you have a link, because the page I found explicitely documents something else. Ah, you're looking at `sf_readf_*`, I think. These read _frames_. — sehe, Oct 15 '13 at 10:56
@sehe nevermind.. was looking at sf_readf_xxx instead of sf_read_xxx. So your explanation of BUFFERSIZE not being a multiple of #channels is correct. — stijn, Oct 15 '13 at 10:57

sehe · Accepted Answer · 2013-10-15T10:58:34.680

The documentation states:

File Read Functions (Items)
sf_count_t  sf_read_short   (SNDFILE *sndfile, short *ptr, sf_count_t items) ;
sf_count_t  sf_read_int     (SNDFILE *sndfile, int *ptr, sf_count_t items) ;
sf_count_t  sf_read_float   (SNDFILE *sndfile, float *ptr, sf_count_t items) ;
sf_count_t  sf_read_double  (SNDFILE *sndfile, double *ptr, sf_count_t items) ;
The file read items functions fill the array pointed to by ptr with the requested number of items. The items parameter must be an integer product of the number of channels or an error will occur

There's your problem, since BLOCK_SIZE is a multiple of 1,2,4 but not of 3 and 5 - so that would fail.

About the problem that program completes as if calculations had taken place: add appropriate error handling.

Processing large files with libsndfile

1 Answers1

File Read Functions (Items)