Problem with using Python's Struct Unpack Recently

Question

In a Python script I record some audio with the following command:

import subprocess
import wave
self.rec_args =['arecord', '--device=pulse', '-f', 'cd', '-t', '/home/USER/audioFile.wav')]
self.rec = subprocess.Popen(self.rec_args, shell=False)

I then open the file:

self.wave_file = wave.open('/home/USER/audioFile.wav', 'rb')

and get the number of channels (2) and sample rate (44100) with:

self.num_channels, self.sample_rate = self.getWaveParameters(self.wave_file)

I can then use the code for getWaveIntegers to get the amplitudes of the sound with:

wi = self.getWaveIntegers(self.wave_file, self.sample_rate, self.num_channels, 0, 20)

where clipStart = 0 and then length of the sample to analyse = 20 seconds.

########################################
    def getWaveIntegers(self, stream, sample_rate, num_channels, clipStart, timeLength):
# https://stackoverflow.com/questions/2226853/
# stream = the already opened wave file to navigate through
# clipStart = the starting position of the clip in the wave file
# timeLength = the length of time of the clip to take from the wave file
# set start of clip
        startPosition = sample_rate * clipStart # converts clipStart time to samplewidth
        stream.setpos(startPosition) # set the starting position in the wave file
# length of clip
        clipLength = sample_rate*timeLength*num_channels # timeLength in terms of channels and sample rate ####?
        clipData = stream.readframes(sample_rate*timeLength) # the clip of the wave file starting at startPosition for time in sample_rate
        integer_data = wave.struct.unpack("%dh"%(clipLength), clipData)
        channels = [ [] for time in range(num_channels) ]
        for index, value in enumerate(integer_data):
            bucket = index % num_channels
            channels[bucket].append(value)
        del clipData
# keep only left? channel
        sampleChannel = []
        for c in range(0, len(channels[0]):
           sampleChannel.append(abs(int(channels[0][c]/100)))
        return sampleChannel
########################################
    def getWaveParameters(self, stream):
# https://stackoverflow.com/questions/2226853/
        num_channels = stream.getnchannels()
        sample_rate = stream.getframerate()
        sample_width = stream.getsampwidth()
        num_frames = stream.getnframes()
        raw_data = stream.readframes( num_frames ) # Returns byte data
        total_samples = num_frames * num_channels
        if sample_width == 1: 
            fmt = "%iB" % total_samples # read unsigned chars
        elif sample_width == 2:
            fmt = "%ih" % total_samples # read signed 2 byte shorts
        else:
            raise ValueError("Only supports 8 and 16 bit audio formats.")
        return num_channels, sample_rate#, sample_width, num_frames, total_samples, fmt # do not need these values for this code
########################################

This has worked properly until recently. I now get the following error:

Traceback (most recent call last):
  File "myfile.py", line 167, in getWaveIntegers
    integer_data = wave.struct.unpack("%dh"%(clipLength), clipData)
struct.error: unpack requires a buffer of 3528000 bytes

The clipLength of some files seem to be: 1764000 bytes (exactly half the 3528000 bytes in the error)

I noticed an update of various Python libraries in the past couple of days to the following version: Python 3.6:amd64 (3.6.8-1~18.04.3, 3.6.9-1~18.04)

I tried the example presented at https://docs.python.org/3.6/library/struct.html

>>> from struct import *
>>> pack('hhl', 1, 2, 3)
b'\x00\x01\x00\x02\x00\x00\x00\x03'
>>> unpack('hhl', b'\x00\x01\x00\x02\x00\x00\x00\x03')
(1, 2, 3)

When I do it on my system (Python 3.6.9 (default, Nov 7 2019, 10:44:02) and [GCC 8.3.0] on linux):

>>> from struct import *
>>> pack('hhl', 1, 2, 3)
b'\x01\x00\x02\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00'

the length is double the length in the example. With:

>>> unpack('hhl', b'\x00\x01\x00\x02\x00\x00\x00\x03')

I get the error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
struct.error: unpack requires a buffer of 16 bytes

but when I do:

>>> unpack('hhl', b'\x01\x00\x02\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00')
(1, 2, 3)

I get the correct answer.

Have I done something strange or is it an error in a Python update. To be honest, I don't fully understand the struct command fully. Thank you very much for reading a post from a windbag like me! John

John · Accepted Answer · 2019-11-29T17:00:20.287

I couldn't figure out the problem with struct unpack as described above so I programmed around it and used numpy. So the code for getWaveIntegers should be:

########################################
import numpy
def getWaveIntegers(self, stream, sample_rate, num_channels, clipStart, timeLength):
# https://stackoverflow.com/questions/2226853/
# stream = the already opened wave file to navigate through
# clipStart = the starting position of the clip in the wave file
# timeLength = the length of time of the clip to take from the wave file
# set start of clip
    startPosition = sample_rate * clipStart # converts clipStart time to samplewidth
    stream.setpos(startPosition) # set the starting position in the wave file
# length of clip
    clipLength = sample_rate*timeLength*num_channels # timeLength in terms of channels and sample rate ####?
    clipData = stream.readframes(sample_rate*timeLength) # the clip of the wave file starting at startPosition for time in sample_rate
    signal = numpy.fromstring(clipData, "Int16")
# https://stackoverflow.com/questions/22636499/
    result = numpy.fromstring(clipData, dtype=numpy.int16)
    chunk_length = int(len(result) / num_channels)
    assert chunk_length == int(chunk_length)
    result = numpy.reshape(result, (chunk_length, num_channels))
# keep only left channel and only 10 samples per second
    leftChannel = []
    for c in range(0, len(result[:, 0]), 50):
       leftChannel.append(abs(int(result[:, 0][c]/100))) # the absolute value of the integer of the frequency divided by 100 (for diagramatic purposes)
    return leftChannel
########################################

I hope this helps someone in the future.

Problem with using Python's Struct Unpack Recently

1 Answers1