1

I have a bytearray of 2^18 (262144) bytes. The goal is to convert it to a numpy array of complex values. Packing within the bytearray is:

[0:1] Imaginary_0 16bit
[2:3] Real_0 16bit
[4:5] Imaginary_1 16bit
[6:7] Real_1 16bit
........
[262140:262141] Imaginary_65535 16bit
[262142:262143] Real_65535 16bit

I have tried two different methods but still get considerably long execution times. Is there a faster approach?

def byte_array_to_complex_loop(byte_array, sample_count):
    y_iq = np.empty([sample_count], dtype=np.complex)
    for i in range(0, sample_count):
        read_data_bytes = byte_array[i*4: i*4+4]
        unpacked_data = struct.unpack("<hh", read_data_bytes)  # unpack into two singed short(s)
        y_iq[i] = complex(unpacked_data[1], unpacked_data[0])
    return y_iq


def byte_array_to_complex_np_iter(byte_array, sample_count):
    np_dtype = np.dtype([('i', '<i2'), ('q', '<i2')])
    np_read_data = np.frombuffer(byte_array, dtype=np_dtype)
    y_iq = [complex(item[1], item[0]) for item in np_read_data]
    return y_iq


def main():
    function_timing.clear()
    sample_count = 0x10000
    bytes_per_sample = 4
    function_timing.add_time("START")
    read_data = bytearray(os.urandom(sample_count*bytes_per_sample))
    function_timing.add_time("GENERATE RANDON BYTE ARRAY")
    y_iq1 = byte_array_to_complex_np_iter(read_data, sample_count)
    function_timing.add_time("Y_IQ CONV. (LIST COMPR)")
    y_iq2 = byte_array_to_complex_loop(read_data, sample_count)
    function_timing.add_time("Y_IQ CONV. (LOOP)")
    function_timing.print_times()

Timing Output shows 0.8105 and 0.5588 seconds for each of the functions. 

- START:                       Delta:0.0000, Tot:0.0000
- GENERATE RANDON BYTE ARRAY:  Delta:0.0030, Tot:0.0030
- Y_IQ CONV. (LIST COMPR):     Delta:0.8105, Tot:0.8135
- Y_IQ CONV. (LOOP):           Delta:0.5588, Tot:1.3723
<<< END >>>
Konrad Rudolph
  • 530,221
  • 131
  • 937
  • 1,214
Borisw37
  • 739
  • 2
  • 7
  • 30

1 Answers1

0

It might be faster to unpack the whole byte array in one shot, and then interpret the data. You can prepend the number of members in your unpack format string:

unpacked = np.array(struct.unpack("{0}<h".format(2*sample_count), byte_array)).reshape(sample_count,2)
return unpacked[1::2] + 1j*unpacked[::2] #2nd int16 is real part, 1st is imag part, in each pair of int16s
mtrw
  • 34,200
  • 7
  • 63
  • 71