2

The following function creates an Audio Analysis Object, which itself has a size of 64 bytes (as per sys.getsizeof), but the function itself is using over 100MB of memory for a 6MB mp3 file.

 def main(input_filename):
     audiofile = audio.LocalAudioFile(input_filename)

I've profiled using memory_profiler (thanks to Huy Nguen) and also used Pympler (from SO answer) to explore the sizes, number and types of objects (results of which are shared below. Even confirmed the size of the inputfile itself from within Python using os.stat(input_filename) with statinfo2.st_size. The total size of the file and object combined appears to be 11MiB (basically MB).

Does this mean I have to dig deeper into the calls made by the LocalAudioFile class which creates the object? And if so, does that need to be done within the code of the external module itself, or perhaps by experimenting with a duplicate of it's code?

                       types |   # objects |   total size
============================ | =========== | ============
                        dict |        2564 |      3.42 MB
                         str |       14790 |      2.43 MB
                       float |       35420 |    830.16 KB
                     unicode |        9552 |    792.63 KB
                        code |        5670 |    708.75 KB
                        list |        3160 |    606.56 KB
                        type |         484 |    427.28 KB
          wrapper_descriptor |        1658 |    129.53 KB
                       tuple |        1227 |     91.00 KB
  builtin_function_or_method |        1094 |     76.92 KB
                     weakref |         894 |     76.83 KB
           method_descriptor |         899 |     63.21 KB
                         set |         151 |     53.21 KB
           getset_descriptor |         645 |     45.35 KB
                         int |        1657 |     38.84 KB

Memory profiling results: 

Line #    Mem usage    Increment   Line Contents
================================================
    30   20.395 MiB    0.000 MiB   @profile
    31                             def main(input_filename, input_filename2):
    32   24.250 MiB    3.855 MiB       audiofile = audio.LocalAudioFile(input_filename) #250kb audio file
    33  139.988 MiB  115.738 MiB       audiofile2 = audio.LocalAudioFile(input_filename2) # 6MB audio file
    #irrelevant profiling code removed
    47  146.531 MiB    0.477 MiB       summary.print_(sum1)
Community
  • 1
  • 1
MikeiLL
  • 6,282
  • 5
  • 37
  • 68
  • It's hard to answer this question. Why are you memory profiling? Why are you surprised at the memory used? (mp3 is a compressed format. It's likely that the analyzer has to uncompress it to do the analysis.) Why are you worried about the sys.getsizeof results when you link to an SO answer that says how unreliable it is? – John Hazen Aug 19 '14 at 05:32
  • @JohnHazen very insightful to consider that the mp3 is likely to be being uncompressed in this process. in terms of sys.getsizeof, it was my understanding that it's with 100 bytes or so accurate, but i think that's not terribly relevant here anyway. The main thing I'm curious about is WHERE the additional 80%(ish) of memory above and beyond the size of the audio file combined with it's analysis object is being created and/or held. I am looking into multithreading the procedure which might be a way to allow for the large memory usage without exceeding the limits of the 1GB server I plan to use. – MikeiLL Aug 19 '14 at 12:03

0 Answers0