2

I have two files that I am reading data from, doing calculations, and plotting a graph with. One file is quite small ~50 KB and raises no problem with the script. The other file is ~ 702, 900 KB (this is the file that causes the problem). I am able to read in the data perfectly fine, though when I calculate the row-by-row medians for this particular file, the script fails and gives me a MemoryError. It looks like the following:

    RMSDataS1 = [y01S1, y02S1, y03S1, y04S1, y05S1, y06S1, y07S1, y08S1, y09S1, 
         y010S1, y011S1, y012S1, y013S1, y014S1, y015S1, y016S1, y017S1, 
         y018S1, y019S1, y020S1, y021S1, y022S1, y023S1, y024S1, y025S1,
         y026S1, y027S1, y028S1, y029S1, y030S1, y031S1, y032S1, y033S1,
         y034S1, y035S1, y036S1, y037S1, y038S1, y039S1, y040S1, y041S1, 
         y042S1, y043S1, y044S1, y045S1, y046S1, y047S1, y048S1, y049S1,
         y050S1, y051S1, y052S1, y053S1, y054S1, y055S1, y056S1, y057S1,
         y058S1, y059S1, y060S1, y061S1, y062S1, y063S1, y064S1, y065S1, 
         y066S1, y067S1, y068S1, y069S1, y070S1, y071S1, y072S1, y073S1,
         y074S1, y075S1, y076S1, y077S1, y078S1, y079S1, y080S1, y081S1,
         y082S1, y083S1, y084S1, y085S1, y086S1, y087S1, y088S1, y089S1, 
         y090S1, y091S1, y092S1, y093S1, y094S1, y095S1, y096S1, y097S1,
         y098S1, y099S1, y0100S1, y0101S1, y0102S1, y0103S1, y0104S1, y0105S1,
         y0106S1, y0107S1, y0108S1]


    MediansS1 = []
    MediansS1 = np.median(RMSDataS1, axis = 0)

Is there any convenient way to get around this? I believe the script is failing when trying to sort the data when calculating the medians.

The error:

    Traceback (most recent call last):
      File "C:\Python27\Lib\site-packages\xy\RMSTrialOriginal-Aera.py", line 511, in <module>
        MediansS1 = np.average(RMSDataS1, axis = 0)
      File "C:\Python27\lib\site-packages\numpy\lib\function_base.py", line 486, in average
        a = np.asarray(a)
      File "C:\Python27\lib\site-packages\numpy\core\numeric.py", line 235, in asarray
        return array(a, dtype, copy=False, order=order)
    MemoryError

Any help would be greatly appreciated!

Wooble
  • 87,717
  • 12
  • 108
  • 131
user1620716
  • 1,463
  • 6
  • 17
  • 26
  • use a memory profiler to find the problem http://stackoverflow.com/questions/110259/python-memory-profiler – root Oct 03 '12 at 18:52
  • If the data is coming from a single file, why is `RMSDataS1` a list of y*S1 variables? Where do the y*S1 variables come from, and how big are they? Also, your description says you get an error when computing the medians, but the traceback shows the error occurring in a call to `np.average`. Does either one cause an error? – Warren Weckesser Oct 10 '12 at 18:53

0 Answers0