8

I'm an admittedly pretty basic Python programmer, trying to learn as I encounter problems implementing various research problems. And I've hit one of those problems - particularly, how to handle loops where I'm returning a bunch of data, rather than the usual "out comes a single number" examples where you just add the result of the loop to everything previous.

Here's a Gist of the unlooped script I'm trying to run: https://gist.github.com/1390355

The really salient point is the end of the model_solve function:

def model_solve(t):
    # lots of variables set
    params = np.zeroes((n_steps,n_params)
    params[:,0] = beta
    params[:,1] = gamma
    timer = np.arange(n_steps).reshape(n_steps,1)
    SIR = spi.odeint(eq_system, startPop, t_interval)
    output = np.hstack((timer,SIR,params))
    return output

That returns the results of the ODE integration bit (spi.odeint) along with a simple "What time step are we on?" timer and essentially two columns of the value of two random variables repeated many, many times in the form of 4950 row and 7 column NumPy array.

The goal however is to run a Monte Carlo analysis of the two parameters (beta and gamma) which have random values. Essentially, I want to make a function that loops somewhat like so:

def loop_function(runs):
  for i in range(runs):
    model_solve(100)
    # output of those model_solves collected here
  # return collected output

That collected output would then be written to a file. Normally, I'd just have each model_solve function write its results to a file, but this code is going to be run on PiCloud or another platform where I don't necessarily have the ability to write a file until the results are returned to the local machine. Instead, I'm trying to get a return of a huge NumPy array of runs*7 columns and 4950 rows - which can then be written to a file on my local machine.

Any clues as to how to approach this?

matehat
  • 5,214
  • 2
  • 29
  • 40
Fomite
  • 2,213
  • 7
  • 30
  • 46
  • Are you trying to read a numpy array returned from a Python function, or how to write the result of a Python function to a file? – D K Nov 24 '11 at 02:04
  • @DK How to take many numpy arrays returned from a Python function and combine them into one array. – Fomite Nov 24 '11 at 02:59
  • It's not actually relevant that the results are NumPy arrays, because we don't use any NumPy functionality to solve the problem being asked about. – Karl Knechtel Jul 01 '22 at 01:19

2 Answers2

14

use a list to save all the results:

results = []
for i in range(runs):
    results.append(model_solve(100))

then get the output array by:

np.hstack(results)
HYRY
  • 94,853
  • 25
  • 187
  • 187
  • 2
    Well, that was...embarrassingly easy. Just did a proof of concept, and this answer seems to work, so I'm accepting it. – Fomite Nov 24 '11 at 05:12
11

Actually, if your code is having a much larger loop you should always try to vectorize your problem. If speed is important you should know that "for loops" are a bottle neck. Also, the append operation is very slow and demands more memory, because it creates copies. So, a better solution should be:

results = [0]*runs # if you want to use lists...

[model_solve(100) for x in results] # do see list comprehension in python

Besides using list, you can also use directly an array to store your results:

resutls=np.zeros([numberOfRuns,ShapeOfModelResults])

for i in range(numberOfRuns):
    results[numberOfRuns,modelSolve(100)] # this will put the result directly in the matrix

I hope my answer will help you write faster and clearer code

oz123
  • 27,559
  • 27
  • 125
  • 187