0

Edit: Solution

The answer was posted below, using list prod from itertools greatly reduces memory usage as it is a list object, something I overlooked in the thread I linked.

Code:

n = 2 #dimension
r = np.arange(0.2,2.4,0.1) #range

grid = product(r,repeat=n)

for g in grid:
    y = np.array(g)
    if np.all(np.diff(g) > 0) == True:
        print(f(y)) #or whatever you want to do

I try to evalute a function with n parameters in a certain range. And I want to be able to change n in a certain range, so the user can determine it with his input.

I got a working code with help from here an it looks like this:

import numpy as np

n = 2 #determine n
r = np.arange(0.2,2.4,0.1) #determine range
grid=np.array(np.meshgrid(*[r]*n)).T.reshape(-1,n) #creates a meshgrid for the range**n

#reduces the number of used points in the grid (the function is symmetrical in its input parameters)
for i in range(0,n-1):
    grid = np.array([g for g in grid if g[i]<g[i+1]])

y = np.zeros((grid.shape[0]))
for i in range(grid.shape[0]):
    y[i] = f(grid[i,:]) #evaluating function for all lines in grid

This only works up to n equals 5 or 6 then the grid-array and the ouput array just gets to big for Spyder to handle it (~10 MB).

Is there a way to only create one line of the meshgrid (combination of parameters) at a time to evaluate the function with? Then I could save those values (grid,y) in a textfile and overwrite them in the next step.

Or is there a way to create all n-dimensional combinations in range without meshgrid but with a variable n and one at a time?

Cœur
  • 37,241
  • 25
  • 195
  • 267
Franz
  • 15
  • 4

1 Answers1

1

You may want to try iterator. It seems itertools.product fits your need.

r = np.arange(0,3,1) #determine range
list(product(r,r))
#[(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, 2)]

More detail:

grid = product([r]*n)
y = np.zeros(... # you should know the exact size
i = 0
for g in grid:
    if ...:      # check if g in ascending order
        y[i] = f(g)
        i += 1

By the way the size of y should be comb(len(r), 3), choose 3 from len(r)

from scipy.special import comb
comb(len(r), 3)

Or an one-liner

y = [f(g) for g in grid if is_ascending(g)]

Where is_ascending is an user defined filter function.

Syrtis Major
  • 3,791
  • 1
  • 30
  • 40
  • That does essentially the same as the posted code., still leaves me with a big grid and y array. Gonna test it for time now but I can't seem the improvement... – Franz Sep 20 '16 at 14:09
  • The point is **no** great array, `grid` here is not an array, the data in it (ie. the `g`s) is calculated until you use it. BTW, do not run `list(...` for large n, the `list(product...` is just for giving some idea. @Franz – Syrtis Major Sep 20 '16 at 14:11
  • @Franz try it, you will find that all you need is an array y for the output. `grid` here do not require much memory. – Syrtis Major Sep 20 '16 at 14:14
  • Okay, after a bit playing around with it I get it! Cheers! – Franz Sep 20 '16 at 14:30
  • However, in this method you still need an array `y` to store the results which might be too big for your machine (though no big `grid` anymore). If `y` itself is still too big, you have to save it to disk in the for-loop. – Syrtis Major Sep 20 '16 at 14:31
  • Yeah, I am writing it into a file. – Franz Sep 20 '16 at 14:51