0

The compiler return an error

Process finished with exit code 137 (interrupted by signal 9: SIGKILL)

when running the following code

import itertools
x = [11, 12, 13]
all_result=[p for p in itertools.product(x, repeat=1000)]

Is there are way to avoid getting this kind of issue by tweaking the module setting. I know in numpy.meshgrid, there is the option sparse to conserve memory.

Is this possible apply something like this with itertools.

Or if there is other existing permutation module than can handle memory better than the itertools.

p.s.

To give better perspective as rise by @Carcigenicate, the outputs of the all_result will be used in the following code. For simplicity, assume repeat set to two, and one of the pair is {11,13}.

tot_length=0.2
steps=0.1
start_val=0
list_no =np.arange(start_val, tot_length, steps)
x, y, z = np.meshgrid(*[list_no for _ in range(3)], sparse=True)
ix = np.array(((x>=y) & (y>=z)).nonzero()).T
final_opt=list_no[ix]
final_opt[:,[0, 1]] = final_opt[:,[1, 0]]
# The value {11,13} will be inputted at the last line below
FTE_DEF=np.concatenate((final_opt[11,:], final_opt[13,:]))
mpx
  • 3,081
  • 2
  • 26
  • 56
  • Hi @Carcigenicate, for simplicity let repeat set to two, assume we got pair of `11` and `13`, then these value will be used to concatenate two arrays. Let, final_opt is an array, then the next procedure will be the following 'np.concatenate((final_opt[11,:], final_opt[13,:])) ' – mpx Oct 21 '20 at 14:59
  • @Carcigenicate If not all, then processing half of 'all_result' is fine as long is memory issue is taken care. But, ultimately, I would concatenate all the possible pair. – mpx Oct 21 '20 at 15:03
  • 1
    `all_result` doesn't appear in your code. – chepner Oct 21 '20 at 15:06
  • Hi @Carcigenicate, see the edited p.s. – mpx Oct 21 '20 at 15:07
  • The basic answer is to NOT create a list. Just use the generator in your code. – Barmar Oct 21 '20 at 15:07
  • `(x for x in product(...)` is effectively identical to `product(...)`. No additional generator expression is needed. – chepner Oct 21 '20 at 15:07
  • The length of `all_result` is `len(x)**1000` (or `3**1000`), which is too large for Python to convert to a floating-point number. Are you sure that you're generating `all_result` correctly? – jjramsey Oct 21 '20 at 15:10
  • @Barmar, appreciate if you can kindly elaborate more or direct me to good reading material.Im open to suggestion as long as the memory issue can be handle – mpx Oct 21 '20 at 15:14
  • 1
    Just use `all_results = itertools.product(...)` instead of making it into a list. – Barmar Oct 21 '20 at 15:16
  • 1
    This should work in the rest of the code unless you need to iterate of the results multiple times. Since you haven't shown how the variable is used, it's hard to tell if it will work for you. – Barmar Oct 21 '20 at 15:17
  • Hi @Barmar, thanks for the suggestion, it does help. – mpx Oct 21 '20 at 16:45
  • Hi @Carcigenicate, I have create a new OP about the usage of the `all_result` which is access able at https://stackoverflow.com/q/64468435/6446053 – mpx Oct 21 '20 at 17:07

0 Answers0