0

I'm getting the error in numba while I'm trying to parallelize a loop that says

Use of a tuple (my_list) of length 200 in a parallel region exceeds
the maximum supported tuple size.  Since Generalized Universal
Functions back parallel regions and those do not support tuples,
tuples passed to parallel regions are unpacked if their size is below
a certain threshold, currently configured to be 100. This threshold
can be modified using the Numba environment variable
NUMBA_PARFOR_MAX_TUPLE_SIZE.

From these instructions I thought I could make a file called .numba_config.yaml with this content

NUMBA_PARFOR_MAX_TUPLE_SIZE: 500

The file is stored in the same directory as main.py and I run the program with python3 main.py

How can I adjust the max permitted tuple size?

A solution appears to be

import os
os.environ["NUMBA_PARFOR_MAX_TUPLE_SIZE"] = "500"
financial_physician
  • 1,672
  • 1
  • 14
  • 34
  • So, why not just use a `list`? – juanpa.arrivillaga Jan 18 '22 at 19:56
  • Using big tuples (eg. with >500 item) is really a problem with Numba, so please don't do that because it generates a very bad code internally (both huge and slow). Tuples are like a C structure with `N` fields. Each field is computed separately by the compiler. Consider using a list or a Numpy array For more information, please read [this related post](https://stackoverflow.com/a/70598981/12939557). – Jérôme Richard Jan 18 '22 at 21:01
  • 1
    I think tuple is the default from my database pull using asynciopg, I'll make it a list. I just assumed they just meant it was an iterable. Thanks! – financial_physician Jan 18 '22 at 21:30
  • For performance, using an array is usually better since it has a fixed size (like a tuple) and fixed datatype. Whereas a list is much more dynamic in that regard, which is nice if you need it, but you'll always pay a performance penalty. – Rutger Kassies Jan 19 '22 at 11:35

0 Answers0