Numpy.fromfunction(): How to use it with strings?

Question

The first line works, but the second doesn't:

print(np.fromfunction(lambda x, y:     10 * x + y , (3, 5), dtype=int))
print(np.fromfunction(lambda x, y: str(10 * x + y), (3, 5), dtype=str))

[[ 0  1  2  3  4]
 [10 11 12 13 14]
 [20 21 22 23 24]]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/cygdrive/c/Users/pafh2/OneDrive/dev/reverb/t.py", line 439, in <module>
    print(np.fromfunction(lambda x, y: str(10 * x + y), (3, 5), dtype=str))
  File "/usr/lib/python3.7/site-packages/numpy/core/numeric.py", line 2027, in fromfunction
    args = indices(shape, dtype=dtype)
  File "/usr/lib/python3.7/site-packages/numpy/core/numeric.py", line 1968, in indices
    res[i] = arange(dim, dtype=dtype).reshape(
ValueError: no fill-function for data-type.
>>>

Which page of the Numpy documentation explains the difference?

StackOverflow won't let me post, telling me, "It looks like your post is mostly code; please add some more details." So now everyone has to read this pointless paragraph I've just added.

Now it says I've still not written enough "details", though I really have, so here's more added spam to appease the the StackOverflow spam-filter "A.I.".

Answer 1

Do you understand what the dtype is supposed to be doing in fromfunction? Reread the docs.

This action is producing the same error:

In [2]: np.arange(3, dtype=str)                                                                        
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-2-9944cc5541d5> in <module>
----> 1 np.arange(3, dtype=str)

ValueError: no fill-function for data-type.

You may be confused about what fromfunction does. It generates arrays with np.indices, and passes them whole the function. It does not iterate or pass values one by one. It is not np.frompyfunc.

In [4]: np.fromfunction(lambda x, y: str(10 * x + y), (3, 5))                                          
Out[4]: '[[ 0.  1.  2.  3.  4.]\n [10. 11. 12. 13. 14.]\n [20. 21. 22. 23. 24.]]'

In [5]: np.fromfunction(lambda x, y: str(10 * x + y), (3, 5), dtype=int)                               
Out[5]: '[[ 0  1  2  3  4]\n [10 11 12 13 14]\n [20 21 22 23 24]]'

Out[4] is float, Out[5] is int. The dtype sets the dtype of indices, which propagates through the function. The str in the lambda turns the 10*x+y array into a string.

fromfunction isn't very useful. You can get the same results with:

In [6]: 10*np.arange(3)[:,None] + np.arange(5)                                                         
Out[6]: 
array([[ 0,  1,  2,  3,  4],
       [10, 11, 12, 13, 14],
       [20, 21, 22, 23, 24]])

====

The code for fromfunction is simple:

args = indices(shape, dtype=dtype)
return function(*args, **kwargs)

all it is doing creating the indices and passing them to the function.

In [10]: np.indices((3,5))                                                                             
Out[10]: 
array([[[0, 0, 0, 0, 0],
        [1, 1, 1, 1, 1],
        [2, 2, 2, 2, 2]],

       [[0, 1, 2, 3, 4],
        [0, 1, 2, 3, 4],
        [0, 1, 2, 3, 4]]])

In [12]: np.mgrid[:3, :5]       # alternatives                                                                       
Out[12]: 
array([[[0, 0, 0, 0, 0],
        [1, 1, 1, 1, 1],
        [2, 2, 2, 2, 2]],

       [[0, 1, 2, 3, 4],
        [0, 1, 2, 3, 4],
        [0, 1, 2, 3, 4]]])
In [15]: np.meshgrid(np.arange(3), np.arange(5), indexing='ij')                                        
Out[15]: 
[array([[0, 0, 0, 0, 0],
        [1, 1, 1, 1, 1],
        [2, 2, 2, 2, 2]]), array([[0, 1, 2, 3, 4],
        [0, 1, 2, 3, 4],
        [0, 1, 2, 3, 4]])]

Answer 2

This has not directly to do with numpy.fromfunction. If you look at the traceback you posted, you can see that the problem originates from this line:

File "/usr/lib/python3.7/site-packages/numpy/core/numeric.py", line 1968, in indices
    res[i] = arange(dim, dtype=dtype).reshape(

If you look at the file, you will find that arange is nothing else than numpy.arange. So you can do the following test which will result in the same ValueError: no fill-function for data-type.:

import numpy as np


a = np.arange(0, 10, dtype=str)

So what does that tell us? A qick internet search for the error message returns this post

Executing numpy.arange(0, 2, 1, numpy.bool_) works fine, because np.bool_ type is still meaningful for range [0,2). However, what is the meaning of range [0,5) for type bool_??
This is the error you get - numpy tries to tell you that you are asking for values in invalid range.

Here is where it gets interesting: if we apply that logic to the str dtype, what would be a valid range? Let's try the example from the post:

numpy.arange(0, 2, 1, str)
# >>> ['0' '1']

This works, so the range and step size seem to satisfy dtype str. What happens if we increase the range?

numpy.arange(0, 3, 1, str)
# >>> ValueError: no fill-function for data-type.

We get the error from before. Why is that? I have no idea. But this is where the root of your problem lies.