20

I can convert a string representation of a list to a list with ast.literal_eval. Is there an equivalent for a numpy array?

x = arange(4)
xs = str(x)
xs
'[0 1 2 3]'
# how do I convert xs back to an array

Using ast.literal_eval(xs) raises a SyntaxError. I can do the string parsing if I need to, but I thought there might be a better solution.

Community
  • 1
  • 1
jdmcbr
  • 5,964
  • 6
  • 28
  • 38
  • The numpy array doesn't provide a `repr` that can be used to reconstruct even a python list. You could doctor the string to recreate a list then create a numpy array from that e.g. `numpy.array(ast.literal_eval(', '.join(xs.split(' '))))` – Paul Rooney Aug 11 '16 at 03:24
  • Is it essential that you use `ast.literal_eval`? If so, then the answer is *no*, you can't get a numpy array from `literal_eval`. From the python documentation of `ast.literal_eval(node_or_string)`: "The string or node provided may only consist of the following Python literal structures: strings, numbers, tuples, lists, dicts, booleans, and None." If what you really want is a convenient way to convert a numpy array to a string and then back to an array, please elaborate on that in the question. – Warren Weckesser Aug 11 '16 at 03:59
  • Sorry for not being clearer. I was curious of there was an analog for `ast.literal_eval` that worked for numpy arrays, but didn't expect to use `ast.literal_eval`. – jdmcbr Aug 11 '16 at 04:00

4 Answers4

21

For 1D arrays, Numpy has a function called fromstring, so it can be done very efficiently without extra libraries.

Briefly you can parse your string like this:

s = '[0 1 2 3]'
a = np.fromstring(s[1:-1], dtype=np.int, sep=' ')
print(a) # [0 1 2 3]

For nD arrays, one can use .replace() to remove the brackets and .reshape() to reshape to desired shape, or use Merlin's solution.

ntg
  • 12,950
  • 7
  • 74
  • 95
Da Tong
  • 2,018
  • 18
  • 25
18

Try this:

xs = '[0 1 2 3]'

import re, ast
ls = re.sub('\s+', ',', xs)
a = np.array(ast.literal_eval(ls))
a  # -> array([0, 1, 2, 3])
wjandrea
  • 28,235
  • 9
  • 60
  • 81
Merlin
  • 24,552
  • 41
  • 131
  • 206
  • I think the other answer better answers my question as asked, yes. I cannot quite understand why you even ask this question. You had clearly answered before I accepted an answer. – jdmcbr Aug 12 '16 at 16:42
  • 3
    The reason is simple. The answer does not work for the code sample given! – Merlin Aug 12 '16 at 16:58
  • It is a self contained example that makes quite clear how to use `np.fromstring`, a function I'd forgotten about, for the problem I was trying to solve. I also said "I can do the string parsing if I need to" in my question. So I didn't learn anything from your answer, but I learned something from the other one. – jdmcbr Aug 12 '16 at 17:05
  • 1
    I am done here.. learning something is upvote, answering marked as correct on SO. The answer as chosen will lead ppl astray in the future. – Merlin Aug 12 '16 at 17:29
  • Both answers are valid depending on the case and for future readers like me both are beneficial. +1 to both. – ntg Feb 27 '19 at 10:14
  • This solution works on 2D array encoded by str() as well, thus it really should be marked as THE answer. – Hackless Sep 22 '22 at 04:37
2

Use np.matrix to convert a string to a numpy matrix. Then, use np.asarray to convert the matrix to a numpy array with the same shape.

>>> s = "[1,2]; [3,4]"
>>> a = np.asarray(np.matrix(s)) 
>>> a
array([[1, 2],
       [3, 4]])

In contrast to the accepted answer, this works also for two dimensional arrays.

Erel Segal-Halevi
  • 33,955
  • 36
  • 114
  • 183
0

if elements of lists are 2D float. ast.literal_eval() cannot handle a lot very complex list of list of nested list.

Therefore, it is better to parse list of list as dict and dump the string.

while loading a saved dump, ast.literal_eval() handles dict as strings in a better way. convert the string to dict and then dict to list of list

k = np.array([[[0.09898942, 0.22804536],[0.06109612, 0.19022354],[0.93369348, 0.53521671],[0.64630094, 0.28553219]],[[0.94503154, 0.82639528],[0.07503319, 0.80149062],[0.1234832 , 0.44657691],[0.7781163 , 0.63538195]]])

d = dict(enumerate(k.flatten(), 1))
d = str(d) ## dump as string  (pickle and other packages parse the dump as bytes)

m = ast.literal_eval(d) ### convert the dict as str to  dict

m = np.fromiter(m.values(), dtype=float) ## convert m to nparray
Soum
  • 63
  • 1
  • 5