0

I'm writing a Python script reading an input file, line by line, each containing 20 values. I'm trying to store the first one in a 1D array, and the 19 others in a second array.

They have been defined as:

x = numpy.zeros((n_lines))
y = numpy.zeros((n_lines,19))

n_lines the number of lines in my input file. and I'm reading it with

for i,line in enumerate(open(infile)):
    x[i],y[i] = line.split(' ')

But I'm encountering an Too many values to unpack error

I'm now doing it by storing it in one single large array and then cutting it into two in the end. I'm not using line.split(' ')[0],line.split(' ')[1:] as it doesn't look optimized to split twice.

Is there a neat (Pythonista) way of unpacking?

(I'm using Python 2.7 if that matters)

wwii
  • 23,232
  • 7
  • 37
  • 77
Learning is a mess
  • 7,479
  • 7
  • 35
  • 71

1 Answers1

2

You're doing the last part right (creating one array and then slicing it), but the first part should be done using numpy.genfromtxt(), like this:

big_array = numpy.genfromtxt(infile) ## probably with more arguments
x, y = numpy.split(big_array, [1])

If a more efficient solution is needed, an alternative would be:

big_array = numpy.empty((nlines, 20))

for i,line in enumerate(open(infile)):
    big_array[i,:] = line.split()

x, y = numpy.split(big_array, [1])

Finally, unpacking directly from line to the former x and y arrays can be done like:

splitted = line.split();
x[i], y[i,:] = splitted[0], splitted[1:]
heltonbiker
  • 26,657
  • 28
  • 137
  • 252
  • It's definitely quicker to code, but as explained here: http://stackoverflow.com/questions/24701696/python-and-memory-efficient-way-of-importing-2d-data inefficient for large scale data import. – Learning is a mess Oct 03 '14 at 13:59
  • 1
    Then you could create the big array as a whole, and then use `numpy.split()` on it as shown. Would that solve the problem? – heltonbiker Oct 03 '14 at 14:03
  • My question maybe wasn't clear enough, I'm sure your splitting solution would work maybe even better than the one I am using now. I'm curious to know the correct way of unpacking from 20 to 1 + 19 than how to read the file or split array. Thank you for your input by the way! – Learning is a mess Oct 03 '14 at 14:17
  • Unpacking from 20 to 1 + 19 is done like this: `one, nineteen = twenty[0], twenty[1:]`. So you could do this, I think: `x[i], y[i,:] = splitted[0], splitted[1:]` since they have respectively the same shape. – heltonbiker Oct 03 '14 at 14:26
  • Thank you, I thought it was possible to do it in one line, without storing, but maybe this is only true in Python 3+ – Learning is a mess Oct 03 '14 at 15:12
  • 2
    @Learning-is-a-mess - 3.something added this feature, ```one, *nineteen = a``` – wwii Oct 03 '14 at 15:26
  • @wwii: Thank you very much for adding the final word! – Learning is a mess Oct 03 '14 at 15:54