5

I have a csv file that looks something like this (actual file has many more columns and rows):

1,2,3,4,5
6,7,8,9,10
11,12,13,14,15
16

Say the name of the file is info.csv If I try to import this using 

data = numpy.genfromtxt('info.csv', delimiter = ',')

then I get the following error:

ValueError: Some errors were detected ! Line #4 (got 1 columns instead of 5)

If I use,

data = numpy.genfromtxt('info.csv', delimiter = ',', skip_footer = 1)

both lines with data 16 and with data 11, 12, 13, 14, 15 are skipped. I don't understand why the line with 11, 12, 13, 14, 15 is being skipped. I would appreciate any help on how I can appropriately use the genfromtxt to import first three lines in the above file.

Thanks

Mike Graham
  • 73,987
  • 14
  • 101
  • 130
Curious2learn
  • 31,692
  • 43
  • 108
  • 125

2 Answers2

9

if you can ignore the 16 at the end of the file try using the

invalid_raise (bool, optional) parameter if set to False it ignores all incomplete lines without throwing an exception

see here (its the last parameter before the examples) http://docs.scipy.org/doc/numpy/reference/generated/numpy.genfromtxt.html

Nikolaus Gradwohl
  • 19,708
  • 3
  • 45
  • 61
  • Thanks! This works well for me. But can you or someone explain why skip_footer is not working. Or, how can I get the row with 16 in it. I will accept the answer later, because if I do so now, those questions will remain unanswered. Thanks again. – Curious2learn Sep 22 '10 at 01:25
  • 2
    it skips 2 rows, becaus genformtxt reads the valid rows into an array and then skips as mamy as you told him, but the line with '16' is never read into the array – Nikolaus Gradwohl Sep 22 '10 at 03:57
  • you can try the 'filling_values' or 'missing_values' parameter to fill the missing 4 values in the line with '16', for example by -1 and or 0 depending on what you do with your array after reading it from disk – Nikolaus Gradwohl Sep 22 '10 at 04:00
  • can you please explain how to use filling_values. I tried, `numpy.genfromtxt('info.csv', delimiter = ',', filling_values = 0)`. However, that still gives the same error. – Curious2learn Sep 22 '10 at 11:18
  • ok - i checked the code of numpy now, filling_values can be used to fill in empty values - so if your last line looks like '16,,,," fill would fill the empty values. if the line has no delimiters genfromtxt can't parse it – Nikolaus Gradwohl Sep 22 '10 at 12:39
0

The command filling_values also helped me. I set it to zero. Thus each empty value is set to zero. It probably doesn't always make sense, but maybe it will help you.

Dark
  • 179
  • 2
  • 12