numpy.loadtxt, ValueError: could not convert string to float

Question

This is sample from large csv file:

6.1;6.1;7.2;8.9;5.0;
8.9;10.0;8.9;6.1;5.0;

If I try to read it to numpy array with np.loadtxt('test.csv', delimiter=';') I get:

ValueError: could not convert string to float:

and don't understand why?

score 43 · Accepted Answer · answered May 24 '13 at 06:58

43

You need to strip off the trailing ';' from the lines.

A possible workaround if you know you have 5 columns is:

np.loadtxt('test.csv', delimiter=';', usecols=range(5))

Or, use genfromtext instead which handles missing values

np.genfromtxt('test.csv', delimiter=';')[:,:-1]

answered May 24 '13 at 06:58

wim

You are right, but I can't strip trailing `;` for some lines as there are missing values. Using `usecols=range(5)` if line ends with `;;` (i.e. last value is missing) yields to error again – theta May 24 '13 at 07:04
1

Maybe I should use other module to read CSV. I tried `pandas` right now, and it reads just fine as anyone would have expected, but OTOH I don't want to load huge package just to read CSV to array, and don't use it again... – theta May 24 '13 at 07:09
Nah I think numpy is a good choice, there is `csv` module but it's more for strings .. – wim May 24 '13 at 07:33
3

I just got a tip to use `np.genfromtxt('test.csv', delimiter=';')` instead - works like a charm. I must have overlooked your last line – theta May 24 '13 at 09:29
Under Windows I have problem with no-ANSI encoded files – themadmax Jun 18 '15 at 13:13

score 18 · Answer 2 · answered Jun 21 '18 at 01:07

So in my case the csv file had column names written in the first row. e.g.

Column1,Column2,Column3
5.4,2.3,2.4
6.7,3.6,9.3

So as given in the docs, all I had to do was use the skiprows parameter

So the API call became,

np.loadtxt('test.csv', delimiter=',', skiprows=1)

2 Answers2