8

I have a set of led data saved on disk (led.csv) as such:

 , LEDC1, LEDC2, LEDC3
0, 54859, 11349, 56859
1, 54850, 12135, 56890
2, 54836, 12400, 56892
3, 54840, 15725, 56897
4, 54841, 19038, 56896
5, 54837, 21232, 56911
.,  ... ,  ... ,  ...

I am reading this data from .csv using pandas read_csv function:

data = pd.read_csv("Data/led.csv", index_col=0)

Providing the index_col argument to this function issues the following (numpy) warning:

C:\Program Files\Python\lib\site-packages\numpy\lib\arraysetops.py:466: 
FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
mask |= (ar1 == a)

What is the proper way of reading a .csv file with existing index using Pandas?

Any insight on the warning is much appreciated...

Mr.Uyan
  • 113
  • 1
  • 9
  • I tried it reading a file with random numbers, setting this numbers as id. It works until 1.000.000 numbers, on 1.000.001 I had this error. – Klaifer Garcia Mar 20 '19 at 19:10

3 Answers3

4

This is one way to get the same result as with index_col = 0 but without the warning. It may not be the most concise way though:

data = pd.read_csv("Data/led.csv")
data.set_index([data.columns.values[0]], inplace=True)
data.index.names = [None]

This is a great post on the type of error and, below it, there is a solution for a named column, e.g., index_col=['0'])

F Humphreys
  • 41
  • 3
3

I'm not sure exactly why you have an error, though a guess is it could occur if you have numeric and non-numeric data in your index column. Then numpy gets confused when it tries to check whether the index is ordered.

A possible hack:

data = pd.read_csv("Data/led.csv")

# assuming first column is named '0'
data['0'] = data['0'].astype(int).fillna(0)
data = data.set_index('0')
jpp
  • 159,742
  • 34
  • 281
  • 339
1

From this discussion, it looks like there is a stray warning in numpy that the developers want to remove: https://github.com/numpy/numpy/issues/6784

My guess (without digging into the pandas source code) is that pandas is first asking whether 0 is a column name (perhaps with 0 in df.columns or something) and then falling back to using 0 as an index when that query returns False.

Hopefully either numpy will remove the warning or pandas will adjust its logic in the future.

FragLegs
  • 424
  • 3
  • 4