Difficulty importing .dat file

Question

I am somehow having difficulty reading in this file into python with pandas read_table function. http://www.ssc.wisc.edu/~bhansen/econometrics/invest.dat

This is my code:

pd.read_table(f,skiprows=[0], sep="")

Which yields error:

TypeError: ord() expected a character, but string of length 0 found

Marcin · Accepted Answer · 2014-12-11T01:40:20.047

22

Dont know about read_table, but you can read this file directly as follows:

import pandas as pd    

with open('/tmp/invest.dat','r') as f:
    next(f) # skip first row
    df = pd.DataFrame(l.rstrip().split() for l in f)

print(df)

Prints:

              0            1             2            3
0     17.749000   0.66007000    0.15122000   0.33150000
1     3.9480000   0.52889000    0.11523000   0.56233000
2     14.810000    3.7480300    0.57099000   0.12111000
...
...

The same can be obtained as follows:

df = pd.read_csv('/tmp/invest.dat', sep='\s+', header=None, skiprows=1)

edited Dec 11 '14 at 01:40

answered Dec 11 '14 at 01:30

Marcin

215,873
14
235
294

Just thought it might be more efficient to do so with built-in functions. Do you know of anyway to do it with built-in functions? – zsljulius Dec 11 '14 at 01:35
you can simply use ``skiprows=0`` – Jeff Dec 11 '14 at 01:35
Thanks a ton. I think the trick is to use regular expression in the sep argument. Cause when I use "\s+", even with read_table, it works. – zsljulius Dec 11 '14 at 01:42
Good to hear its ok now:-) – Marcin Dec 11 '14 at 01:43
Thank you also for the `\s+`. The number of space being inconsistent during the import you can't use a simple `' '` space. – Yohan Obadia Dec 13 '16 at 21:24

Difficulty importing .dat file

1 Answers1

Linked