1

I am reading a text file using pandas.read_csv. A small example of the file can be found here. Below, is the way in which I am reading the file.

df = pd.read_csv('file.txt', error_bad_lines=False, sep=r'\t+', header=None, dtype='|U', engine='python')

I am able to read the file properly and do all work. But, I keep getting messages like the ones shown below, and I will like to ignore them.

Skipping line 689: Expected 81 fields in line 689, saw 265. Error could possibly be due to quotes being ignored when a multi-char delimiter is used.

There are hundreds of such lines (for different line numbers). I have already tried the answer for hiding of warnings shown in here (Hide all warnings). Unfortunately, it does not work. Also, I will like to suppress (not show) only the specific kind of error messages that I have mentioned in this post (Skipping line ... delimiter is used.).

How can I do this?

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Siddharth Satpathy
  • 2,737
  • 4
  • 29
  • 52

1 Answers1

2

Use argument warn_bad_lines=False:

pd.read_csv(error_bad_lines=True, warn_bad_lines=False)
Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Sander van den Oord
  • 10,986
  • 5
  • 51
  • 96