0

I have a data like this:

## df
Col1     Col2                  Col3
A234     
B678     "L" shaped, cool        41

The delimiter is \t.

df = pd.read_csv(filename, sep = "\t", dtype = str, quotechar = '""')

This is not correct.
Otherwise, I tried to import using R language data.table.

> df <- fread(filename, sep = "\t")

Found and resolved improper quoting out-of-sample.
>>. If the fields are not quoted (e.g. field separator does not appear within any field), try quote="" to avoid this warning.

Any solution on Python?

Or I need to use csv package with argument quoting = csv.QUOTE_ALL.

text file looks: (you can copy below into a text document)

Col1    Col2    Col3
A234        
B678    "L" shaped, cool    41
Peter Chen
  • 1,464
  • 3
  • 21
  • 48
  • There are some spaces in `"L" shaped, cool`, leading to the improper output – PPP Feb 24 '22 at 16:22
  • yes. That is an issue. `R` language `data.table`'s `fread()` can handle this issue successfully. Any solution on Python? – Peter Chen Feb 24 '22 at 16:25
  • According to [How to make separator in pandas read_csv more flexible wrt whitespace, for irregular separators?](https://stackoverflow.com/questions/15026698/how-to-make-separator-in-pandas-read-csv-more-flexible-wrt-whitespace-for-irreg), perhaps `delim_whitespace=True` may help. – PPP Feb 24 '22 at 16:29
  • Could you provide the string of sample csv with original separator? – PPP Feb 24 '22 at 16:39
  • 1
    You have extra " in quotechar. try `pd.read_csv(filename, sep = "\t", dtype = str, quotechar = '"')` If this doesn't work, could you explain "This is not correct."? error? what is the output you get? – Emma Feb 24 '22 at 16:56
  • import csv ; df = pd.read_csv(filename, sep='\t', quoting = csv.QUOTE_NONE) ? – rehaqds Feb 24 '22 at 20:42

1 Answers1

0

Seems like a possible issue is the use of 4 spaces instead of a \t in your file. A possible solution is to use the sep to explicitly indicate that 4 spaces must be used a separator:

  df = pd.read_csv(filename, sep=" "*4, engine="python")  # Explicit engine to avoid ParserWarning
aaossa
  • 3,763
  • 2
  • 21
  • 34