0

I have an excel file that contains a table of numbers. When I opened the file and tried to cast the number in the table to integer, I somehow got the following error.

ValueError: could not convert string to float: '2.500.000\ufeff'

Weirdly, when I checked the original file using software like WPS, the number looked fine, it only shows '2.500.000'. I couldn't fix something that doesn't look wrong. Other numbers in table though could be read just fine.

It's just when I opened the file using pandas, I got the above error. It seemed to only occur in just one data. I have tried to google it, but nothing seemed to explain what ufeff is. I assume it's some kind of encoding, but I have no clue. How do I fix this?

catris25
  • 1,173
  • 3
  • 20
  • 40
  • 3
    Does this answer your question? [u'\ufeff' in Python string](https://stackoverflow.com/questions/17912307/u-ufeff-in-python-string) – Umair Ayub Feb 28 '20 at 04:20
  • @Umair it seemed like the number is using some kind of encoding, but only that number only, not the whole file. I applied some kind of encoding, like `encoding='utf-8-sig'` to open the file like in the example from that question you referred, but it doesn't seem to give any effect. – catris25 Feb 28 '20 at 04:34
  • Its BOM, try to manually convert it like `your_variable.encode('utf-8-sig')` – Umair Ayub Feb 28 '20 at 04:59
  • It might be an invisible character. You could try re-entering that cell in WPS (clear and re-enter manually) and see if that fixes it? – nneonneo Feb 28 '20 at 07:00
  • @Umair I tried that, but now none of the data gets converted. I got this error instead `b'\xef\xbb\xbf128000'`. Because like I said, it seemed like other data in the file have different encoding (if that's possible). – catris25 Feb 28 '20 at 07:01
  • @nneonneo apparently, there is more than one data with this problem. I tried retyping the data again for that particular row, but I just found out I have multiple data with the same problem. So I guess re-entering the data doesn't sound like a feasible solution if you have many of them. – catris25 Feb 28 '20 at 07:14
  • So I ended up copying the file content and pasting it to a new file, and did a few other tricks. I didn't run into any other error, so I guess it works so far. It's not the best solution I know, but it works. – catris25 Feb 28 '20 at 10:05

0 Answers0