UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0

Question

1. I tried to migrate the django file and the sqlite3 database file to MySQL data, and I encountered such an error.

2. create database: create database mysite_db default charset=utf8mb4 default collate utf8mb4_unicode_ci;

   Data migration in db_sqlite3: python manage.py dumpdata > data.json

   input mysql: python manage.py loaddata data.json

---

File "C:\blog_env\lib\site-packages\django\core\management\commands\loaddata.py", line 113, in loaddata
    self.load_label(fixture_label)
  File "C:\blog_env\lib\site-packages\django\core\management\commands\loaddata.py", line 168, in load_label
    for obj in objects:
  File "C:\blog_env\lib\site-packages\django\core\serializers\json.py", line 66, in Deserializer
    stream_or_string = stream_or_string.decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte
(blog_env) PS C:\blog_env\mysite>

Answer 1

I had the same problem recently... here's my solution:

Confirm the encoding format of 'file_path':

1. download and open the file with Notepad++
2. check the lower right corner; there you can see whether your file was encoded in the compatible format, or if it has the Byte Order Marker or BOM sign,
3. if either of these is true, simply 'save as' the correct/required format.

You should be fine with the above.

Answer 2

Just go on the location of file from Where the error is coming.

In my case the error was coming from json.py file in serializers, because my error is like:

File "C:\Users\User\AppData\Local\Programs\Python\Python38\Lib\site-packages\django\core\serializers\json.py",line 66 in Deserializer stream_or_string = stream_or_string.decode() UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

After going to location , just open json.py file with your idle or notepad, so that you can edit that file. Just go in Deserializer function on the line:

stream_or_string = stream_or_string.decode()

and change this line to

stream_or_string = stream_or_string.decode('UTF-16')

and save it,Now your error will be solved.

Answer 3

Partly with Abrahams solution above..

If your using VS Code, look in the lower right hand corner and it might say something like UTF-<xx>. Mine is in UTF-16. Simply click on it, then click 'save with encoding', and select 'utf-8'. I keep forgetting that certain commands from the CLI, when creating files, will create them as UTF-8 (certain commands when running in powershell) $PSDefaultParametervalues['Out-File:Encoding'] = 'utf8'