I got UnicodeDecodeError in Django while dumping data into JSON, Unicode does not recognize characters of the Russian alphabet

Question

I'm new to Django so I need some help. I needed to create fixtures and did it with this command

python manage.py dumpdata products.ProductCategory > category.json

Then I got this

  {
    "model": "products.productcategory",
    "pk": 1,
    "fields": {
      "name": "╬фхцфр",
      "description": "╬яшёрэшх юфхцф√"
    }
  }

And when I try to loaddata Django gives me an error: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

Are there any ways to encode this JSON or dump data correctly?

I searched in the documentation, but unfortunately I did not find anything worthwhile

Answer 1

It's helpful to post the full stack trace, as it's difficult to determine the origin of the error without it. Is it possible you could update your question to include the full stack trace for the loaddata error?

That being said, it's possible your database is not configured to use UTF-8. Depending on what database system you use, you may need to add extra configuration.

See the docs here: https://docs.djangoproject.com/en/4.1/ref/unicode/

If you are using SQLite, then you shouldn't need to do any extra configuration.

As a note, dumpdata has an option for output: -o OUTPUT. This could be used instead of >. The full command would be:

python manage.py dumpdata products.ProductCategory -o category.json

One last thing you could try, is this:

• Open the file in Notepad.
• Select Save As.
• Select Encoding "UTF-8".
• Save the file.

See here: Python: Can dumpdata cannot loaddata back. UnicodeDecodeError