0

I'm migrating a Django app from 1.11.16 to 4.0.5 and django-picklefield will be migrated also from 2.0 to 3.1 .

A django object has the following line:

from picklefield.fields import PickledObjectField

class A(models.Model):
    ...
    log_list = PickledObjectField(default=list)

.

Using the old code (1.11.16) and the new code (4.0.5) results a totally different result for the query:

#old
Obj.objects.get(id=4737).log_list
[{u'date': datetime.datetime(2022, 6, 27, 12, 54, 50, 746392),
u'message': u'Fetching trademark search started',
u'typ': u'info'},
{u'date': datetime.datetime(2022, 6, 27, 12, 54, 53, 423384),
u'message': u'Fetching trademark search finished',
u'typ': u'info'}]
#type == list

#new
Obj.objects.get(id=4737).log_list
gAJdcQEofXECKFgEAAAAZGF0ZXEDY2RhdGV0aW1lCmRhdGV0aW1....
#so it results exactly what is stored in the db.
#type == str

With picklefield version 3.1 is there anything else I need to do / change to have the behaviour than with version == 2.0 ? What should I do with the new setup to get the list instead of the str?

user2194805
  • 1,201
  • 1
  • 17
  • 35
  • Does this version upgrade also involve upgrade of the python version from 2.x to 3.x? – Abdul Aziz Barkat Jun 30 '22 at 15:28
  • Yes, from 2.7.18 to 3.8.10 (or similar). – user2194805 Jun 30 '22 at 15:31
  • and as I can see Django >= 4.0 does not support django-picklefield < 3.0 (because of the force_text vs force_str import problem) . – user2194805 Jun 30 '22 at 15:33
  • Does this answer your question? [Unpickling a python 2 object with python 3](https://stackoverflow.com/questions/28218466/unpickling-a-python-2-object-with-python-3) – Abdul Aziz Barkat Jun 30 '22 at 15:43
  • From what I understand by looking at django-picklefield's [code](https://github.com/gintas/django-picklefield/blob/f1ea1803deaa45b7c006a98cc8697cdf6df915b8/picklefield/fields.py#L162-L164) if it fails to unpickle it's simply going to return the original value it got. From there you can follow above linked question to actually unpickle the data. Best would be to create a data migration to do this and potentially also shifting to using [JSONField](https://docs.djangoproject.com/en/4.0/ref/models/fields/#jsonfield) – Abdul Aziz Barkat Jun 30 '22 at 15:47
  • 1
    Thanks for pointing this out to me. So, actually the data is good, but the new version of python can't use it because of the change in the bytes vs str between python 2 and 3... . – user2194805 Jun 30 '22 at 15:50
  • So the data is good, but needs to be converted. Do You think would it worth to migrate all PickledObjectField to JsonField? Is that more robust? – user2194805 Jun 30 '22 at 18:07
  • It would be good to use `JSONField` due to some reasons: 1) It will use an actual type native to the database; 2) it won't rely on pickling the object. Of course there might be an issue with the JSONField about serializing the dates. One part that the pickle can be said to limit you to is that it adds a hard dependency on Python, other languages cannot read it (at least easily). Whereas with a JSON it is as simple as deserializing the JSON string. – Abdul Aziz Barkat Jun 30 '22 at 18:16
  • Yes, I just tried, and I use heavily datetime objects with the PickledField. At this project I don't mind to be binded to python, however, moving this stuff from python2 to python3 seems to be impossible at the moment... . – user2194805 Jun 30 '22 at 18:22

0 Answers0