How can I fix pickling Arabic text problem?

Question

I am getting the problem while trying to pickle Arabic text to store it in a variable. This is code:

# load the  train data & store it..
with open('files/0007.txt', 'rb') as f:
    train_data = pickle.load(f)

Error:

UnpicklingError                           Traceback (most recent call last)
<ipython-input-5-fa360e8454ee> in <module>
      1 # load the  train data & store it in a file
      2 with open('files/0007.txt','rb') as f:
----> 3     train_data=pickle.load(f)

UnpicklingError: invalid load key, '\xd8'.

Was the pickle file created by the same version of python you are using to load the pickle file? — user1558604, Mar 14 '21 at 14:58
possible duplicate of https://stackoverflow.com/a/17307606/3358570 — Talha Junaid, Mar 14 '21 at 14:58
Also, it seems odd that a pickle file has a .txt extension. Most people use .pkl for pickle files — user1558604, Mar 14 '21 at 14:58
"what do you mean by pickle arabic text" i mean: when i use same code with english text/corpus it works, but not works with arabic text/corpus — El Houssain Sabri, Mar 14 '21 at 15:02
here i use this code with texts in english, and it works:with open('files/train_qa.txt','rb') as f: train_data=pickle.load(f) — El Houssain Sabri, Mar 14 '21 at 15:03

score 0 · Accepted Answer · answered Mar 14 '21 at 15:02

0

Reason : file is corrupted

Try compressing the file, upload it and then unzip on the session. It looks like the pickle file is not saved correctly when you upload/downloaded it.

answered Mar 14 '21 at 15:02

Bhargav - Retarded Skills

3,154
1
6
22

How can I fix pickling Arabic text problem?

1 Answers1