I have an application that ran for months on Stretch and Python 2.7 and 3.5, on a Raspberry Pi Model 3B+, however when I moved it to Buster and Python 3.7 I ran into a very strange problem. The error message I get is "error saving session_id: write() argument 1 must be unicode, not str" from a json.dump save of a list.
(code: json.dump(client.user_data, fout))
If I run the script as sudo, to test for the eventual running as root by systemd, it works fine. When I run it as root through sudo su, it also works fine. When I run it through systemd I get the error. For many hours I have been going in circles trying to zoom in on the problem, but now I'm baffled.
At first I though that the issue was with the unicode changes in Python, so I tried to eliminate that as a source of the problem. I even went as far as to add a function to really make sure I'm dealing with unicode, and not strings. (function utfy_dict)
The list I'm using and trying to save looks like this, with some fields edited for pricacy: (taken from a print(data) output":
{u'sessionId': u'555C703C-607x-bla-bla', u'userInfo': {u'username': u'my_email@gmail.com', u'city': u'my_city', u'isActivated': True, u'firstname': u'Paul', u'lastname': u'mylastname', u'userID': my userId, u'zipcode': u'my Zipcode', u'telephone': u'', u'deviceCount': 0, u'streetAddress': u'my street', u'securityQuestion3': u'NotUsed', u'securityQuestion2': u'NotUsed', u'securityQuestion1': u'NotUsed', u'country': u'NL', u'userLanguage': u'nl-NL', u'tenantID': 5, u'latestEulaAccepted': False}}
The relevant pieces:
try:
# save the session-id so we don't need to re-authenticate every polling cycle
print("save session_id")
# here comes the offending piece: an exception is raised
utfy_dict(client.user_data)
print(client.user_data)
print("\n")
with io.open(session_id, "w", encoding="utf-8") as fout:
json.dump(client.user_data, fout)
except Exception as error:# (IOError, ValueError,TypeError):
write_log("error", "error saving session_id: {}".format(error))
return
def utfy_dict(dic):
'''
https://stackoverflow.com/questions/33699343/convert-every-dictionary-value-to-utf-8-dictionary-comprehension
'''
if isinstance(dic,bytes):
return(dic.encode("utf-8")) # the key is this!
elif isinstance(dic,dict):
for key in dic:
dic[key] = utfy_dict(dic[key])
return(dic)
elif isinstance(dic,list):
new_l = []
for e in dic:
new_l.append(utfy_dict(e))
return(new_l)
else:
return(dic)
Systemd service file:
# This service installs a Python script that monitors the CV temperatures.
[Unit]
Description=Installing the cv_mon monitoring script
Requires=basic.target
After=multi-user.target
[Service]
ExecStart=/usr/bin/python /home/pi/cv_app.py
Restart=always
# wait for 10s before the SIGKILL gets send
TimeoutStopSec=10s
# The number of times the service is restarted within a time period can be set
# If that condition is met, the RPi is rebooted
#
StartLimitBurst=4
StartLimitInterval=180s
# actions can be none|reboot|reboot-force|reboot-immidiate
StartLimitAction=none
[Install]
WantedBy=multi-user.target
The get_evo_data function gets called every three minutes. I would like to reiterate that the code ran fine for more than a year on another platform, and it also runs fine as sudo or root, just not under systemd.
Here is an error report: (note the reference to Python 2.7/json ???)
2019-07-31 00:15:30,095 ERROR JSON decode error reading evo_saved_data
2019-07-31 00:15:30,099 ERROR Got exception: write() argument 1 must be unicode, not str
2019-07-31 00:15:30,102 INFO Traceback (most recent call last):
File "/home/pi/cv_app.py", line 717, in main
get_evo_data()
File "/home/pi/cv_app.py", line 314, in get_evo_data
json.dump(client.user_data, fout)
File "/usr/lib/python2.7/json/__init__.py", line 190, in dump
fp.write(chunk)
TypeError: write() argument 1 must be unicode, not str