0

I have an application that ran for months on Stretch and Python 2.7 and 3.5, on a Raspberry Pi Model 3B+, however when I moved it to Buster and Python 3.7 I ran into a very strange problem. The error message I get is "error saving session_id: write() argument 1 must be unicode, not str" from a json.dump save of a list.

(code: json.dump(client.user_data, fout))

If I run the script as sudo, to test for the eventual running as root by systemd, it works fine. When I run it as root through sudo su, it also works fine. When I run it through systemd I get the error. For many hours I have been going in circles trying to zoom in on the problem, but now I'm baffled.

At first I though that the issue was with the unicode changes in Python, so I tried to eliminate that as a source of the problem. I even went as far as to add a function to really make sure I'm dealing with unicode, and not strings. (function utfy_dict)

The list I'm using and trying to save looks like this, with some fields edited for pricacy: (taken from a print(data) output":

{u'sessionId': u'555C703C-607x-bla-bla', u'userInfo': {u'username': u'my_email@gmail.com', u'city': u'my_city', u'isActivated': True, u'firstname': u'Paul', u'lastname': u'mylastname', u'userID': my userId, u'zipcode': u'my Zipcode', u'telephone': u'', u'deviceCount': 0, u'streetAddress': u'my street', u'securityQuestion3': u'NotUsed', u'securityQuestion2': u'NotUsed', u'securityQuestion1': u'NotUsed', u'country': u'NL', u'userLanguage': u'nl-NL', u'tenantID': 5, u'latestEulaAccepted': False}}

The relevant pieces:

        try:
            # save the session-id so we don't need to re-authenticate every polling cycle
            print("save session_id")
            # here comes the offending piece: an exception is raised
            utfy_dict(client.user_data)
            print(client.user_data)
            print("\n")
            with io.open(session_id, "w", encoding="utf-8") as fout:
                json.dump(client.user_data, fout)
        except Exception as error:# (IOError, ValueError,TypeError):
            write_log("error", "error saving session_id: {}".format(error))

    return

def utfy_dict(dic):
    '''
    https://stackoverflow.com/questions/33699343/convert-every-dictionary-value-to-utf-8-dictionary-comprehension
    '''
    if isinstance(dic,bytes):
        return(dic.encode("utf-8"))  # the key is this!
    elif isinstance(dic,dict):
        for key in dic:
            dic[key] = utfy_dict(dic[key])
        return(dic)
    elif isinstance(dic,list):
        new_l = []
        for e in dic:
            new_l.append(utfy_dict(e))
        return(new_l)
    else:
        return(dic)

Systemd service file:

# This service installs a Python script that monitors the CV temperatures.

[Unit]
Description=Installing the cv_mon monitoring script
Requires=basic.target
After=multi-user.target

[Service]
ExecStart=/usr/bin/python /home/pi/cv_app.py
Restart=always
# wait for 10s before the SIGKILL gets send
TimeoutStopSec=10s

# The number of times the service is restarted within a time period can be set
# If that condition is met, the RPi is rebooted
#
StartLimitBurst=4
StartLimitInterval=180s
# actions can be none|reboot|reboot-force|reboot-immidiate
StartLimitAction=none

[Install]
WantedBy=multi-user.target

The get_evo_data function gets called every three minutes. I would like to reiterate that the code ran fine for more than a year on another platform, and it also runs fine as sudo or root, just not under systemd.

Here is an error report: (note the reference to Python 2.7/json ???)

2019-07-31 00:15:30,095 ERROR    JSON decode error reading evo_saved_data
2019-07-31 00:15:30,099 ERROR    Got exception: write() argument 1 must be unicode, not str
2019-07-31 00:15:30,102 INFO     Traceback (most recent call last):
  File "/home/pi/cv_app.py", line 717, in main
    get_evo_data()
  File "/home/pi/cv_app.py", line 314, in get_evo_data
    json.dump(client.user_data, fout)
  File "/usr/lib/python2.7/json/__init__.py", line 190, in dump
    fp.write(chunk)
TypeError: write() argument 1 must be unicode, not str
paulv
  • 1
  • 1
  • I just added User=root and Group=root to the Service section of the Unit, to no avail. I suspect now it is due to a path issue. – paulv Jul 31 '19 at 20:11
  • I found the problem at last. It was not a programming error after all. It was a configuration problem. This quest for help should not have been here in the first place. Sorry! For the solution, look at the command invocation for the Python script in the systemd unit file. It still uses the path to Python(2.7), hence the reference to the 2.7 version of json. Changing that to Python3 fixed it. Duh! – paulv Jul 31 '19 at 20:26
  • When I executed as root, the shebang in the script took care of selecting the correct Python version. Systemd cannot deal with that. – paulv Jul 31 '19 at 20:46

0 Answers0