Persist datetime values for use with gviz api

Question

I have an python application that collects data from an MQTT broker and presents it to a website via the gviz python API:


DESCRIPTION= [      \
              ('Zeit', 'datetime'),     \
              ('Temperatur', 'number'),  \
              ('Feuchtigkeit', 'number'),\
              ('Batterie', 'number')    \
              ]

def sendAnswer(conn):
#protect against the data supplier
    Mutex.acquire()
    Trans = deepcopy(DatArr)
    Mutex.release()
#create and populate the DataTable    
    data_table = gviz_api.DataTable(DESCRIPTION)
    data_table.LoadData(Trans)
    Answer = data_table.ToJSon()
#send ti to the webserver
    lng = len(Answer)
    try:
        conn.sendall(bytes("L{:06d};".format(lng),"UTF-8"))
        conn.sendall(bytes(Answer,"UTF-8"))
    except BaseException:
        # if anything goes wrong, try again next time
        pass

def on_message(client, userdata, message):
    global Last, DatArr
#get the data from the broker
    cur = json.loads(str(message.payload, encoding='utf-8'))
    if cur == Last and len(DatArr)>2 : return
    now = datetime.now()
# protect against the webserver
    Mutex.acquire()
#add the data
    DatArr.append([now, cur["temp"], cur["hum"], cur["bat"]])
#cleanup old values
    Last = cur
    for i in range(len(DatArr)):
        if now - DatArr[0][0] > timedelta(days=1): 
            DatArr.pop(0) 
        else:
            break
    Mutex.release()

This works, but instead of keeping the data in the python variable I want to persist it in a file (preferrably JSON). But I cannot JSON.dump() a datetime variable and I cannot .LoadData() a string into a gviz DataTable. The python gviz also lacks an "addRow()". Any suggestions?

Much thanks in advance!

I'm not familiar with the python gviz API, but when using the google charts data table as json, you can use the [date string representation](https://developers.google.com/chart/interactive/docs/datesandtimes#dates-and-times-using-the-date-string-representation) which does not require the `new` operator -- not sure if this helps... — WhiteHat, Mar 24 '20 at 17:50
Thanks, but you are referring to a page that deals with Java Script: "The date and datetime DataTable column data types utilize the built-in JavaScript Date class." The question was about a python implementation. My problem is, that I can dump the datetime as string (using the str() operator) but then the type information is lost and gviz refuses to load the data. Thanks anyway :-) — tommiport5, Mar 24 '20 at 18:04

score 0 · Answer 1 · answered Mar 25 '20 at 18:44

Based on the answers to this question: JSON datetime between Python and JavaScript I found a solution and implemented it in a python module:

import json
import datetime

class DateTimeJSONEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime.datetime):
            return dict(nested_datetime=obj.isoformat())
        else:
            return super(DateTimeJSONEncoder, self).default(obj)

def datetime_decoder(d):
    if len(d) == 1 and 'nested_datetime' in d: 
        return datetime.datetime.strptime(d['nested_datetime'], '%Y-%m-%dT%H:%M:%S.%f')
    result = {}
    for prop in d:
        if isinstance(d[prop], dict):
            result[prop] = datetime_decoder(d[prop])
        else:
            result[prop] = d[prop]
    return result

The class and the function go as named parameters into the json.dump and json.load functions like this:

 DatArr = json.load(DatFile, object_hook=djson.datetime_decoder)

and

json.dump(DatArr, DatFile, cls=djson.DateTimeJSONEncoder)

This persists the formerly global variable DatArr in the json file DatFile. Thanks to all the posters to the above question for providing the information.

Persist datetime values for use with gviz api

1 Answers1