How to store multiple values received from socket in a structure in python

Question

I want to receive multiple messages from socket and I would like to store it in an structure like in c++. which will be used for further verification.

Here is my code:

def Receive(event,self):
   
    message = socket.recv(0.1)

message is an object which has attributes like message.arbitration , message.timestamp I want to store the message.timestamp and other attributes from upcoming recv socket API in structure some how like this:

struct canmessage[0].timestamp[0]=message.timestamp

For example if I receive 100 messages, I want to store the 100 messages in a structure in python.

How to implement this logic in python.

I have tried list and dict but nothing worked out.

Can anyone please help resolve this issue.

Answer 1

The struct module can convert Python data to and from a byte stream suitable for transmission over a socket. I can be used when you have a fixed format structure being transmitted or received.

In the following example, the sent or received packet follows the format of an network-ordered (big-endian) double followed by a 10-byte UTF-8-encoded string padded with null bytes if necessary. The storage structure uses a datetime object and a Python Unicode str:

from datetime import datetime
import struct

class Message:

    def __init__(self, ts, arb):
        self.timestamp = ts
        self.arbitration = arb

    def __repr__(self):
        return f'Message(timestamp={self.timestamp!r}, arbitration={self.arbitration!r})'

    def serialize(self):
        return struct.pack('!d10s', self.timestamp.timestamp(), self.arbitration.encode())

    @classmethod
    def deserialize(cls, raw):
        ts, arb = struct.unpack('!d10s', raw)
        return Message(datetime.fromtimestamp(ts), arb.decode().rstrip('\x00'))

x = Message(datetime.now(), 'Yes?')
data = x.serialize()
print(data)
msg = Message.deserialize(data)
print(msg)
print(msg.timestamp.ctime(), msg.arbitration)

Output:

b'A\xd8D\xbb\x15R\xdb4Yes?\x00\x00\x00\x00\x00\x00'
Message(timestamp=datetime.datetime(2021, 8, 10, 14, 15, 1, 294629), arbitration='Yes?')
Tue Aug 10 14:15:01 2021 Yes?

If you don't have a fixed format, JSON is a popular, readable format for transmitting data:

from datetime import datetime
import json

class Message:

    def __init__(self, ts, arb):
        self.timestamp = ts
        self.arbitration = arb

    def __repr__(self):
        return f'Message(timestamp={self.timestamp!r}, arbitration={self.arbitration!r})'

    def json(self):
        data = {'timestamp': self.timestamp.timestamp(), 'arbitration': self.arbitration}
        return json.dumps(data, ensure_ascii=False).encode()

    @classmethod
    def from_json(cls, raw):
        data = json.loads(raw)
        return Message(datetime.fromtimestamp(data['timestamp']), data['arbitration'])

x = Message(datetime.now(), 'Yes?')
data = x.json()
print(data)
msg = Message.from_json(data)
print(msg)
print(msg.timestamp.ctime(), msg.arbitration)

Output:

b'{"timestamp": 1628630908.680338, "arbitration": "Yes?"}'
Message(timestamp=datetime.datetime(2021, 8, 10, 14, 28, 28, 680338), arbitration='Yes?')
Tue Aug 10 14:28:28 2021 Yes?

You can also use the pickle module which is meant for serializing/deserializing arbitrary Python objects, but note you have to trust the data received as unpickling arbitrary objects can be unsafe.

from dataclasses import dataclass
import pickle

@dataclass
class Message:
    timestamp: datetime = datetime.now()
    arbitration: str = ''

data = pickle.dumps(Message(datetime.now(), 'Yes?'))
print(data)
msg = pickle.loads(data)
print(msg)

Output:

b'\x80\x04\x95j\x00\x00\x00\x00\x00\x00\x00\x8c\x08__main__\x94\x8c\x07Message\x94\x93\x94)\x81\x94}\x94(\x8c\ttimestamp\x94\x8c\x08datetime\x94\x8c\x08datetime\x94\x93\x94C\n\x07\xe5\x08\n\x0e\x12#\x0e^\xa4\x94\x85\x94R\x94\x8c\x0barbitration\x94\x8c\x04Yes?\x94ub.'
Message(timestamp=datetime.datetime(2021, 8, 10, 14, 18, 35, 941732), arbitration='Yes?')

Transmitting and receiving the byte data packets is left as an exercise for the OP . The struct format is fixed-sized (read 18 bytes from the socket). json can use socket.makefile and readline() for reading a line at a time if transmitted in a single newline-terminated line. pickle can read a pickled object(s) directly from a socket wrapped in a makefile.

Answer 2

Maybe this thread here is helpful. It is suggested to use the dataclass module to get struct like behaviour:

from dataclasses import dataclass
@dataclass
class Message:
  timestamp: list = []
  arbitration: list = []

message = socket.recv(0.1)
can_messages = [Message()]
can_messages[0].timestamp.append(message.timestamp)

I am not sure if this is exactly what you are looking for, but I hope it can help you.