Using delimiters in a Python TCP stream

Question

I am working on a program using TCP protocol to collect ADS-B messages from an antenna. Since I am new to Python, I used the following scripts to establish connection. The problem is that I receive several messages at the same time (since TCP is stream-oriented). I would like to separate each message using a "\n" delimiter for instance (each message has "@" at the beginning and ";" at the end and the length varies). I have no idea of how to tell Python to separate each message like this, do you have any idea ? Thanks a lot

Python version 3.7.6, Anaconda, Windows 10

import socketserver


class MyTCPHandler(socketserver.BaseRequestHandler):
     """
#     The request handler class for our server.

#     It is instantiated once per connection to the server, and must
#     override the handle() method to implement communication to the
#     client.
#     """

     def handle(self):
        # self.rfile is a file-like object created by the handler;
        # we can now use e.g. readline() instead of raw recv() calls
        self.data = self.rfile.readline().strip()
        print("{} wrote:".format(self.client_address[0]))
        print(self.data)
        # Likewise, self.wfile is a file-like object used to write back
        # to the client
        self.wfile.write(self.data.upper())

if __name__ == "__main__":
      print ("Server online")
      HOST, PORT = "localhost", 10100
      # Create the server, binding to localhost on port 10002
      with socketserver.TCPServer((HOST, PORT), MyTCPHandler) as server:
          # Activate the server; this will keep running until you
          # interrupt the program with Ctrl-C
          server.serve_forever()

import socket
import sys



def tcp_client():

    HOST, PORT = "192.168.2.99", 10002
    data = " ".join(sys.argv[1:])
    
    
    # Create a socket (SOCK_STREAM means a TCP socket)
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
        
        # Connect to server and send data
        sock.connect((HOST, PORT))
        
        
        while True :
            
            sock.sendall(bytes(data + "\n", "utf-8"))
            # Receive data from the server 
            received = str(sock.recv(1024), "utf-8")
            
                 
            print("{}".format(received))

Welcome to the site, Constantin. Is it the received data that you would like separated by \n and @? — forgetso, Jun 29 '20 at 12:30
Hi and thank you ! The received data looks like this : @325A423013558C3985A142EB3667BD472E374F90; and I would like the data to be separated by \n — Constantin, Jun 29 '20 at 12:36

Mateo Lara · Accepted Answer · 2020-06-29T12:48:00.587

You could try using:

acumulator = ""
while True:

    received = str(sock.recv(1024), "utf-8")
    divided_message = received.split('\n')
    if len(divided_message) >= 2:
        print('One mesage: ', acumulator + divided_message[0].strip())
        for i in range(1, len(divided_message) - 1):
            print('One mesage: ', divided_message[i].strip())
        if '\n' in divided_message[-1]:
            print('One mesage: ', divided_message[-1].strip())
            acumulator = ''
        else:
            acumulator = divided_message[-1]
    else:
        acumulator += divided_message[0]

If the message is separated by /n you can divide the message applying a selection technique, like the one presented above. If your messages have a fixed length you could just modify your delimiter.

Thanks a lot ! I tried it and it appears to be working ! I have one line for each message — Constantin, Jun 29 '20 at 13:33

score 0 · Answer 2 · answered Jun 29 '20 at 12:44

I would suggest you use the following approach:

Assuming your messages can't be more than 4GB long, just send the length, packed into exactly 4 bytes, and then you send the data itself. So, the other side always knows how much to read: Read exactly 4 bytes, unpack it into a length, then read exactly as many bytes as that:

def send_one_message(sock, data):
    length = len(data)
    sock.sendall(struct.pack('!I', length))
    sock.sendall(data)

def recv_one_message(sock):
    lengthbuf = recvall(sock, 4)
    length, = struct.unpack('!I', lengthbuf)
    return recvall(sock, length)

That's almost a complete protocol. The only problem is that Python doesn't have a recvall counterpart to sendall, but you can write it yourself:

def recvall(sock, count):
    buf = b''
    while count:
        newbuf = sock.recv(count)
        if not newbuf: return None
        buf += newbuf
        count -= len(newbuf)
    return buf

More detailed description here

Using delimiters in a Python TCP stream

2 Answers2

Linked