0

I am reading data from 3 serial ports concurrently using pyserial and logging this data to 3 separate time-stamped text files with python. However, after some time(less than about 5 minutes), the data written to the text files is not printed correctly.

example of the accurate data:

2017-10-16 10:41:27

Sensor Reading Avg Reading/Avg topLeft: 1
7.00 0.14 midLeft: 2 8.00 0.25 botLeft: 3 9.00 0.33 topRight: 4 10.00
0.40 midRight: 5 11.00 0.45 botRight: 6 12.00 0.50

example of the inaccurate data:

2017-10-16 10:55:11

Sensor Readin 3 g Reading/Avg topLeft: 1
7.00 0.14 midLeft: 2 8.00 0.25 botLeft: 3 9.00 00.50 ht: 4 10.00 0.40 midRight: 5 11.00
0.45 botRight: 6 12.00 0.50 0.45 botRight: 6 12.00 0.50

here is the code I'm using:

# to use microseconds, delete ".replace(microsecond = 0)"

## includes
from __future__ import print_function
import threading, serial, time, io, datetime
from serial import Serial

device1 = "_base_station" ## device name used for filename
addr1 = "COM39" ## edit serial port here

device2 = "_collector" ## device name used for filename
addr2 = "COM40" ## edit serial port here

device3 = "_receiver" ## device name used for filename
addr3 = "COM42" ## edit serial port here

baud = 57600 ## edit baud rate here

## function for initializing serial ports
def init_port(addr, baud):
    return serial.Serial(
        port = addr,\
        baudrate = baud,\
        parity=serial.PARITY_NONE,\
        stopbits=serial.STOPBITS_ONE,\
        bytesize=serial.EIGHTBITS,\
        timeout=.01)

## initialize serial ports
ser1 = init_port(addr1, baud)
ser2 = init_port(addr2, baud)
ser3 = init_port(addr3, baud)

## print port numbers so the user knows the script is working
print("Collecting data from port: " + ser1.portstr + "\r\n") ## give status update
print("Collecting data from port: " + ser2.portstr + "\r\n") ## give status update
print("Collecting data from port: " + ser3.portstr + "\r\n") ## give status update

def handle_data(ser, device):
    while True:
        line = ser.readline() ## get line from serial port
        filename = datetime.datetime.now().strftime("%Y-%m-%d-%H") + device + ".txt" ## create filename based on device and time. updates at midnight.
        filehandle = open(filename, 'a') ## append data to file.

        if str(line) == "#\r\n": ## logging delimiter
            filehandle.write("\r\n" + str(datetime.datetime.now().replace(microsecond = 0)) + "\r\n") ## start each data entry with a timestamp
        else:
            filehandle.write(str(line)[:-1]) ## log line (strip off extra line, data is already being parsed by line)
            filehandle.flush()

        filehandle.close() ## close file
    ser.close() ## close serial port

## create threads
thread1 = threading.Thread(target = handle_data, args = [ser1, device1])
thread2 = threading.Thread(target = handle_data, args = [ser2, device2])
thread3 = threading.Thread(target = handle_data, args = [ser3, device3])

## start threads
thread1.start()
thread2.start()
thread3.start()

I have also tried using multiple scripts without threading and the same problem continues.

Any theories as to why this is happening? Any solutions?

Thanks in advance.

Joseph Forrest
  • 11
  • 3
  • When using multithreading or multiprocessing it's a very good idea to use the idiom: `if __name__ == "__main__":` – Aaron Oct 16 '17 at 16:58
  • It looks like you're concurrently writing to the same file without any synchronization. Launching multiple processes or threads for this, though, seems like overkill, you can do something like this https://stackoverflow.com/questions/16255807/pyserial-is-there-a-way-to-select-on-multiple-ports-at-once – pvg Oct 16 '17 at 17:02
  • Thanks pvg. It looks like its writing to the same file, but the device name is being passed to the filename from the thread argument. I'll check it out. Also, thanks Aaron. I'm brand new to Python, so all help is welcome. – Joseph Forrest Oct 16 '17 at 17:23
  • It's not entirely clear the file name is unique every time but one thing you should do is just take out the concurrency part altogether. Read from one port at a time with a fixed filename over some period of time without threading and see if your data looks good. If not, you have some other problem – pvg Oct 16 '17 at 17:46

1 Answers1

0

The timeout looks suspiciously quick, I would extend the timeout in the serial port. I suspect the fragmented data you are getting is from when the timeout beats the incoming data.

def init_port(addr, baud):
    return serial.Serial(
        port = addr,\
        baudrate = baud,\
        parity=serial.PARITY_NONE,\
        stopbits=serial.STOPBITS_ONE,\
        bytesize=serial.EIGHTBITS,\
        timeout=1.0)

You are using .readline(), it will be looking for '\n' termination, but it will exit early if the timeout is exceeded.

grambo
  • 283
  • 2
  • 10