Serial missing data in RPI3 using Python

Question

I'm using raspberry pi 3 and this code to send a request to a device and receive the response from.

#!/usr/bin/python3.7    

import socket               # Import socket module
import thread
import time
import serial

ser = serial.Serial(
    port='/dev/ttyUSB0',
    baudrate=115200,
    parity=serial.PARITY_NONE,
    stopbits=serial.STOPBITS_ONE,
    bytesize=serial.EIGHTBITS
)

input = '5A03010d0a75'    
print "Sending request... "+input
ser.write(input.decode("hex"))
print "Request sent."

output=""
while True:
  output += ser.read(1)
  #time.sleep(0.1)
  print "Reading... "+output.encode('hex')

It handles the response but there are missing bytes, it should receive a 56 bytes length string instead of 53.

This is the output:

a5030119010000010001000a20120118180130090100020505030117501701051421000301040120010516039833630004060104c200007d

There are 3 missing bytes

The serial configuration is what the manufacturer says in the documentation. This device works well with my other application made in Delphi.

EXTRA This is a comparison from my delphi app and this py script:

Delphi app
A5030119010000010001000A20120118180130090100020505030117501701051421000301040120010516039833630004060104C200007D
Python script
a503011901000001000100    1201181801300901000205050301175017010514210003010401  010516039833630004060104c200007d

What happens if you read everything in the buffer instead of 1 byte at a time (removing the 1 from `serial.read(1)`)? And what about setting a timeout in the instantiation of your port? — Marcos G., Jul 08 '19 at 18:02
My best guess is your print is slowing down the loop enough for the buffer to get filled and at some iteration it's overflowing and dropping data. Have you tried printing only at the end of the loop? — Marcos G., Jul 08 '19 at 18:08
For the first idea, same output, for the second idea using `while ser.inWaiting() > 0:` and print out of the loop, same result — Martin Ocando Corleone, Jul 08 '19 at 18:16
Does your Delphi code uses some kind of flow control? What driver are you using? Do you know its buffer size? — Marcos G., Jul 08 '19 at 18:39
For my delphi app, I used the CPort library, just called its functionsthe buffer size is settled at 1024 — Martin Ocando Corleone, Jul 08 '19 at 19:59

score 1 · Answer 1 · answered Jul 09 '19 at 00:39

1

The solution was to set the max byte to the serial.read() method This should be related to the device work behavior

#!/usr/bin/python3.7
#sudo python /home/testing.py

import serial
import time


ser = serial.Serial(
    port='/dev/ttyUSB0',
    baudrate=115200,
    parity=serial.PARITY_NONE,
    stopbits=serial.STOPBITS_ONE,
    bytesize=serial.EIGHTBITS,
    timeout=5
)

input = '5A03010d0a75'    
print "Sending request... "+input
ser.write(input.decode("hex"))
print "Request sent."

output=""
time.sleep(1)
while ser.inWaiting() > 0:
  output += ser.read(10) #setting it to 10 will fix this problem

print "Reading... "+output.encode('hex')

answered Jul 09 '19 at 00:39

Martin Ocando Corleone

134
13

This was my [first](https://stackoverflow.com/users/11476836/marcos-g) suggestion above. Together with my [second](https://stackoverflow.com/users/11476836/marcos-g) tip of dropping the `print` reduced the computing overhead enough to avoid overflowing the RX buffer. Probably a matter of opinion but I think most people here would agree that the very least you should always do in these cases is to mention the helpful comments. Maybe I'm too polite but for me, the right way, in this case, would have been to invite the originator of the helpful comments to write an answer first. – Marcos G. Jul 09 '19 at 05:55
Some examples of polite behaviour: [1](https://stackoverflow.com/users/8689000/samox), [2](https://stackoverflow.com/a/56926205/11476836), [3](https://stackoverflow.com/a/56881819/11476836) – Marcos G. Jul 09 '19 at 05:57
Excuse me if I do sounds rude, isn't my intention, but I don't how your first suggestion is equal to this answer, about the second one, even if I put the print inside then it works, the problem was to set the max byte, I'm still investigating why, but works. Anyway, I appreciate of course your help. Kind regards – Martin Ocando Corleone Jul 09 '19 at 17:26
I'm sorry Martin, you're 100% right, I jumped into conclusions too fast. The reason your code works when you ask to read 10 bytes instead of 1 is that you reduce a lot the overhead (in this case the number of iterations on the loop that are not doing anything, only wasting time). My first comment above was in that direction but I was actually wrong because `serial.read(1)` is exactly the same as `serial.read()`. I wrongly though `serial.read()` meant to read as much data as available in the RX buffer. Please accept my apologies. – Marcos G. Jul 09 '19 at 17:47
If you want to improve your code you can replace `ser.read(10)` with `ser.read(ser.inWaiting())`. That should read all data available in the RX buffer reducing the iterations on the while loop to a minimum. That is what I should have written on my comment above. – Marcos G. Jul 09 '19 at 17:59

Serial missing data in RPI3 using Python

1 Answers1