How to read from serial port like picocom on Linux?

Question

I have a gps module that sends data (NMEA sentence) every 1 seconds to the serial port. I've been trying to read it from a c++ program.

When reading the serial port with picocom, data is displayed in a clean way, each line has a NMEA sentence).

The result of my program is close but lines are sometimes mixed.

This is my code:

#include <iostream>
#include <stdio.h>
#include <string.h>
#include <fcntl.h> 
#include <errno.h> 
#include <termios.h> 
#include <unistd.h> 

int main(){

    struct termios tty;
    memset(&tty, 0, sizeof tty);

    int serial_port = open("/dev/ttyUSB0", O_RDWR);

    // Check for errors
    if (serial_port < 0) {
        printf("Error %i from open: %s\n", errno, strerror(errno));
    }

        // Read in existing settings, and handle any error
    if(tcgetattr(serial_port, &tty) != 0) {
        printf("Error %i from tcgetattr: %s\n", errno, strerror(errno));
    }

    tty.c_cflag &= ~PARENB; // Clear parity bit, disabling parity (most common)
    tty.c_cflag &= ~CSTOPB; // Clear stop field, only one stop bit used in communication (most common)
    tty.c_cflag |= CS8; // 8 bits per byte (most common)
    tty.c_cflag &= ~CRTSCTS; // Disable RTS/CTS hardware flow control (most common)
    tty.c_cflag |= CREAD | CLOCAL; // Turn on READ & ignore ctrl lines (CLOCAL = 1)
    tty.c_lflag &= ~ICANON;
    tty.c_lflag &= ~ECHO; // Disable echo
    tty.c_lflag &= ~ECHOE; // Disable erasure
    tty.c_lflag &= ~ECHONL; // Disable new-line echo
    tty.c_lflag &= ~ISIG; // Disable interpretation of INTR, QUIT and SUSP
    tty.c_iflag &= ~(IGNBRK|BRKINT|PARMRK|ISTRIP|INLCR|IGNCR|ICRNL); // Disable any special handling of received bytes
    tty.c_oflag &= ~OPOST; // Prevent special interpretation of output bytes (e.g. newline chars)
    tty.c_oflag &= ~ONLCR; // Prevent conversion of newline to carriage return/line feed
    tty.c_cc[VTIME] = 10;   
    tty.c_cc[VMIN] = 0;
    // Set in/out baud rate to be 9600
    cfsetispeed(&tty, B9600);
    cfsetospeed(&tty, B9600);

    // Save tty settings, also checking for error
    if (tcsetattr(serial_port, TCSANOW, &tty) != 0) {
        printf("Error %i from tcsetattr: %s\n", errno, strerror(errno));
    }

    // Allocate memory for read buffer, set size according to your needs
    char read_buf [24];
    memset(&read_buf, '\0', sizeof(read_buf));

    while(1){
        int n = read(serial_port, &read_buf, sizeof(read_buf));
        std::cout << read_buf ;
    }

    return 0;
}

How does picocom manage to display data correctly? Is is due to my buffer size or maybe VTIME and VMIN flags ?

Answer 1

Your are getting "framing" errors.

You cannot rely on read() to always get exactly one NMEA sentence from beginning to end.

You need to add the data read to the end of a buffer, then detect the start and end of each NMEA sentence in the buffer, removing each detected sentence from the beginning of the buffer as it is found.

Like this:

FOREVER
  read some data and add to end of buffer
  if start of buffer does not have start of NMEA sentence
    find start of first NMEA sentence in buffer
    if no sentence start found
      CONTINUE
    delete from begining of buffer to start of first sentence
  find end of first NMEA sentence in buffer
  if no sentence end in buffer
    CONTINUE
  remove first sentence from buffer and pass to processing

It is important, if you expect a NMEA application to work reliably in the real world, to handle framing errors. This sort of thing:

         received                                       output
$GPRMC,,V,,,,,,,,,N*53
                                                $GPRMC,,V,,,,,,,,,N*53
$GPVTG,,,,,,,,N*30
                                                $GPVTG,,,,,,,,N*30
$GPRMC,,V,,,,,,,,,N*53$GPVTG,,,,,,,,N*30
                                                $GPRMC,,V,,,,,,,,,N*53
                                                $GPVTG,,,,,,,,N*30
$GPRMC,,V,,,
                                                ----
,,,,,,N*53
                                                $GPRMC,,V,,,,,,,,,N*53

The code to do this is available at https://gist.github.com/JamesBremner/291e12672d93a73d2b39e62317070b7f

Answer 2

How does picocom manage to display data correctly?

The "correctness" of the displayed output is merely the human tendency to perceive or attribute "order" (and/or a pattern) to naturally occurring events.

Picocom is just a "minimal dumb-terminal emulation program" that, like other terminal emulation programs, simply displays what is received.
You can tweak the line-termination behavior, for example append a carriage return when a line feed is received (so that Unix/Linux text files display properly).
But otherwise, what you see displayed is what was received. There is no processing or formatting applied by picocom.

Based on the outputs you have posted, the GPS module clearly is outputting lines of ASCII text terminated with line feed and carriage return.
Regardless of how this text is read by a (terminal emulator) program, i.e. a byte at a time or some random number of bytes each time, so long as each received byte is displayed in the same order as received, the display will appear orderly, legible and correct.

---

Is is due to my buffer size or maybe VTIME and VMIN flags ?

The VTIME and VMIN values are not optimal, but the real issue is that your program has a bug that causes some of the received data to be displayed more than once.

while(1){
    int n = read(serial_port, &read_buf, sizeof(read_buf));
    std::cout << read_buf ;
}

The read() syscall simply returns a number a bytes (or an error indication, i.e. -1), and does not return a string.
Your program does nothing with that number of bytes, and simply displays whatever (and everything) that is in that buffer.
Whenever the latest read() does not return sufficient bytes to overwrite what is already in the buffer, then old bytes will be displayed again.

You can confirm this bug by comparing output from your original program with the following tweak:

unsigned char read_buf[80];

while (1) {
    memset(read_buf, '\0', sizeof(read_buf));  // clean out buffer
    int n = read(serial_port, read_buf, sizeof(read_buf) - 1);
    std::cout << read_buf ;
}

Note that the buffer size passed to the read() needs to be one less that the actual buffer size in order to preserve at least one byte location for a string terminator.

Failure to test the return code from read() for an error condition is another problem with your code.
So the following code is an improvement over yours:

unsigned char read_buf[80];

while (1) {
    int n = read(serial_port, read_buf, sizeof(read_buf) - 1);
    if (n < 0) {
        /* handle errno condition */
        return -1;
    }
    read_buf[n] = '\0';
    std::cout << read_buf ;
}

---

You are not clear as to whether you are just trying to emulate picocom, or another version of your program was having issues reading data from your GPS module and you decided to post this XY problem.
If you intend to read and process the lines of text in your program, then you do not want to emulate picocom and use noncanonical reads.
Instead you can and should use canonical I/O so that read() will return a complete line in your buffer (assuming that the buffer is large enough).

Your Linux program is not reading from a serial port, but from a serial terminal.
When the received data is line-terminated text, there is no reason to read raw bytes when (instead) the terminal device (and line discipline) can parse the received data for you and detect the line termination characters.
Instead of doing all the extra coding/processing suggested in another answer, utilize the capabilities already built into the operating system.

For reading lines see Serial Communication Canonical Mode Non-Blocking NL Detection and Working with linux serial port in C, Not able to get full data, as well as Canonical Mode Linux Serial Port for a simple and complete C program.

---

ADDENDUM

I'm having troubles to understand "Instead you can and should use canonical I/O so that read() will return a complete line in your buffer".

I don't know how to write that to be more clear.

Have you read the termios man page?

In canonical mode:

• Input is made available line by line. An input line is available when one of the line delimiters is typed (NL, EOL, EOL2; or EOF at the start of line). Except in the case of EOF, the line delimiter is included in the buffer returned by read(2).

---

Should i expect that each call to read() will return a full line with $... or should i implement some logic to read and fill the buffer with a full line of ASCII text?

Are you wondering if there is a difference between my meaning of "complete" versus your use of "full"?

Did you read the comment where I already wrote "If you write your program as I suggest, [then] that $ should be the first char in the buffer"?
So yes, you should expect "that each call to read() will return a full line with $..." .

You need to study what I already wrote as well as the links provided.

Answer 3

If you just want to print the NMEA frames correctly on your terminal you can first determine the number of bytes stored in the buffer with FIONREAD, just change your loop to:

// Allocate memory for read buffer, set size according to your needs
int bytesWaiting;
while(1){

    ioctl(serial_port, FIONREAD, &bytesWaiting);
    if (bytesWaiting > 1){
        char read_buf [bytesWaiting+1];
        memset(&read_buf, '\0', sizeof(read_buf));
        int n = read(serial_port, &read_buf, sizeof(read_buf));
        std::cout << read_buf;
        }
    }

return 0;
}

I've tested your code with the modified loop using gpsfeed+ which generates gps coordinates and outputs them on NMEA format through the serial port and the printout is perfect (see screenshot). As indicated in the comments below this is just a quick tweak on the original code to make it work properly, at least from a visual point of view but it might not work if your device is sending frames at high frequency.

Of course, there are many more ways to do this, the best I can think of for this particular problem with termios is to use canonical read. See for instance this example from TLDP.