MARK parity detection with CMSPAR very unreliable (9 bit UART)

Question

edit-2: Turns out this might actually be a limitation of many USB based adapters, as they always transfer multiple bytes at once to the host, without a place to put information about the individual bytes. So if any of the bytes produced an error of some sorts, the whole transfer will be marked, leaving no way to determine which byte caused it.

edit: I just tried this code with the internal UART of the MT7688 where it works marvelously. So this seems to be a bug in the ftdi_sio driver or maybe even the hardware itself. I might try to implement this with libftdi1/ libftd2xx to rule that out.

---

I am trying to communicate with a embedded device that uses a serial protocol using Linux.

For this I am using a FT232RL USB-Serial adapter.

The protocol marks the first byte of a frame with the 9th (parity) bit set.

This I can reproduce with the CMSPAR option (mark/space parity) and indeed, sending frames this way works fine and very reliable.

static void _write_byte_mark(int fd, unsigned char c) {

    struct termios tio = {0};
    if (tcgetattr(fd, &tio))
        printf("%s:%d : error %d\n", __func__, __LINE__, errno);

    tio.c_cflag |= PARODD;
    if (tcsetattr(fd, TCSADRAIN, &tio))
        printf("%s:%d : error %d\n", __func__, __LINE__, errno);

    // write byte with MARK parity
    if (write(fd, &c, sizeof(c)) <= 0)
        printf("%s:%d : error %d\n", __func__, __LINE__, errno);

    // set back to SPACE parity
    tio.c_cflag &= ~PARODD;
    if (tcsetattr(fd, TCSADRAIN, &tio))
        printf("%s:%d : error %d\n", __func__, __LINE__, errno);
}

void send_packet(int fd, const unsigned char* pkg, size_t size) {
    const unsigned char preamble = 0xff;

    // write preamble to sync UARTs (protocol will ignore this)
//  write(fd, &preamble, sizeof(preamble));
    // send first byte with 9th bit set to indicate start of frame
    _write_byte_mark(fd, pkg[0]);
    // send the rest of the pkg
    write(fd, pkg + 1, size - 1);
}

When I receive such packets on Linux, the parity 'error' should be indicated by reading a sequence of the escape byte 0xFF, 0x00 and the the actual value of the byte with the 9th bit set.

static uint16_t uart_read_byte(int fd) {

    uint8_t c;
    read(fd, &c, sizeof(c));

    if (c == 0xFF) { // got escape byte
        read(fd, &c, sizeof(c));

        if (c == 0x00) { // parity bit set
            read(fd, &c, sizeof(c));

            return c | 0x100;
        } else if (c != 0xFF)
            fprintf(stderr, "invalid byte sequence: %x %x\n", 0xFF, c);
    }

    return c;
}

void receive_packets(int fd) {

    struct pollfd pfd = {.fd = fd, .events = POLLIN};
    int bytes = 0;

    while (1) {

        poll(&pfd, 1, -1);
        ++bytes;

        uint16_t byte = uart_read_byte(fd);

        if (byte & 0x100) {
            printf("--[%d bytes]--\n", bytes);
            bytes = 0;
        }

        printf("> %02x|%c\n", byte, byte);
    }
}

This however only 'kind of' works. Only in some cases, the complete sequence 0xFF, 0x00, 0x18 is detected. (0x18 just happens to be the first byte of my packet here), more often then not I will not read the escaping 0xFF but just 0x00, 0x18 (I not even sure how this is possible as 0x00 is never being sent) and sometimes I get parity errors on random payload bytes.

I first suspected a race condition because I might change the TTY settings while I read from it, but even when I use two UARTs and separate processes (one for RX only, the other for TX only) I get results like this.

To set up the TTY, I use

int uart_init_linux(const char* dev, unsigned long baudrate) {

    #define XMIT_FIFO_SIZE 14

    int fd = open(dev, O_RDWR | O_NOCTTY);
    if (fd < 0) {
        printf("can't open %s: error %d\n", dev, errno);
        return fd;
    }

    // Set port Settings
    struct termios tio;
    tcgetattr(fd, &tio);

    cfmakeraw(&tio);
    cfsetspeed(&tio, baudrate);

    tio.c_cflag |=  CMSPAR;         // Set "stick" parity (either mark or space)
    tio.c_cflag &= ~PARODD;         // Select space parity so that only address byte causes error

    // NOTE: The following block overrides PARMRK and PARENB bits cleared by cfmakeraw.
    tio.c_cflag |=  PARENB;         // Enable parity generation
    tio.c_iflag |=  INPCK;          // Enable parity checking
    tio.c_iflag |=  PARMRK;         // Enable in-band marking 
    tio.c_iflag &= ~IGNPAR;         // Make sure input parity errors are not ignored

    if (tcsetattr(fd, TCSADRAIN, &tio))
        printf("tcsetattr failed on %s (fh %d)!\n", dev, fd);

    struct serial_struct serial;

    // set xmit fifo size
    ioctl(fd, TIOCGSERIAL, &serial);
    printf("\tchange xmit buffer from %d to %d\n", serial.xmit_fifo_size, XMIT_FIFO_SIZE);
    serial.xmit_fifo_size = XMIT_FIFO_SIZE;
    ioctl(fd, TIOCSSERIAL, &serial);

    return fd;
}

with uart_init_linux("/dev/ttyUSB0", B500000);

Now is this somehow a hardware issue? Do I not read fast enough? Are there some flags I miss setting?

I have a hard time believing that RX is so unreliable when tools like hterm and screen can receive just fine without so many erroneous bits.