Reading from a sensor and writing to a CSV file

Question

I have a sensor which prints to my COM4 port. I can read from this using serial like:

import serial
ser = serial.Serial('COM4')
while True:
    if ser.in_waiting > 0:
        temp = ser.readline()
        print(temp)

Is there an efficient way to write this to a CSV file instead of printing to the console?

The problem I'm experiencing is if I stop the script midstream then the changes to the file seem to get thrown away, even if I include a step of writing the header in before the while True loop.

Does this answer your question? [How do I read and write CSV files with Python?](https://stackoverflow.com/questions/41585078/how-do-i-read-and-write-csv-files-with-python) — Pranav Hosangadi, Jun 02 '21 at 14:50
I should have been more specific, for my case when I have a live sensing that's being read from, if I stop the reading script then the file doesn't get updated and all changes are lost. — Morten Nissov, Jun 02 '21 at 19:01
Ah, I see. You want to `flush()` the file after writing to it. https://stackoverflow.com/questions/3976711/csvwriter-not-saving-data-to-file-the-moment-i-write-it — Pranav Hosangadi, Jun 02 '21 at 19:24
does this result in a significant slow-down? For example versus opening the file at each `readline()` and added the latest measurement? — Morten Nissov, Jun 02 '21 at 19:36
File IO is expensive, so I expect it does. You could try it and see if it's too much for your liking. Alternatively, you could listen for `SIGINT`/`SIGTERM` and close the file before exiting when that happens. https://stackoverflow.com/questions/18499497/how-to-process-sigterm-signal-gracefully — Pranav Hosangadi, Jun 02 '21 at 19:39

Thunder Coder · Answer 1 · 2021-06-02T20:35:50.913

1

import csv
import serial
ser = serial.Serial('COM4')
while true:
  with open("csv-file-name.csv", 'w') as csvfile:
      writer = csv.writer(csvfile, delimiter=",")
      if ser.in_waiting > 0:
        temp = ser.readline()
        writer.writerow(temp)

Check out this page for more info.

edited Jun 02 '21 at 20:35

answered Jun 02 '21 at 14:48

Thunder Coder

104
14

Are you replacing 'import serial' with 'import csv'? – Jun 02 '21 at 14:51
1

No, you should have both. Edited the code – Thunder Coder Jun 02 '21 at 14:52
This is what i had originally but if I stop this code abruptly then it's as if the file hasn't been touched. I chalked it up to improper closing of the file. – Morten Nissov Jun 02 '21 at 19:00
That is because the with clause closes the file once whatever is in it is done running, you should try switching the order and putting the with clause within the while loop. Edited the code. – Thunder Coder Jun 02 '21 at 20:33
then this is roughly equivalent to writing writer.flush() in the while loop as well? Except opening the file probably takes more effort than flushing to the file? – Morten Nissov Jun 03 '21 at 07:15
Yup, I would think it's pretty much the same, `flush()` is probably better programmatically though. Did you try the code? – Thunder Coder Jun 03 '21 at 07:56

score 0 · Accepted Answer · answered Jun 18 '21 at 11:28

I ended up, following Pranav Hosangadi's advice, handling the sigterm call manually as

import serial
import signal

def signal_handler(signal, frame):
    global interrupted
    interrupted = True


signal.signal(signal.SIGINT, signal_handler)
signal.signal(signal.SIGTERM, signal_handler)
interrupted = False

if __name__ == '__main__':
    ser = serial.Serial('COM4')
    with open(filename, 'w') as f:
        while True:
            if ser.in_waiting > 0:
                temp = ser.readline()
                f.write(temp)

            if interrupted:
                break

score 0 · Answer 3 · edited Jul 09 '21 at 10:26

0

readline is blocking, so in_waiting is not necessary.

Instead of print, just use write to a file:

import serial

port = serial.Serial('COM4')
with open("output.csv", "w") as output:
    for line in port:
        output.write(line)

edited Jul 09 '21 at 10:26

Peter Mortensen

30,738
21
105
131

answered Jun 18 '21 at 11:34

Daniel

42,087
4
55
81

Reading from a sensor and writing to a CSV file

3 Answers3