How can I write a stream of integers with timestamps using writerow to a .csv file?

Question

I have a continuous stream of integers I am receiving from an Arduino uno I have set up. The input is going into PsychoPy (v1.85.2) and I would like to have this stream of numbers continuously saved into a .csv file with timestamps for datalogging purposes.

I have confirmed that I'm receiving input from the Arduino using print port.readline() I'm not sure why, but the actual integer stream simply isn't writing to the .csv file. Only the timestamps are written to the .csv file.

This is my code in PsychoPy:

import serial
import time
import csv

port = serial.Serial("COM3", 9600)
# by default, the Arduino resets on connection,
# give it some time to wake-up.
time.sleep(1)


csvfile = "C:\Users\xxxx\Desktop\csvfile.csv"
while True:
    res = port.readline()
    with open (csvfile,'a') as output:
        writer = csv.writer(output)
        now = time.strftime('%d-%m-%Y %H:%M:%S')
        writer.writerow([now, res])

I'm not sure if this is an issue with the serial reading from the Arduino, the fact that I'm running it through PsychoPy, or (most likely) some error in my code. Assistance is appreciated!

Maybe it's something to do with `port.readline()` returning a string that is terminated with `\n`? Try diagnosing with `writer.writerow([now, 'x' + res + 'y', 'z'])` But from a general coding performance point of view, you shouldn't be opening the file and creating a `csv.writer` on every iteration of the loop. Do those things just once, before the loop starts. — Michael MacAskill, Mar 20 '18 at 21:10
@MichaelMacAskill Thank you so much for your help, it was incredibly useful. `port.readline` was indeed returning a string terminating with `\n`. To fix this, I used a `.strip()` to get rid of any whitespace and converted the string to a float. — Shravan Thaploo, Mar 22 '18 at 14:07

Shravan Thaploo · Accepted Answer · 2018-03-22T14:28:51.123

2

The issue was was that port.readline() was returning a string with a \n(a new line) at the end of the string. To fix this, I used a .strip() to remove all the whitespace before and after the string and finally converted the string to a float value.

while True:
res = port.readline()
resa = float(res.strip())
with open (csvfile,'a') as output:
    writer = csv.writer(output)
    now = time.strftime('%d-%m-%Y %H:%M:%S')
    writer.writerow([resa, now])

edited Mar 22 '18 at 14:28

answered Mar 22 '18 at 14:15

Shravan Thaploo

31
5

You should accept your own answer here to mark this as solved. But again, your file opening and creation of the `csv.writer` should happen before the start of your `while:` loop. At the moment you are doing this operations every time a row is written to the file, which is inefficient: they only need to be done once. – Michael MacAskill Mar 22 '18 at 20:36
@MichaelMacAskill I'm hesitant to mark my answer as correct just yet. I tried opening the file and taking the `csv.writer` out of the loop but I seem to get a _ValueError: invalid literal for float(): 0.0.00_ Fortunately, this doesn't happen when I keep the writer within the loop. I'm not sure why this happens and I've looked here for an explanation but to no avail: https://stackoverflow.com/questions/21943877/python-valueerror-invalid-literal-for-float#_=_ Do you have any ideas as to why this might be happening? – Shravan Thaploo Mar 27 '18 at 13:37
If it works, and performance isn't an issue, just go with what you have. The perfect is the enemy of the good. – Michael MacAskill Mar 27 '18 at 20:38

How can I write a stream of integers with timestamps using writerow to a .csv file?

1 Answers1