Writing live data from an Arduino to a CSV in Python

Question

I am using Python 3.5 and I am having a little bit of trouble trying to print my live data to a CSV for post processing. I am using an Arduino to gather data from a strain gauge and a photo sensor for a live updating Dyno for my schools car team. So far I have managed to get the last line of data to print to a CSV but that is not enough.

I need it to print all of this data after the team is done with their tests to see what their modifications did to the engine. I do have all of their relevant data displaying in a serial monitor, but it would be great for them if they could have the data for reviewing previous tests.

import serial
import csv
import time
import numpy as np
import warnings
import serial
import serial.tools.list_ports

arduino_ports = [
    p.device
    for p in serial.tools.list_ports.comports()
    if 'Arduino' in p.description
]
if not arduino_ports:
    raise IOError("No Arduino found")
if len(arduino_ports) > 1:
    warnings.warn('Multiple Arduinos found - using the first')

Arduino = serial.Serial(arduino_ports[0])
Arduino.flush()
Arduino.reset_input_buffer()

start_time=time.time()
Distance = 0.5 # This is how long the lever arm is in feet

while True:
    while (Arduino.inWaiting()==0):
        pass
    try:
        data = Arduino.readline()
        dataarray = data.decode().rstrip().split(',')
        Arduino.reset_input_buffer()
        Force = float(dataarray[0])
        RPM = float (dataarray[1])
        Torque = Force * Distance
        HorsePower = Torque * RPM / 5252
        Run_time = time.time()-start_time
        print (Force , 'Grams',"," , RPM ,'RPMs',"," ,Torque,"ft-lbs",",", HorsePower, "hp", Run_time, "Time Elasped")
    except (KeyboardInterrupt, SystemExit,IndexError,ValueError):
        pass
    with open('DynoData.csv', 'w') as outfile:
        outfileWrite = csv.writer(outfile)
        outfileWrite.writerow([Force, RPM])

    outfile.close()

hunteke · Accepted Answer · 2017-10-22T01:09:11.033

2

You have wrapped your file opening and closing code inside of the while loop: the with block. To a first-order approximation, the with statement uses a Context Manager to ensure that "something happens, no matter what". In this case, it ensures that the file is closed as soon as the with block ends. This has the effect of creating a new file (named DynaData.csv) for every loop iteration, and thus the only data that survives is the last piece of data written, a single line of your CSV output.

Note that this means your attempt to close the file:

outfile.close()

is redundant at best, if the interpreter doesn't outright throw an exception.

If you want to constantly write to the file as data comes in, enclose your while loop inside the with open... block:

with open('DynoData.csv', 'w') as outfile:
    outfileWrite = csv.writer(outfile)
    while True:
        ...
        # code to collect your data omitted
        ...
        outfileWrite.writerow([Force, RPM])

For completeness, note that you can optionally outfile.flush() at every iteration as well. The file will flush it's unwritten buffer at each newline written or when the internal buffer is full anyway, so experiment if you need or want that.

edited Oct 22 '17 at 01:09

answered Oct 18 '17 at 03:26

hunteke

3,648
1
7
17

That seemed to have solved my issue, however when it saves the information it seems to have a blank space between rows. This isn't a critical issue however to have a neat spreadsheet of all of their data I think it would be nicest to not a spaces between each row. For example a sample set with 20 data points would have 40 rows used. – Matthew Espindola Munn Oct 18 '17 at 23:36
Just noted one mistake in what I gave you: move the ` = csv.writer(outfile)` to _outside_ of the while loop. Obviously not detrimental, as it worked, but not correct either. That's the only source of an extra newline I can see. Now there is exactly one statement that writes to the file. See if that fixes your issue? – hunteke Oct 19 '17 at 12:39
Sadly there was no change in the output file with that change, but I appreciate knowing the proper way to make this portion of a code. – Matthew Espindola Munn Oct 20 '17 at 21:30
Regarding the extra newlines, have you seen [CSV in Python adding an extra carriage return](https://stackoverflow.com/questions/3191528/csv-in-python-adding-an-extra-carriage-return)? – hunteke Oct 22 '17 at 01:08
It would appear that by adding `newline= ' '` has fixed my problem. Thank you – Matthew Espindola Munn Oct 23 '17 at 18:03

Writing live data from an Arduino to a CSV in Python

1 Answers1