3

So before I start I know this is not the proper way to go about doing this but this is the only method that I have for accessing the data I need on the fly.

I have a system which is writing telemetry data to a .csv file while it is running. I need to see some of this data while it is being written but it is not being broadcast in a manner which allows me to do this.

Question: How do I read from a CSV file which is being written to safely.

Typically I would open the file and look at the values but I am hoping to be able to write a python script which is able to examine the csv for me and report the most recent values written without compromising the systems ability to write to the file.

I have absolutely NO access to the system or the manner in which it is writing to the CSV I am only able to see that the CSV file is being updated as the system runs.

Again I know this is NOT the right way to do this but any help you could provide would be extremely helpful.

This is mostly being run in a Windows environment

badrobit
  • 743
  • 1
  • 11
  • 25
  • 1
    So what's your question? What you could do is store the number of lines read in the previous run and then pass this as a param to `read_csv` as `skiprows=num_lines` and just update `num_lines` with each read – EdChum Feb 25 '15 at 18:25
  • So I would open the file as normal but would this block the system from writing to the CSV file while I am reading from it? – badrobit Feb 25 '15 at 18:29
  • You'd have to open the file and lock the access to prevent writing that's a separate question though: http://stackoverflow.com/questions/186202/what-is-the-best-way-to-open-a-file-for-exclusive-access-in-python – EdChum Feb 25 '15 at 18:38
  • I am not looking to prevent writing though as it needs to keep updating I just want to make sure I am not doing anything that would stop the system from being able to write to the csv file – badrobit Feb 25 '15 at 19:18

1 Answers1

1

You can do something like:

tailf csv_file | python your_script.py

and read from sys.stdin

Reut Sharabani
  • 30,449
  • 6
  • 70
  • 88