Python readline ignoring header in csv file

Question

I've written some code that is supposed to be aware of if a CSV file exists and append to it if it's the right format, or erase it and create a new file.

The part where it checks if the format is correct (by checking the CSV headers to see if they are equal) is not working, because for some weird reason the readline() function is ignoring the headers (which should be the first line of the CSV file.

Do note that I don't want to use extra dependencies such as Pandas.

    import os, csv

    path = 'intent_outputs.csv'
    response = {'query':'eggs', 'header':'eggs', 'fulfillmentText':'are tasty', 'buttontext':'VISIT PAGE', 'buttonurl':'eggs.com', 'contextbuttontext':'eggs', 'contextbuttonurl':'eggs.com'}
    output = open(path, 'a')
    csv_columns = ['Query', 'Header', 'Text', 'Button Text', 'Button URL', 'Context Button Text', 'Context Button URL']
    writer = csv.DictWriter(output, fieldnames=csv_columns)
    if not os.path.exists(path):
        print("DOES NOT EXIST")
        output = open(path, 'w')
    else:
        print("EXISTS")
        output = open(path, 'r')
    if(output.readline() != ",".join(csv_columns)):
        print("NOT EQUAL")
        try:
            output = open(path, 'w')
            writer.writeheader()
        except IOError:
            print("IOError")
    else:
        print("EQUAL")
        output = open(path, 'a')
    try:
        row = {'Query':response['query'], 'Header':response['header'], 'Text':response['fulfillmentText'], 'Button Text':response['buttontext'], 'Button URL':response['buttonurl'], 'Context Button Text':response['contextbuttontext'], 'Context Button URL':response['contextbuttonurl']}
        writer.writerow(row)
    except IOError:
        print("I/O error")

Here's ~~Johnny~~ intent_outputs.csv:

Query,Header,Text,Button Text,Button URL,Context Button Text,Context Button URL
eggs,eggs,are tasty,VISIT PAGE,eggs.com,eggs,eggs.com

I want to read the first line (starting with "Query....") but it's being actively ignored.

You seem to be opening the file with `output = open(path, 'a')` and later `output = open(path, 'w')` without closing the file in between; this seems like a breathing ground for unintended behavior. Are you sure that this is a good way to go? — Ralf, Jul 23 '19 at 11:05
I used those tags because in the former I have to provide an instance of `open()` to initialize `csv.DictWriter()` (the `'a'` flag is to prevent overwriting and permit appending to the file if all of the if-else checks pass), and in the latter the `'w'` tag is so that I can erase the file and overwrite new headers. I am not sure it is a great idea, though...... — Mayukh Nair, Jul 23 '19 at 11:07
The problem Ralf pointed to is not the flags, but the fact you open the file twice without closing it in-between. You should probably tweak the logic to use the [`with open(my_file, flag) as foo`](https://stackoverflow.com/questions/1369526/what-is-the-python-keyword-with-used-for) context manager. — Leporello, Jul 23 '19 at 12:09

score 1 · Answer 1 · answered Jul 23 '19 at 11:11

1

readline() output includes newline \n character at the end. So output.readline() returns

Query,Header,Text,Button Text,Button URL,Context Button Text,Context Button URL\n

Maybe that is why this condition if(output.readline() != ",".join(csv_columns)): returns True.

answered Jul 23 '19 at 11:11

suitsense

38
3

Uh, no? Just tested and it hasn't been returning `True` for me. – Mayukh Nair Jul 23 '19 at 11:14
Oh, sorry... Can you post the output logs or stack trace? – suitsense Jul 23 '19 at 11:29
There's no logging or stack trace as such. The program runs without countering an exception. It's constantly ignoring the first line of the CSV though, which is weird. – Mayukh Nair Jul 23 '19 at 11:37

score 0 · Accepted Answer · answered Jul 23 '19 at 12:15

Ended up solving this myself. So I rewrote the function to not rely on the opened files, and apart from some extra logic bugs in the code above, the script below works perfectly as intended.

@Suitsense's answer made me aware of the \n that readline() adds at the end, so thank you!

    path = 'intent_outputs.csv'
    response = {'query':'eggs', 'header':'eggs', 'fulfillmentText':'are tasty', 'buttontext':'VISIT PAGE', 'buttonurl':'eggs.com', 'contextbuttontext':'eggs', 'contextbuttonurl':'eggs.com'}
    csv_columns = ['Query', 'Header', 'Text', 'Button Text', 'Button URL', 'Context Button Text', 'Context Button URL']
    output = ""
    writer = ""
    try:
        output = open(path,'x')
    except FileExistsError:
        output = open(path,'r')
    if(open(path,'r').readline() != ",".join(csv_columns)+"\n"):
        try:
            output = open(path, 'w')
            writer = csv.DictWriter(output, fieldnames=csv_columns)
            writer.writeheader()
        except IOError:
            print("IOError: Couldn't write header")
    else:
        output = open(path, 'a')
        writer = csv.DictWriter(output, fieldnames=csv_columns)
    row = {'Query':response['query'], 'Header':response['header'], 'Text':response['fulfillmentText'], 'Button Text':response['buttontext'], 'Button URL':response['buttonurl'], 'Context Button Text':response['contextbuttontext'], 'Context Button URL':response['contextbuttonurl']}
    writer.writerow(row)

Python readline ignoring header in csv file

2 Answers2