Consistent order of headers when outputting a DataFrame to CSV

Question

I'm looping through a JSON response and appending each element to a Pandas DataFrame in Python.

At the end of the loop, I output the DataFrame to a CSV. However, each time I output the CSV, the headers aren't consistent, as in the order of the columns seem to change.

How can I make the order of the headers (and columns) consistent each time? See the code that I'm working with below:

    output = pd.DataFrame()
    for item in json_resp:
        json_struct = {
            'col_1':json_resp['data'],
            'col_2':json_resp['data_2'],
            'col_2':json_resp['data_2']
        }
        output = output.append(json_struct, ignore_index=True)
    output.to_csv('csv_output.csv', index=False, encoding='utf-8-sig')

score 0 · Accepted Answer · answered Oct 01 '19 at 01:09

0

You can specify the columns to save on to_csv with a list. e.g.

output.to_csv('csv_output.csv', columns=['col_1', 'col_2', 'col_3'], headers=True, index ...

Also check out Pandas Writing Dataframe Columns to csv

answered Oct 01 '19 at 01:09

Sebastian Oliva

335
3
7

score 0 · Answer 2 · answered Oct 01 '19 at 01:12

The function to_csv saves the columns in the order you provided. You can either sort the columns upon inserting or before saving:

# Upon inserting - slow
output = output.append(json_struct, ignore_index=True, sort=True)

# Before saving, with known column names
output = output[['col_1', 'col_2', 'col_3']]

# Before saving, with unknown column names
output = output.sort_index(axis=1)

Consistent order of headers when outputting a DataFrame to CSV

2 Answers2