1

based on this I am loading multiple files of a folder into one dataframe:

import pandas as pd

files = glob.glob("TransactionData\Promorelevant*.csv")
dfs = [pd.read_csv(f, header=None, sep=";") for f in files]

salesdata = pd.concat(dfs,ignore_index=True)

My original windows folder looks like this:

A-2017-05-08
B-2017-05-09
...

So it is sorted based on a datestemp (which is in the filename). Is this order also kept while reading the file? I need this order to apply a filter later on, so that I know when for an entry it appears upper in the dataframe it has to filter out, if it also appears later.

PV8
  • 5,799
  • 7
  • 43
  • 87
  • 1
    Do you mean if the order is preserved in `concat` ? Because the order in `files` is definitely preserved when you call `pd.read_csv` in the list comprehension – kosnik Nov 18 '19 at 11:58
  • Yes, then I mean this – PV8 Nov 18 '19 at 12:10

1 Answers1

2

According to this reply

Will passing ignore_index=True to pd.concat preserve index succession within dataframes that I'm concatenating?

you can preserve the order of the files list by specifying additionally sort=False. Your last line of code would then look like

salesdata = pd.concat(dfs, ignore_index=True, sort=False)
kosnik
  • 2,342
  • 10
  • 23