0

I have 30 different csv files and each row begin with date and some similar features measured for each of 30 items daily. The value of each feature is not important, but the rank they gain after sorting in each day is important. How can I have one merged csv from 30 separate csv with the rank of each feature?

Rayhan
  • 1
  • the question is vague, you should provide a clear explanation of what you want with input/output examples and the code you used – mozway Jan 31 '22 at 08:43
  • After running that code it gave me a concatenated Dataframe with repeated 'date'. Now all features of 30 items concatenated without sorted column:'F'. In addition after sorting, I'm asking the rank replace to value of column : 'F'. 'date','item','F' date1, a,1.36 date2,a, 1.14 date'n',a, 2.18 date1, b, 1.28 date2, b, 3.19 date'n', b, 2.11 date1, c, 0.9 date2, c, 4.44 date'n', c, 1.02 – Rayhan Feb 01 '22 at 08:33
  • Please provide enough code so others can better understand or reproduce the problem. – Community Feb 09 '22 at 23:10

1 Answers1

0

If your files are all the same format, you can do a loop and consolidate in a single data frame. From there you can manipulate as needed:

import pandas as pd
import glob

path = r'C:\path_to_files\files' # use your path
all_files = glob.glob(path + "/*.csv")

li = []

for filename in all_files:
    df = pd.read_csv(filename, index_col=None, header=0)
    li.append(df)

frame = pd.concat(li, axis=0, ignore_index=True)

Other examples of the same thing: Import multiple csv files into pandas and concatenate into one DataFrame.

Max888
  • 132
  • 6
  • After running your code it gave me a dataframe like bellow. – Rayhan Feb 01 '22 at 08:18
  • Ok, just fyi I can not see it. If you're thinking where to sort, you can consider sorting by 2 columns (eg, the date and daily rank). – Max888 Feb 01 '22 at 08:29
  • After running that code it gave me a concatenated Dataframe with repeated 'date'. Now all features of 30 items concatenated without sorted column:'F'. In addition after sorting, I'm asking the rank replace to value of column : 'F'. 'date','item','F' date1, a,1.36 date2,a, 1.14 date'n',a, 2.18 date1, b, 1.28 date2, b, 3.19 date'n', b, 2.11 date1, c, 0.9 date2, c, 4.44 date'n', c, 1.02 – Rayhan Feb 01 '22 at 08:36
  • ok, it's hard to visualise. Could you put the data into a Dataframe so I can run it? See https://stackoverflow.com/questions/20109391/how-to-make-good-reproducible-pandas-examples for some examples on how to do this. – Max888 Feb 01 '22 at 11:05
  • I tried this, but doesn’t work! frame121 = frame12.sort_values(['date','F']) import collections for key, value in enumerate(frame121['date'].values): reapetedItemList = [item for item, count in collections.Counter(value).items() if count > 1] for date in reapetedItemList: while frame121['date'].values == date: for i in range(30): if frame121['F'][i+1] > frame121['F'][i]: frame121['F'][i] = i else: frame121['F'][i+1] = i i = i+1 – Rayhan Feb 01 '22 at 11:12