reading file names from a list and then appending them does not append files

Question

I have a list contains names of the files.

I want to append content of all the files into the first file, and then copy that file(first file which is appended) to new path.

This is what I have done till now: This is part of code for appending (I have put a reproducable program in the end of my question please have a look on that:).

if (len(appended) == 1):
    shutil.copy(os.path.join(path, appended[0]), out_path_tempappendedfiles)
else:

    with open(appended[0],'a+') as myappendedfile:
        for file in appended:
                myappendedfile.write(file)
    shutil.copy(os.path.join(path, myappendedfile.name), out_path_tempappendedfiles)

this one will run successfully and copy successfully but it does not append files it just keep the content of the first file.

I have also tried this link it did not raises error but did not append files. so the same code except instead of using write I used shutil.copyobject

with open(file,'rb') as fd:
shutil.copyfileobj(fd, myappendedfile)

the same thing happend.

Update1 This is the whole code:

Even with the update it still does not append:

import os

import pandas as pd
d = {'Clinic Number':[1,1,1,2,2,3],'date':['2015-05-05','2015-05-05','2015-05-05','2015-05-05','2016-05-05','2017-05-05'],'file':['1a.txt','1b.txt','1c.txt','2.txt','4.txt','5.txt']}
df = pd.DataFrame(data=d)
df.sort_values(['Clinic Number', 'date'], inplace=True)
df['row_number'] = (df.date.ne(df.date.shift()) | df['Clinic Number'].ne(df['Clinic Number'].shift())).cumsum()

import shutil
path= 'C:/Users/sari/Documents/fldr'
out_path_tempappendedfiles='C:/Users/sari/Documents/fldr/temp'

for rownumber in df['row_number'].unique():
    appended = df[df['row_number']==rownumber]['file'].tolist()
    if (len(appended) == 1):
        shutil.copy(os.path.join(path, appended[0]), out_path_tempappendedfiles)
    else:
        with open(appended[0],'a') as myappendedfile:
            for file in appended:
                fd=open(file,'r')
                myappendedfile.write('\n'+fd.read())
                fd.close()

        Shutil.copy(os.path.join(path, myappendedfile.name), out_path_tempappendedfiles)

Would you please let me know what is the problem?

score 0 · Answer 1 · answered Jul 23 '18 at 02:07

0

you can do it like this, and if the size of files are to large to load, you can use readlines as instructed in Python append multiple files in given order to one big file

import os,shutil
file_list=['a.txt', 'a1.txt', 'a2.txt', 'a3.txt']
new_path=

with open(file_list[0], "a") as content_0:
    for file_i in file_list[1:]:
        f_i=open(file_i,'r')
        content_0.write('\n'+f_i.read())
        f_i.close()
shutil.copy(file_list[0],new_path)

answered Jul 23 '18 at 02:07

smartass

26
4

thanks for the response. Thats very weird but it did not appended yet. I will update with my whole code, its the same except I have other columns so I queried part of that based on the conditions I have – sariii Jul 23 '18 at 13:21
your code not appending the content of the files rather the names of the files – sariii Jul 23 '18 at 15:11
That's interesting, I makeit right in my simple tasks. Whatever, I find you have worked it out in the latter answer～ – smartass Jul 25 '18 at 01:41

score 0 · Accepted Answer · answered Jul 23 '18 at 17:17

so this how I resolve it. that was very silly mistake:| not joining the basic path to it. I changed it to use shutil.copyobj for the performance purpose, but the problem only resolved with this:

os.path.join(path,file)

before adding this I was actually reading from the file name in the list and not joining the basic path to read from actual file:|

for rownumber in df['row_number'].unique():
    appended = df[df['row_number']==rownumber]['file'].tolist()
    print(appended)
    if (len(appended) == 1):
        shutil.copy(os.path.join(path, appended[0]), new_path)
    else:
        with open(appended[0], "w+") as myappendedfile:
            for file in appended:
                with open(os.path.join(path,file),'r+') as fd:
                    shutil.copyfileobj(fd, myappendedfile, 1024*1024*10)
                    myappendedfile.write('\n')
        shutil.copy(appended[0],new_path)

reading file names from a list and then appending them does not append files

2 Answers2