Copying files of the same name from multiple directories into one directory

Question

I am trying to copy multiple files of the same name from different directories into one and have them not overwrite each other by adding some number before the name. I have a file structure like this, where the image.fits files are different files, but have the same name because they are automatically generated and the parent folder name is also auto generated:

~/Sources/<unknown>/<foldername1>/image.fits
~/Sources/<unknown>/<foldername2>/image.fits
~/Sources/<unknown>/<foldername3>/image.fits
...

Is there a way to copy these files into one folder like this:

~/Sources/<target_folder>/1_image.fits
~/Sources/<target_folder>/2_image.fits
~/Sources/<target_folder>/3_image.fits

Like mentioned above the folder names are also automatically generated, so I want to use some kind of wildcard (*) to access them if possible. The command can either be some command, a shell script or python code, whatever works.

EDIT: The final solution I used is based on the one from @Kasper and looks like this:

import os
import shutil

if __name__ == '__main__':
    os.system('mkdir ~/Sources/out')
    child_dirs = next(os.walk('~/Sources/'))[1]
    num=1
    for dir in child_dirs:
        child_child_dirs = next(os.walk('~/Sources/{}'.format(dir)))[1]
        for ch_dir in child_child_dirs:
            if exists('~/Sources/{}/{}'.format(dir, ch_dir))==True:
                shutil.move('~/Sources/{}/{}'.format(dir, ch_dir), '~/Sources/out/{}_image.fits'.format(num))
                num+=1
            else:
                continue

Check out: [Create file but if name exists add number](https://stackoverflow.com/questions/13852700/create-file-but-if-name-exists-add-number) — DarrylG, Jul 26 '22 at 12:40

score 2 · Accepted Answer · answered Jul 26 '22 at 12:55

2

This should do the magic:

import os
import shutil

if __name__ == '__main__':
    child_dirs = next(os.walk('.'))[1]
    os.mkdir('out')
    num = 1
    for dir in child_dirs:
        shutil.copy2('{}/image.fits'.format(dir), 'out/{}_image.fits'.format(num))
        num+=1

It does the following:

Gets the current child folders.
Creates a new folder called out.
Loops over the folder child folders and copies the files to the new folders.

Note: the script should be ran on the parent folder.

answered Jul 26 '22 at 12:55

Kasper

588
3
16

Is it possible to modify this so it works from a different folder? Something like: shutil.copy2('{}/{}/image.fits' ... – Eli Jul 26 '22 at 13:57
1

Yeah, you can only change the path of the `out` folder and the source path. Changes should be something like: 1) `child_dirs = next(os.walk(path))[1]` 2) `os.mkdir('{}/out'.format(path))` 3) `shutil.copy2('{}/{}/image.fits'.format(path,dir), '{}/out/{}_image.fits'.format(path,num))`. – Kasper Jul 26 '22 at 14:40

Yogesh Thambidurai · Answer 2 · 2022-07-26T14:25:47.363

1

TARGET_FOLDER_NAME="<target_folder>"
TARGET_FILE_NAME="image.fits"

mkdir -p ~/Sources/$TARGET_FOLDER_NAME/

COUNT=1
find ~/Sources/ -type f -name $TARGET_FILE_NAME | while IFS= read -r -d '' file;
do
    mv $file ~/Sources/$TARGET_FOLDER_NAME/$COUNT_$TARGET_FILE_NAME
    COUNT=$((COUNT+1))
done

edited Jul 26 '22 at 14:25

answered Jul 26 '22 at 12:57

Yogesh Thambidurai

222
4
10

1

This is a good approach, but if you're not going to quote the names (eg `mv "${file}" ...`) there is not much point in using `-print0`. The main reason to use `-print0` is to avoid issues with whitespace and special characters in the file names, but that benefit is lost if the names are not consistently quoted. – William Pursell Jul 26 '22 at 14:05

M. Nejat Aydin · Answer 3 · 2022-07-26T13:45:14.637

0

This two-line solution, which doesn't require any programming, is not exactly what you want, but should be close enough, assuming you have cp from GNU coreutils, which is highly probable since the question is tagged with ubuntu-18.04.

shopt -s globstar
cp --backup=numbered Sources/**/image.fits target_folder/

edited Jul 26 '22 at 13:45

answered Jul 26 '22 at 13:02

M. Nejat Aydin

9,597
1
7
17

score 0 · Answer 4 · answered Jul 26 '22 at 13:19

It is supper easy

first

save your file in an array

mapfile -t files < <(find a b c d e -type f -name \*.txt)

Second

# dry-run
COUNTER=0
for file in ${files[*]}; do ((++COUNTER)); echo  $file dist/${COUNTER}-${file##*/}; done

output:

a/file.txt dist/1-file.txt
b/file.txt dist/2-file.txt
c/file.txt dist/3-file.txt
d/file.txt dist/4-file.txt
e/file.txt dist/5-file.txt

thrid

# safe run
# cp not mv
COUNTER=0
for file in ${files[*]}; do ((++COUNTER)); cp $file dist/${COUNTER}-${file##*/}; done

last

Delete your files after checking your destination directory

Or one line:

COUNTER=0; for file in {a,b,c,d,e}/*; do ((++COUNTER)); echo $file dist/${COUNTER}-${file##*/}; done

Replace echo with cp or mv

HatLess · Answer 5 · 2022-07-26T16:57:35.990

0

#!/usr/bin/env bash

new_dir="<target_folder>"
mkdir -p "$HOME/Sources/$new_dir/"
find "$HOME/Sources/" -type f -name 'image.fits' | while read -r fileName; do
    mv "$fileName" "$HOME/Sources/$new_dir/$((n++))_${fileName##*/}"
done

edited Jul 26 '22 at 16:57

answered Jul 26 '22 at 13:22

HatLess

10,622
5
14
32

Copying files of the same name from multiple directories into one directory

5 Answers5

first

Second

thrid

last