1

I'm working on a bash script that needs to (recursively) move all files and folders within a source folder into a destination folder.

In trying to make this as robust as possible, and address potential argument list too long - errors, I opted to use the find command (to safely determine the files to move) piped into xargs (to efficiently group the moves together). I'm also using -print0 and -0 to address potential problems with spaces.

I've written the following test script:

#!/bin/bash

# create test source file structure, and destination folder
mkdir -p sourceDir/{subdir1,subdir\ with\ space\ 2}
mkdir -p destDir
touch sourceDir/subdir1/{a1.txt,noExtension1,file1\ with\ space.txt}
touch sourceDir/subdir\ with\ space\ 2/{a2.txt,noExtension2,file2\ with\ space.txt}

#move files/folders from source to destination
find sourceDir -mindepth 1 -print0 | xargs -0 mv --target-directory=destDir

Which seems to work (i.e. the files are moved) but for some reason I get numerous errors as follows: 

mv: cannot stat `sourceDir/subdir with space 2/file2 with space.txt': No such file or directory
mv: cannot stat `sourceDir/subdir with space 2/noExtension2': No such file or directory
mv: cannot stat `sourceDir/subdir with space 2/a2.txt': No such file or directory
mv: cannot stat `sourceDir/subdir1/file1 with space.txt': No such file or directory
mv: cannot stat `sourceDir/subdir1/noExtension1': No such file or directory
mv: cannot stat `sourceDir/subdir1/a1.txt': No such file or directory

Should there not be a way to do this (for the source files indicated in my script) without generating errors?

Why are these errors being generated (since the files and folders are in fact being moved)?

Community
  • 1
  • 1
Jonny
  • 3,807
  • 8
  • 31
  • 48

3 Answers3

1

I've sussed it.

Short answer: I was missing the -maxdepth 1 tag. The following command works

find sourceDir -mindepth 1 -maxdepth 1 -print0 | xargs -0 mv --target-directory=destDir
# as does
find sourceDir -mindepth 1 -maxdepth 1 -exec mv --target-directory=destDir '{}' +

Longer answer:

My original find command was listing the paths to each and every file, and then trying to move each and every one of them. Once a top level folder had been moved, there was then no need to move any files/folders underneath - but the script was still trying to!

The script below demonstrates this (without actually performing the moves):

#!/bin/bash

mkdir -p sourceDir/{subdir1/subsubdir1,subdir\ with\ space\ 2}
touch sourceDir/subdir1/{subsubdir1/deeperFile.log,a1.txt,noExtension1,file1\ with\ space.txt}
touch sourceDir/subdir\ with\ space\ 2/{a2.txt,noExtension2,file2\ with\ space.txt}
touch sourceDir/rootFile.txt

echo -e "\n--- Output of 'ls -1: sourceDir':"
echo "$(ls -1 sourceDir)"

echo -e "\n--- original incorrect attempt was trying to move each of the following:"
find sourceDir -mindepth 1

echo -e "\n--- working attempt only needs to move each of the top level files/folders, everything underneath any folders will get moved automatically"
find sourceDir -mindepth 1 -maxdepth 1

giving output:

--- Output of 'ls -1: sourceDir':
rootFile.txt
subdir1
subdir with space 2

--- original incorrect attempt was trying to move each of the following:
sourceDir/rootFile.txt
sourceDir/subdir with space 2
sourceDir/subdir with space 2/file2 with space.txt
sourceDir/subdir with space 2/noExtension2
sourceDir/subdir with space 2/a2.txt
sourceDir/subdir1
sourceDir/subdir1/file1 with space.txt
sourceDir/subdir1/noExtension1
sourceDir/subdir1/a1.txt
sourceDir/subdir1/subsubdir1
sourceDir/subdir1/subsubdir1/deeperFile.log

--- working attempt only needs to move each of the top level files/folders, everything     underneath any folders will get moved automatically
sourceDir/rootFile.txt
sourceDir/subdir with space 2
sourceDir/subdir1
Jonny
  • 3,807
  • 8
  • 31
  • 48
  • Or you can add `-type d` to find top dirs. Check my answer below. – Sanket Parmar Jun 18 '14 at 13:46
  • With just `-type d` this will ignore any top level files. Also without the -maxdepth 1 stated in my own answer, your solution will still list any sub-sub-folders - and again fail trying to move them. See my updated answer for full details. – Jonny Jun 18 '14 at 14:01
  • In your question, you have mentioned only directories. So I added `-type d` – Sanket Parmar Jun 18 '14 at 14:05
  • The first line said: "(recursively) move all files and folders", though I perhaps didn't make it clear in the sample script. – Jonny Jun 18 '14 at 14:08
0

Why don't do that directly with the find command ?

$ find [...] -exec mv '{}' [...] ';'

Where {} means each file found and ; the end of the arguments passed to mv

alcala
  • 1,153
  • 2
  • 10
  • 14
  • In answer to why not do it directly with `find` - If I just use find with mv, it will execute the mv command for each individual file. If there are thousands, or hundreds of thousands of files to move this will soon become slow/inefficient. Passing through xargs (or using `+` with `-exec` are ways to improve on this). See this answer for more detail: http://stackoverflow.com/a/11942775/1448678 – Jonny Jun 18 '14 at 13:09
0

By default the output of find command starts with parent directory, then directory contain.

Simple find command on your directroy structure gives following output:

sourceDir/subdir1
sourceDir/subdir1/a1.txt
sourceDir/subdir1/noExtension1
sourceDir/subdir1/file1 with space.txt
sourceDir/subdir with space 2
sourceDir/subdir with space 2/noExtension2
sourceDir/subdir with space 2/file2 with space.txt

If you use this output in xargs or -exec , It will first process the parent directory and after that its contents. In this case your command executes fine and move all whole sourceDir/subdir1 directory to destDir/subdire1. After that there is no files/directory like sourceDir/subdir1/a1.txt or sourceDir/subdir1/noExtension1 in sourceDir because it is already moved to destDir.

To avoid this use -depth option. 

-depth Process each directory's contents before the directory itself.

In your case you can add -type d option to move top level source dir to destDir.

find sourceDir   -mindepth 1 -type d  -print0 | xargs -0 mv --target-directory=destDir

Or 

find sourceDir -mindepth 1 -type d  -exec mv -t destDir "{}"  \+

It will solve your problem.

Sanket Parmar
  • 1,477
  • 12
  • 9
  • Thanks for the input, but this still doesn't work. Without the `-maxdepth 1` stated in my own answer, your solution will list any sub-sub-folders - and again fail trying to move them. See my updated answer for full details. – Jonny Jun 18 '14 at 13:59
  • Agreed. `-maxdepth1` will make it more generic. – Sanket Parmar Jun 18 '14 at 14:02