Copy tree structure including single file in leaf

Question

After having run several simulations on a cluster, I would like to copy the results back to my local machine. The results are organised is as follows:

foo1
   |bar1
       |run1
           |file1.txt
           |file2.txt
           |results.txt
       |run2
           |file1.txt
           |file2.txt
           |results.txt
   |bar2
       |run1
           |file1.txt
           |file2.txt
           |results.txt
       |run2
           |file1.txt
           |file2.txt
           |results.txt
foo2
... (etc.)

Each run* folder contains several files, needed to run the simulation, but I am only interested in copying the results, say results.txt back to my local machine while maintaining the rest of the tree structure (the directories basically specifies the parameters for the simulation).

How can I now copy only the results.txt files from remote to local, while maintaining the tree structure of the directories? Such that I on local end up with

foo1
   |bar1
       |run1
           |results.txt
       |run2
           |results.txt
   |bar2
       |run1
           |results.txt
       |run2
           |results.txt
foo2
... (etc.)

Your question is how to copy from remote to local? or how to copy only those files from the remote? — Inian, Jun 07 '18 at 11:33
@Inian, How to copy only the `results.txt` files from remote to local, while maintaining the tree structure of the directories. I have also included a sketch of the wanted outcome in the question — Nicky Mattsson, Jun 07 '18 at 11:34
I think the answer is either [here](https://stackoverflow.com/questions/22393908/scp-a-folder-to-a-remote-system-keeping-the-directory-layout) or [here](https://stackoverflow.com/questions/1650164/bash-copy-named-files-recursively-preserving-folder-structure) — oliv, Jun 07 '18 at 11:43
@oliv, Not completely. The first link want to exclude other folders, I do not. The second link want to copy everything keeping the tree structure, whereas I am only interested in copying a single file. On the other hand I admit that the solutions are very similar — Nicky Mattsson, Jun 07 '18 at 11:58

score 2 · Accepted Answer · answered Jun 07 '18 at 11:46

2

With find and tar:

find -name result.txt | tar cvfz results.tgz -T -

then you need to copy the tar and untar it:

tar xvfz results.tgz

answered Jun 07 '18 at 11:46

perreal

94,503
21
155
181

score 1 · Answer 2 · answered Jun 07 '18 at 11:48

You could do this in two stages:

find . -type f -name results.txt | xargs tar -f results.tar

This finds all of the files (-type f) named results.txt and then pipes the results into a tarball.

The second stage would be to scp back from the cluster to your local machine.

Copy tree structure including single file in leaf

2 Answers2