os.walk one level down

Question

I have a folder (A) that is structured like this:

A\b
A\c
A\d

I would like to copy folder b, c, and d one level down 3 times like this:

A\b\b1\b2\b3
A\c\c1\c2\c3
A\d\d1\d2\d3

Where b1,b2, and b3 are copies of the folder b, the same goes for c and d.

When I run the code below I end up with the desired structure but also adds three copies in each sub-folder so A\b\b1 has b1,b2, and b3 folders. The problem has to do with the loop but I can't pinpoint it

import os
import sys
import shutil

list1= [1,2,3]
path = "C:\ladg\oner"
root, dirs, files = os.walk(path).next()

for dir in dirs:
    for i in list1:
        new_folder_path = os.path.join(root,dir, dir+ str(i))
        shutil.copytree(os.path.join(root, dir), new_folder_path)

Copy the original directory, say `b`, to a directory `tmp_b` (with a unique name). Copy it to `b1` and `b1\b2`, then move the temporary copy to `b1\b2\b3`. — Alex Reinking, Jul 10 '18 at 04:48

iridescent · Accepted Answer · 2018-07-10T15:59:56.547

2

Try this. You're currently iterating through the list without changing directories.

import os
import sys
import shutil

list1= [1,2,3]
path = "C:\ladg\oner"
root, dirs, files = os.walk(path).next()

for dir in dirs:
    curr_path = os.path.join(root, dir)
    for i in list1:
        new_folder_path = os.path.join(curr_path, dir + str(i))
        shutil.copytree(curr_path, new_folder_path)
        os.chdir(new_folder_path)
        curr_path = new_folder_path

In a follow-up question, if you want to achieve a scenario where you have this resulting directory:

A
-- b
   -- b1
   -- b2 
   -- b3
-- c
   -- c1
   -- c2
   -- c3

You can use the following code. This is different from the first suggestion as now you're creating a tmp_dir to store your copies before finally replacing the old curr_dir with this new tmp_dir. This is necessary to maintain an intact copy of curr_dir when you do subsequent copies.

import os
import sys
import shutil

list1= [1,2,3]
path = "~/A"
root, dirs, files = os.walk(path).next()

for dir in dirs:
    curr_dir = os.path.join(root, dir)
    tmp_dir = os.path.join(root, "tmp_" + dir)
    os.mkdir(tmp_dir)
    for i in list1:
        new_folder_path = os.path.join(tmp_dir, dir+ str(i))
        shutil.copytree(curr_dir, new_folder_path)
    shutil.rmtree(curr_dir)
    shutil.move(tmp_dir, curr_dir)

edited Jul 10 '18 at 15:59

answered Jul 10 '18 at 05:00

iridescent

383
1
13

When I run this in Python 2, it crashes on the `copytree` line, saying `OSError: [Errno 2] No such file or directory: 'A/b/b1'` – Alex Reinking Jul 10 '18 at 05:08
Can u try changing the path to include directory A? e.g. in linux it's `/ladg/oner/A` – iridescent Jul 10 '18 at 05:22
I did before commenting. – Alex Reinking Jul 10 '18 at 05:32
Perhaps it has to do with how `os.path.join` does not work in the same manner as linux for windows. You can check out this article https://stackoverflow.com/questions/2422798/python-os-path-join-on-windows and also check where the new folders are created when you run `shutil.copytree` – iridescent Jul 10 '18 at 06:25
What if I want to create three copies of each subfolder inside the subfolder so b1, b2, and b3 are all under b. The same thing for the c and d directory. Also, I'd like to vote the answer up but I have 0 reputations. Maybe someone can give me some rep – lamaee201 Jul 10 '18 at 15:02
I'm sorry but I don't fully understand your question. So you'd like to create 3 copies of folder `b` under itself? – iridescent Jul 10 '18 at 15:07
Yes, exactly. Three copies of each folder under itself – lamaee201 Jul 10 '18 at 15:15
If I understand you correctly, this is the end result that you want - folder `A` has 3 folders `b, c, d` and each of these `b, c, d` has 3 copies of itself inside itself (i.e. `b` has `b1, b2, b3`). If this is the case, the original code you posted will achieve this. – iridescent Jul 10 '18 at 15:19
Actually, the original code will create the three copies inside the folder itself then repeats the process again to create more copies inside the newly created three copies. – lamaee201 Jul 10 '18 at 15:33
My bad. I was wrong. Upon closer inspection, I see what you meant by it creating more copies under the newly created copies. This is caused by `shutil.copytree` copying the entire directory into the newly created copy. – iridescent Jul 10 '18 at 15:42
1

@lamaee201 I just revised my answer with a second solution that can do what you're looking for – iridescent Jul 10 '18 at 16:00
@iridescent awesome, that did what I needed. Thanks! – lamaee201 Jul 10 '18 at 19:27

score 0 · Answer 2 · answered Jul 10 '18 at 05:03

The basic strategy is to copy the original directory, say b, to a directory tmp_b (with a unique name). Copy it to b1 and b1\b2, then move the temporary copy to b1\b2\b3

Here is some code that implements this:

#!/usr/bin/env python3
import os, sys, shutil
import tempfile

def make_nested_copies(path, N):
    root, dirs, _ = next(os.walk(path))
    for current_dir_name in dirs:
        temp = tempfile.mkdtemp()
        temp_path = os.path.join(temp, current_dir_name)

        destination_path = os.path.join(root, current_dir_name)
        shutil.copytree(destination_path, temp_path)

        for i in range(1,N):
            destination_path = os.path.join(destination_path, current_dir_name + str(i))
            shutil.copytree(temp_path, destination_path)

        destination_path = os.path.join(destination_path, current_dir_name + str(N))
        shutil.move(temp_path, destination_path)

if __name__ == '__main__':
    make_nested_copies(sys.argv[1], 3)

Testing with the following directory structure:

A
|-- A.txt
|-- b
|   `-- b.txt
|-- c
|   `-- c.txt
`-- d
    `-- d.txt

Results in:

A
|-- A.txt
|-- b
|   |-- b.txt
|   `-- b1
|       |-- b.txt
|       `-- b2
|           |-- b.txt
|           `-- b3
|               `-- b.txt
|-- c
|   |-- c.txt
|   `-- c1
|       |-- c.txt
|       `-- c2
|           |-- c.txt
|           `-- c3
|               `-- c.txt
`-- d
    |-- d.txt
    `-- d1
        |-- d.txt
        `-- d2
            |-- d.txt
            `-- d3
                `-- d.txt

os.walk one level down

2 Answers2