How to open a folder loop through opening other files within that folder in python

Question

This current question is building on from this question.

I am trying to create a python script that will loop through all the text files in the specified folder. The text files contain directories to files that will be moved to a different specified folder. When looping through a text file, it takes the file from the file directory on each line of that text file.

The end goal is to have all the files which are referenced in the text file to move into one specified folder (\1855).

import shutil
dst = r"C:/Users/Aydan/Desktop/1855"

with open(r'C:\Users\Aydan\Desktop\RTHPython\Years') as my_folder:
    for filename in my_folder:
        text_file_name = filename.strip()
        with open (text_file_name) as my_file:
            for filename in my_file:
                file_name  = filename.strip()
                src = r'C:\Users\Aydan\Desktop' + file_name    
                shutil.move(src, dst)

One text file (1855.txt) contains:

/data01/BL/ER/D11/fmp000005578/BL_ER_D11_fmp000005578_0001_1.txt
/data01/BL/ER/D11/fmp000005578/BL_ER_D11_fmp000005578_0002_1.txt
/data01/BL/ER/D11/fmp000005578/BL_ER_D11_fmp000005578_0003_1.txt

and another text file (1856.txt) contains:

/data01/BL/ER/D11/fmp000005578/BL_ER_D11_fmp000005578_0004_1.txt
/data01/BL/ER/D11/fmp000005578/BL_ER_D11_fmp000005578_0005_1.txt
/data01/BL/ER/D11/fmp000005578/BL_ER_D11_fmp000005578_0006_1.txt

This is the error I get when I run the above script:

Traceback (most recent call last):
  File "<pyshell#11>", line 1, in <module>
    with open(r'C:\Users\Aydan\Desktop\RTHPython\Years') as my_folder:
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\Aydan\\Desktop\\RTHPython\\Years'

This script doesn't seem to be moving the files named here to the C:/Users/Aydan/Desktop/1855 destination, even though in the script I'm trying to follow the same logic of iterating through each item in the text file, but applying that logic to a folder instead of inside text file.

Any help to find a solution would be brilliant! If you need any more info about the files just ask.

Thanks!

Aydan.

Is it really possible to open a folder with `open`? I guess you want all .txt files in that folder, so how about: `my_folder = glob.glob('C:\Users\Aydan\Desktop\RTHPython\Years\*.txt')` — Lucy The Brazen, Jul 20 '17 at 14:06
@ManuelBurgstaller I don't want it to use all .txt files at once, I want it to be able to cycle one by one. I'm not too sure if it was possible or not to open a folder with `open`, that's why I came here to see if there was a different solution :P — Aydan Howell, Jul 20 '17 at 14:08
replace `with open(r'C:\Users\Aydan\Desktop\RTHPython\Years') as my_folder:` with `my_folder = os.listdir(r'C:\...') `, and also shift the tabs — ssm, Jul 20 '17 at 14:11
@ssm, this only works if there are no other files than the .txt files he wants to use. But if those are the only files in that folder, it will totally work. — Lucy The Brazen, Jul 20 '17 at 14:13
That is true. It is probably more pythonic to do `my_folder = [ os.path.join(folder, f) for f in os.listdir(folder) if f.endswith('.txt')]`. Your answer is very good. I just shy away from `glob.glob` and `os.walk` for some reason. I feel that `os.listdir` is a bit like what I would do when I work on the terminal ... — ssm, Jul 20 '17 at 14:17
@ssm, so there is no problem with glob.glob, it just isn't your prefered methode? Just asking because I'm pretty new to python and I don't want to pick up bad habits quite early — Lucy The Brazen, Jul 20 '17 at 14:27
Yes, a lot of people use `glob.glob`. I just prefer to use another function thats all ... — ssm, Jul 20 '17 at 14:28

Lucy The Brazen · Accepted Answer · 2017-07-20T14:20:11.877

1

Since you can't open whole folders with the open method, you can get cycle through every .txt file in that folder like that:

import shutil
import glob
dst = r"C:/Users/Aydan/Desktop/1855"

for filename in glob.glob(r"C:\Users\Aydan\Desktop\RTHPython\Years\*.txt"):
    text_file_name = filename.strip()
    with open (text_file_name) as my_file:
        for filename in my_file:
            file_name  = filename.strip()
            src = r'C:\Users\Aydan\Desktop' + file_name    
            shutil.move(src, dst)

edited Jul 20 '17 at 14:20

answered Jul 20 '17 at 14:11

Lucy The Brazen

135
1
9

When debugging, it has an error with these lines: `for filename in glob.glob('C:\Users\Aydan\Desktop\RTHPython\Years\*.txt'): text_file_name = filename.strip()`. Error: `SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape` – Aydan Howell Jul 20 '17 at 14:19
I see what you have done there, adding the r. May I ask, what does the r actually do when in front of a file directory? Anyways, thank you veryy much for solving the problem! :) – Aydan Howell Jul 20 '17 at 14:23
1

I guess it has something to do with the enconding, as stated in the error message, I sadly can't reproduce the error, since in Python 2.7 (which is what I'm using) it works without the "r". – Lucy The Brazen Jul 20 '17 at 14:25
1

Ah, that's understandable. Thanks very much again for the help, greatly appreciated! :D – Aydan Howell Jul 20 '17 at 14:26

How to open a folder loop through opening other files within that folder in python

1 Answers1