Filter specified file extensions to other file

Question

I have many files in different paths with various file extensions.

My input.txt content as follows,

/path/to/dir1/readme.html
/path/to/dir1/file.c
/path/to/dir1/file1.c
/path/to/dir1/a.html
/path/to/dir2/abc.java
/path/to/dir1/sample.js
/path/to/dir2/a.bin
/path/to/dir1/as.json
.......................
...........................
..............................

I need to filter and move the specified extension files from all occurrences of input.txt file to output.txt file.

For this, i have below script.

import shutil

input_file = 'input.txt'
output_file = 'output.txt'

file_extensions = ['.html', '.c', '.cpp', '.h', '.py', '.txt', '.js', '.json', '.csv']

with open(input_file, 'r') as input_file, open(output_file, 'w') as output_file:
    for line in input_file:
        file_path = line.strip()

        if any(file_path.endswith(ext) for ext in file_extensions):
            output_file.write(file_path + '\n')
            shutil.move(file_path, file_path + '.processed')

    print('Matching file moved to output.txt.')

Expected output.txt should be like below.

/path/to/dir1/readme.html
/path/to/dir1/file.c
/path/to/dir1/file1.c
/path/to/dir1/sample.js
/path/to/dir1/as.json

Above script doesn't work, it fails with below errors

Traceback (most recent call last):
  File "/usr/lib/python3.8/shutil.py", line 791, in move
    os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: '/path/to/dir1/readme.html' -> '/path/to/dir1/readme.html.processed'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "split_src_libs.py", line 24, in <module>
    shutil.move(file_path, file_path + '.processed')
  File "/usr/lib/python3.8/shutil.py", line 811, in move
    copy_function(src, real_dst)
  File "/usr/lib/python3.8/shutil.py", line 435, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.8/shutil.py", line 264, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: '/path/to/dir1/readme.html'

What is causing this issue?

Any help would be appreciated to filter & move the files to output.txt file.

Note: Moved files shouldn't be exist in input.txt file.

Answer 1

shutil.move moves/renames the file on your filesystem.

That fails, probably because the file does not exist, or the python script does not have permission to move it.

---

If you just want to append the name to output.txt, remove the shutil line.

input_file = 'input.txt'
output_file = 'output.txt'

file_extensions = ['.html', '.c', '.cpp', '.h', '.py', '.txt', '.js', '.json', '.csv']

with open(input_file, 'r') as input_file, open(output_file, 'w') as output_file:
    for line in input_file:
        file_path = line.strip()

        if any(file_path.endswith(ext) for ext in file_extensions):
            output_file.write(file_path + '\n')

    print('Matching file moved to output.txt.')

Now if I run the script, the output.txt contains:

/path/to/dir1/readme.html
/path/to/dir1/file.c
/path/to/dir1/file1.c
/path/to/dir1/a.html
/path/to/dir1/sample.js
/path/to/dir1/as.json

---

That said, consider using the following, using os.path.splitext to get the file extension, then you can just do an if ext in file_extensions, so you don't need the any and loop:

import os

input_file = 'input.txt'
output_file = 'output.txt'

file_extensions = ['.html', '.c', '.cpp', '.h', '.py', '.txt', '.js', '.json', '.csv']

with open(input_file, 'r') as input_file, open(output_file, 'w') as output_file:
    for line in input_file:
        filename, file_extension = os.path.splitext(line.strip())

        if file_extension in file_extensions:
            output_file.write(line)

    print('Matching file moved to output.txt.')

• Extracting extension from filename in Python

Answer 2

As was said earlier, the error comes from the file you tried to move not existing.

But since you don't need to actually move the files, only to filter your input file, no need for shutil.move. Here's a working solution, updated to rewrite input.txt with only the unmoved lines:

input_file = 'input.txt'
output_file = 'output.txt'
file_extensions = ['.html', '.c', '.cpp', '.h', '.py', '.txt', '.js', '.json', '.csv']

unmoved_lines = []

with open(input_file, 'r') as _input_file, open(output_file, 'w') as output_file:
    for line in _input_file:
        if f'.{line.strip().split(".")[-1]}' in file_extensions:
            output_file.write(line)
        else:
            unmoved_lines.append(line)
            
with open(input_file, 'w') as _input_file:
    _input_file.write(''.join(unmoved_lines))

output.txt:

/path/to/dir1/readme.html
/path/to/dir1/file.c
/path/to/dir1/file1.c
/path/to/dir1/a.html
/path/to/dir1/sample.js
/path/to/dir1/as.json

input.txt:

/path/to/dir2/abc.java
/path/to/dir2/a.bin