How to run a script for several files

Question

I wrote a simple script that adds a column with a single value to an existing file, I want to run it for several other files located in one folder, any idea how I could do this?

The python script looks something like this:

!/usr/bin/env python

from qi.h5 import *
import numpy as np
import pandas as pd
import sys
import project


path= "/qdata/projects4/solidground/bsh_o13/import/horizon/raw/Unit_X.Q_085"
#output_dir = "/qdata/projects4/solidground/bsh_o13/import/horizon/raw/test"



df = pd.read_csv(files, delim_whitespace = True, header = None)
df[len(df.columns)] = 1
df_no_indices = df.to_string(index=False)


print(df_no_indices)

Possibly use a for loop with all the files paths (in a list...?) you want to run it to...? Please elaborate. Are you given the file paths or do you want to loop through all files in a specific directory? — Codeman, Mar 14 '22 at 12:39
put code in function which gets path - and later execute it many times with different paths. And later you can use `os.listdir(directory)` to get list with all files in directory and use for-loop to run function for every item on list. It may need also `os.path.join(directory, filename)` to create full path. — furas, Mar 14 '22 at 14:38

furas · Answer 1 · 2022-03-14T14:53:09.137

If you put code in function which gets path then you can simply run it with different paths

import pandas as pd

# --- functions ---

def my_function(path):
    df = pd.read_csv(files, delim_whitespace=True, header=None)  # PEP8: arguments witout spaces around `=`
    df[len(df.columns)] = 1
    df_no_indices = df.to_string(index=False)
    print(df_no_indices)
    
# --- main ---

first_path = "/qdata/projects4/solidground/bsh_o13/import/horizon/raw/Unit_X.Q_085"
my_function(first_path)

my_function('/other/path/to/file')
my_function('/different/path/to/file')

And if you put paths on list then you can use for-loop

import pandas as pd

# --- functions ---

def my_function(path):
    df = pd.read_csv(files, delim_whitespace=True, header=None)  # PEP8: arguments witout spaces around `=`
    df[len(df.columns)] = 1
    df_no_indices = df.to_string(index=False)
    print(df_no_indices)
    
# --- main ---

my_files = [
    "/qdata/projects4/solidground/bsh_o13/import/horizon/raw/Unit_X.Q_085",
    '/other/path/to/file',
    '/different/path/to/file',
]

for path in my_files:
    my_function(path)

And later you can use os.listdir(), os.walk() or grep.grep() to create list with files - and use this list with for-loop.

Some functions may give filename without directory and they need

path = os.path.join(directory, filename)

to create full path.

How to run a script for several files

1 Answers1