1

So I have these loops that take data from files in a directory and appends that data to a list with the filename of the file from where the data came attached to the beginning of each string of data. I have another directory with files in it with the same file names except for ".txt" at the end that I need to append this data to. I figured it be easiest to match file names to ensure the correct data gets appended to the correct files. I'm just having trouble figuring out how to do this. Bit of a newbie here.

dst2 = "directory/where/files/with/data/are/stored/"   # Where all my files are stored 

mypath = dst2
files = [f for f in os.listdir(mypath) if isfile(join(mypath, f))]    # LIST of filenames in the directory of your config files

data = os.listdir(mypath)

names = []   #All the file names will be stored here
for i in files:
    x = (str(i).split())
    for split in x:
        names.append(split)  

for i, file in enumerate(data):
    a = np.genfromtxt(dst2+file)
    a2 = a[:,0]                   # Pulls out first column = wavelength        
    b2 = (a[:,1])                 # Pulls out second column = use this to make simulated spectrum without noise
    mu, sigma = 0, 0.01            # mean and standard diviation 
    noise = np.random.normal(mu, sigma ,[1265,])     # creates noise
    B2 = b2 + noise               # simulated spectrum without noise
    sigma2 = np.repeat(sigma, 1265) # sigma = constant noise at 1%  
    DataToAppend = [] 
    DataToAppend.append({"{:.2f}".format(names[i])+'_Wavelength':a2,'Freq':B2,'Noise':sigma2}) # append the column with they're associated file name
    print(DataToAppend)

OutPut:

[{'ColdWaterIceCH4Mabs_-6.0P0_100.0.cfg.txt_Wavelength': array([0.2       , 0.2004    , 0.2008008 , ..., 2.48939933, 2.49437813,
       2.49936688]), 'Freq': array([0.71150625, 0.71394532, 0.70672077, ..., 0.33867644, 0.34125075,
       0.36570275]), 'Noise': array([0.01, 0.01, 0.01, ..., 0.01, 0.01, 0.01])}]
[{'ColdBasaltCO2Mabs_-4.0P0_1.0.cfg.txt_Wavelength': array([0.2       , 0.2004    , 0.2008008 , ..., 2.48939933, 2.49437813,
       2.49936688]), 'Freq': array([0.58510933, 0.57541622, 0.58468481, ..., 0.19431411, 0.19020661,
       0.17070037]), 'Noise': array([0.01, 0.01, 0.01, ..., 0.01, 0.01, 0.01])}] etc..

Now I need to append this data to the end of files in a different directory as 3 seperate columns plus add some text by matching up the filenames in this directory to the ones in the first directory. So far I have this but it is very much wrong.

dir = '/Users/ccruzarc/Desktop/PSG_GPC_framework_master/Cfgs/ColdRetrievalsConfig/1'         # Replace with path to your directory: absolute or relative
pattern = '' # Replace with your target substring
matching_files = [f for f in os.listdir(dir) if pattern in f]

NewConfigs = []
if matching_files = names[i]:
    os.system("find . -type f ! -path "/Users/ccruzarc/Desktop/PSG_GPC_framework_master/Cfgs/ColdRetrievalsConfig/1" -exec sh -c "<DATA> /n >> {}.format(DataToAppend[i]) /n </DATA> {}" \;"
    NewConfigs.append()
    print(NewConfigs)
  • Can you provide an example of what the desired result needs to be? Also, using `os.system` seems a bit odd. You can use python itself to [append to a file](https://stackoverflow.com/a/4706520/9267296) – Edo Akse May 24 '21 at 03:16
  • Also, [f-strings](https://realpython.com/python-f-strings/) are awesome... – Edo Akse May 24 '21 at 03:22

0 Answers0