So I have these loops that take data from files in a directory and appends that data to a list with the filename of the file from where the data came attached to the beginning of each string of data. I have another directory with files in it with the same file names except for ".txt" at the end that I need to append this data to. I figured it be easiest to match file names to ensure the correct data gets appended to the correct files. I'm just having trouble figuring out how to do this. Bit of a newbie here.
dst2 = "directory/where/files/with/data/are/stored/" # Where all my files are stored
mypath = dst2
files = [f for f in os.listdir(mypath) if isfile(join(mypath, f))] # LIST of filenames in the directory of your config files
data = os.listdir(mypath)
names = [] #All the file names will be stored here
for i in files:
x = (str(i).split())
for split in x:
names.append(split)
for i, file in enumerate(data):
a = np.genfromtxt(dst2+file)
a2 = a[:,0] # Pulls out first column = wavelength
b2 = (a[:,1]) # Pulls out second column = use this to make simulated spectrum without noise
mu, sigma = 0, 0.01 # mean and standard diviation
noise = np.random.normal(mu, sigma ,[1265,]) # creates noise
B2 = b2 + noise # simulated spectrum without noise
sigma2 = np.repeat(sigma, 1265) # sigma = constant noise at 1%
DataToAppend = []
DataToAppend.append({"{:.2f}".format(names[i])+'_Wavelength':a2,'Freq':B2,'Noise':sigma2}) # append the column with they're associated file name
print(DataToAppend)
OutPut:
[{'ColdWaterIceCH4Mabs_-6.0P0_100.0.cfg.txt_Wavelength': array([0.2 , 0.2004 , 0.2008008 , ..., 2.48939933, 2.49437813,
2.49936688]), 'Freq': array([0.71150625, 0.71394532, 0.70672077, ..., 0.33867644, 0.34125075,
0.36570275]), 'Noise': array([0.01, 0.01, 0.01, ..., 0.01, 0.01, 0.01])}]
[{'ColdBasaltCO2Mabs_-4.0P0_1.0.cfg.txt_Wavelength': array([0.2 , 0.2004 , 0.2008008 , ..., 2.48939933, 2.49437813,
2.49936688]), 'Freq': array([0.58510933, 0.57541622, 0.58468481, ..., 0.19431411, 0.19020661,
0.17070037]), 'Noise': array([0.01, 0.01, 0.01, ..., 0.01, 0.01, 0.01])}] etc..
Now I need to append this data to the end of files in a different directory as 3 seperate columns plus add some text by matching up the filenames in this directory to the ones in the first directory. So far I have this but it is very much wrong.
dir = '/Users/ccruzarc/Desktop/PSG_GPC_framework_master/Cfgs/ColdRetrievalsConfig/1' # Replace with path to your directory: absolute or relative
pattern = '' # Replace with your target substring
matching_files = [f for f in os.listdir(dir) if pattern in f]
NewConfigs = []
if matching_files = names[i]:
os.system("find . -type f ! -path "/Users/ccruzarc/Desktop/PSG_GPC_framework_master/Cfgs/ColdRetrievalsConfig/1" -exec sh -c "<DATA> /n >> {}.format(DataToAppend[i]) /n </DATA> {}" \;"
NewConfigs.append()
print(NewConfigs)