How to automate loading multiple files into numpy arrays using a simple "for" loop?

Question

I usually load my data, that -in most cases- consists of only two columns using np.loadtxt cammand as follows:

x0, y0 = np.loadtxt('file_0.txt', delimiter='\t', unpack=True)
x1, y1 = np.loadtxt('file_1.txt', delimiter='\t', unpack=True)
.
.
xn, yn = np.loadtxt('file_n.txt', delimiter='\t', unpack=True)

then plot each pair on its own, which is not ideal!

I want to make a simple "for" loop that goes for all text files in the same directory, load the files and plot them on the same figure.

score 1 · Answer 1 · answered Dec 27 '20 at 14:19

import os
import matplotlib.pyplot as plt

# A list of all file names that end with .txt
myfiles = [myfile for myfile in os.listdir() if myfile.endswith(".txt")]

# Create a new figure
plt.figure()

# iterate over the file names
for myfile in myfiles:
   # load the x, y
   x, y = np.loadtxt(myfile, delimiter='\t', unpack=True)

   # plot the values
   plt.plot(x, y)

# show the figure after iterating over all files and plotting.
plt.show()

score 0 · Answer 2 · answered Dec 27 '20 at 14:10

0

Load all the files in a dictionary using:

d = {}
for i in range(n):
    d[i] = np.loadtxt('file_' + str(i) + '.txt', delimiter='\t', unpack=True)

Now, to access kth file, use d[k] or:

xk, yk = d[k]

Since, you have not mentioned about the data in the files and the plot you want to create, it's hard to tell what to do. But for plotting, you can refer Mttplotlib or Seaborn libraries.

answered Dec 27 '20 at 14:10

Rahul Vishwakarma

1,446
2
7
22

Thanks. But, how to load files of arbitrary names?! – ali_05 Dec 27 '20 at 14:15
[Refer this](https://stackoverflow.com/a/53503041/13720565) – Rahul Vishwakarma Dec 27 '20 at 14:19

score 0 · Answer 3 · answered Dec 27 '20 at 14:19

You can also use glob to get all the files -

from glob import glob
import numpy as np
import os

res = []

file_path = "YOUR PATH"
file_pattern = "file_*.txt"

files_list = glob(os.path.join(file_path,file_pattern))

for f in files_list:
    print(f'----- Loading {f} -----')
    x, y = np.loadtxt(f, delimiter='\t', unpack=True)
    res += [(x,y)]

res will contain your file contents at each index value corresponding to f

How to automate loading multiple files into numpy arrays using a simple "for" loop?

3 Answers3