How can I read files from a list of directory?

Question

Here I have many patient fold, each fold has 10 images. And I have a list which stores some directories what I want.

How can I load/read the files only from the directory on the list(data_paths)?

# data_paths is the list
data_paths = ['./data/preprocessed_data/train/Patient009969', './data/preprocessed_data/train/Patient009928', './data/preprocessed_data/train/Patient009966'] 
data_and_seg_arr = [np.load(ii, mmap_mode='r') for ii in data_paths]


Eror:
IsADirectoryError: [Errno 21] Is a directory: './data/preprocessed_data/train/Patient009969'

I need to keep this data form. because I will have to do :

pids = [ii.split('/')[-1].split('.')[0] for ii in data_paths]
data = OrderedDict()
for ix, pid in enumerate(pids):
    data[pid] = {'data': data_and_seg_arr[ix][..., 0], 'seg': data_and_seg_arr[ix][..., 1], 'pid': pid}
return data

It looks like you want to iterate through files within those directories. You would need another for loop inside your current one. [This answer should help.](https://stackoverflow.com/a/10378012/8557739) — T Tse, Sep 21 '18 at 03:00
You need file as an input for `np.load`. So `data_paths` should contain files, not directory — ipramusinto, Sep 21 '18 at 03:04

score 0 · Answer 1 · answered Sep 21 '18 at 03:17

Have you ever tried to load only one file from your list?
Maybe you should check your "file's property", since I think the exception Errno 21 is raised by python open() not by numpy.
https://github.com/numpy/numpy/blob/v1.15.1/numpy/lib/npyio.py#L384

The whole error message will help.

score 0 · Answer 2 · answered Sep 21 '18 at 03:24

0

You'll need to create another loop iterating through the files inside the list of paths.

import os
for path in data_paths:
      for file in os.listdir(path):
             data_and_seg_arr = [np.load(file, ...

answered Sep 21 '18 at 03:24

Lucca Portes

76
10

score 0 · Answer 3 · answered Sep 21 '18 at 04:40

Assuming all the files are in jpg or png format

path="/path/to/image"
#NOTE THIS IS REALLY IMPORTANT
#IF YOUR PATH IS LIKE THIS /path/to/image YOU NEED TO ADD A / AT THE END, IF 
#YOUR PATH ALREADY HAVE THE "/" AT THE END YOU DON'T NEED TO ADD IF
#EXAMPLE PROBLEM PATH:/path/to/image
#PATH WITH NO PROBLEM /path/to/image/
jpg = glob.glob(path + "/*.jpg")
png = glob.glob(path + "/*.png")
imagelist=jpg+png


>>>print(type(imagelist))
<class 'list'>

Hope this solve your problem

How can I read files from a list of directory?

3 Answers3