Copy loaded Images to test, train and validation folders

Question

Here I have written a code to load images then after that what should I do to split those images into three folders ; train, test and validation using ratios 70%, 15% and 15% respectively.

      from os import listdir
        from PIL import Image as PImage
        import split_folders
        import os, os.path
        import numpy as np
        #imgs.append(Image.open(os.path.join(path,image))

        def loadImages(path):
            imagesList = listdir(path)
            loadedImages = []
            for image in imagesList:
                with open(os.path.join(path, image), 'rb') as i:
                    img = PImage.open(i)
                    loadedImages.append(img)
            return loadedImages

        path = "./Inputs/"
        imgs = loadImages(path)

        for img in imgs:
            print(img)




train, validate, test = np.split(imgs.sample(frac=1), [int(.7*len(imgs)), int(.85*len(imgs))])

this is wrong since this not supports list object

So any solution?

Can you please clarify a little bit more? – LOrD_ARaGOrN Dec 03 '18 at 09:57 — LOrD_ARaGOrN, Dec 03 '18 at 09:57

score 1 · Accepted Answer · answered Dec 03 '18 at 10:00

1

As far as I understand, function loadImages returns python list object. Since there's no list.sample() method, the string

train, validate, test = np.split(imgs.sample(frac=1), [int(.7*len(imgs)), int(.85*len(imgs))])

is invalid. So, there's some ways to shuffle all your data; for example, you can use into numpy.random.shuffle to reorder list values inplace:

np.random.shuffle(imgs)  # now this list is shuffled
train, validate, test = np.split(imgs, [int(.7*len(imgs)), int(.85*len(imgs))])

Note, np.random.shuffle leaves the type of object intact (if it's a list). I guess other parts of the code are correct.

lst = [1,2,3]
type(lst)
Out:
<class'list'>

np.random.shuffle(lst)

type(lst)
Out:
<class'list'>

Also, sample method is one of the pandas package default methods of data resampling, but i guess you need no pandas here

answered Dec 03 '18 at 10:00

Mikhail Stepanov

3,680
3
23
24

Yes, loadImages returns a list object. yeah, no need of "pandas" here. Thank you very much for your advice @Mikhail – EverythingNeedToBeKnown Dec 03 '18 at 10:23
Then how can I load them to a folder @Mikhail – EverythingNeedToBeKnown Dec 03 '18 at 10:43
can I use shutil.copy , I think no, it's expect string – EverythingNeedToBeKnown Dec 03 '18 at 10:44
Something like this will work: https://stackoverflow.com/questions/14452824/how-can-i-save-an-image-with-pil; if you need new folder, you can create it safely as was stated in this ansewer: https://stackoverflow.com/questions/273192/how-can-i-safely-create-a-nested-directory-in-python. Besides, I'm not very familiar with a `PIL` :) – Mikhail Stepanov Dec 03 '18 at 10:48

score 0 · Answer 2 · answered Dec 11 '18 at 07:06

To copy files into folders we can use shutil.copyfile(os.path.join(resized_dir,x),os.path.join(train_dir,x)) that is shutil.copyfile method:

Below code is the answer for this question; (got list image file names instead of list of images;

for x in train:
  shutil.copyfile(os.path.join(resized_dir,x),os.path.join(train_dir,x))

for y in validate:
  shutil.copyfile(os.path.join(resized_dir,y),os.path.join(valid_dir,y))

for z in test:
  shutil.copyfile(os.path.join(resized_dir,z),os.path.join(test_dir,z))

Copy loaded Images to test, train and validation folders

2 Answers2