5

From my understanding, RandomHorizontalFlip etc. replace image rather than adding new images to dataset. How do I increase my dataset size by adding augmented images to dataset using PyTorch?

I have gone through the links posted & haven't found a solution. I want to increase the data size by adding flipped/rotated images - but the post addresses the in-place processing of images.

Thanks.

Vidyadhar Mudium
  • 77
  • 1
  • 5
  • Check https://stackoverflow.com/questions/51677788/data-augmentation-in-pytorch – kHarshit Mar 03 '19 at 15:32
  • 2
    Possible duplicate of [Data Augmentation in PyTorch](https://stackoverflow.com/questions/51677788/data-augmentation-in-pytorch) – Jatentaki Mar 04 '19 at 14:24

2 Answers2

2

Why do you want it? Generally speaking, it is enough to increase the number of epochs over the dataset, and your model will see the original and the augmented version of every image at least once (assuming a relatively high number of epochs).

Explanation:

For instance, if your augmentation has a chance of 50% to be applied, after 100 epochs, for every sample you will get ~50 samples of the original image and ~50 augmented samples. So, increasing the dataset size is equivalent to add epochs but (maybe) less efficient in terms of memory (need to store the images in memory to have high performances).

Idan Azuri
  • 623
  • 4
  • 17
  • Doesn’t increasing the epochs defeat the purpose of augmentation ? Model will tend to over fit as we increase number of epochs. Am I missing something here? Please correct me. – Vidyadhar Mudium Oct 03 '19 at 16:01
  • 1
    Not necessarily. First, it depends on the task you are trying to solve. For example, train a generative model such as GAN, VAE won't overfit your dataset (as long you are avoiding mode collapse). In the case of a classification task, it also very hard to overfit your model by adding epochs (don't forget that you just augmented your data to make your model generalize better). – Idan Azuri Oct 04 '19 at 10:03
-1

It needs some explanation. transforms.Compose creates a thing that is in the <class 'torchvision.transforms.transforms.Compose'> (you can know its class by using type()
e.g.

train_tfm = transforms.Compose([transforms.Resize((128, 128)),transforms.ToTensor(),])
print(type(train_tfm))

so it is not the data set and therefore cannot be concerted directly

Search in the code to see how the variable that carries augmentation instructions is carried on.

There should be something of a data reader, which could be in the class of <class 'torchvision.datasets.folder.DatasetFolder'>

then it is fine to use:

concat_dataset = ConcatDataset([train_set_1, train_set_2])
YY C
  • 9
  • 1
  • 3