Dumping Image data and load using pytorch dataloader

Question

I want to dump the data so that I can load it back for training my model.

My code snipped for dumping the data:

for batch_idx, (image, label) in enumerate(dataloader):
    image, label = image.to(device), label.to(device)
    perturbed_image = attack.perturb(image, label)
    
    #---------- Classifier ----------
    predict_A = classifier(perturbed_image)
    pred_label = torch.max(predict_A.data, 1)[1]
    
    if pred_label != label:
        adv_data.append( (perturbed_image.to("cpu"), label.to("cpu")) )

Is there any other way I can dump it correctly so as to load it in the torch.utils.data.DataLoader.

score 0 · Accepted Answer · answered Aug 06 '20 at 08:19

0

The most straight forward approach would be to use torch.save to save the actual tensors of perturbed_image and label as binary files, and then use a custom Dataset. Note that the saved tensors are not single image/label, but rather batches of images/labels. Your new custom Dataset should account for it.

answered Aug 06 '20 at 08:19

Shai

111,146
38
238
371

Thank you @Shai. Can you please provide a code snippet of how would I use custom Dataset to load it to the `torch.utils.data.DataLoader`. – feed_me_pi Aug 06 '20 at 15:09
@x2212 please look at the linked answer – Shai Aug 06 '20 at 15:25

Dumping Image data and load using pytorch dataloader

1 Answers1