0

I have a coco style dataset and struggling to code the dataloader, in which Dataset class is the same as this code and __getitem__ returns the same outputs in the link (images, image_metas, rpn_match, rpn_bbox, gt_class_ids, gt_boxes, gt_masks)

When I am trying to load with pytorch default collate_fn I am getting the following error:

# Parameters for Dataloader
params = {'batch_size': 8,
          'shuffle': True,
          'num_workers': 4}
val_set = Dataset(dataset_val, config, augment=True)
val_loader = DataLoader(val_set, **params)
batch = next(iter(val_loader))
RuntimeError: stack expects each tensor to be equal size, but got [1] at entry 0 and [5] at entry 2

Then I tried to define the custom collate_fn according to this link and this link:

def my_collate_fn(data):
     zipped = zip(data)
     return list(zipped)

val_loader = DataLoader(val_set, **params, collate_fn = my_collate_fn)

where I cannot get the correct items of batch, for example:

# Loop over epochs
for epoch in range(0,1): #max_epochs
    # Training
    for inputs in train_loader:
        # Transfer to GPU
        images = inputs[0]
        image_metas = inputs[1]
        rpn_match = inputs[2]
        rpn_bbox = inputs[3]
        gt_class_ids = inputs[4]
        gt_boxes = inputs[5]
        gt_masks = inputs[6]

where inputs[0] returns the first item of batch (including images, image_metas, rpn_match, rpn_bbox, gt_class_ids, gt_boxes, gt_masks) Not the images of all 8 items in the batch.

How can I resolve this issue? How to load the data correctly by collate_fn?

S.EB
  • 1,966
  • 4
  • 29
  • 54
  • Assuming the input of your collate function `data` to be something like `[(A0, B0, ..., G0), (A0, B0, ..., G0), ..., (A7, ..., G7)]` (batch size 8), you should stack each element (those share same alphabet) and return them as a list. – Hayoung Aug 18 '23 at 07:55

0 Answers0