What are possible values for data_augmentation_options in the TensorFlow Object Detection pipeline configuration?

Question

I have successfully trained an object detection model with TensorFlow with the sample configurations given here: https://github.com/tensorflow/models/tree/master/object_detection/samples/configs

Now I want to fine tune my configuration to get better results. One of the promising options I see in there is "data_augmentation_options" under "train_config". Currently, it looks like this:

train_config: {
  batch_size: 1
  ...
  data_augmentation_options {
    random_horizontal_flip {
    }
  }
}

Are there other options to do random scaling, cropping or tweaking of brightness?

score 94 · Accepted Answer · answered Oct 24 '17 at 01:59

94

The list of options is provided in preprocessor.proto:

NormalizeImage normalize_image = 1;
RandomHorizontalFlip random_horizontal_flip = 2;
RandomPixelValueScale random_pixel_value_scale = 3;
RandomImageScale random_image_scale = 4;
RandomRGBtoGray random_rgb_to_gray = 5;
RandomAdjustBrightness random_adjust_brightness = 6;
RandomAdjustContrast random_adjust_contrast = 7;
RandomAdjustHue random_adjust_hue = 8;
RandomAdjustSaturation random_adjust_saturation = 9;
RandomDistortColor random_distort_color = 10;
RandomJitterBoxes random_jitter_boxes = 11;
RandomCropImage random_crop_image = 12;
RandomPadImage random_pad_image = 13;
RandomCropPadImage random_crop_pad_image = 14;
RandomCropToAspectRatio random_crop_to_aspect_ratio = 15;
RandomBlackPatches random_black_patches = 16;
RandomResizeMethod random_resize_method = 17;
ScaleBoxesToPixelCoordinates scale_boxes_to_pixel_coordinates = 18;
ResizeImage resize_image = 19;
SubtractChannelMean subtract_channel_mean = 20;
SSDRandomCrop ssd_random_crop = 21;
SSDRandomCropPad ssd_random_crop_pad = 22;
SSDRandomCropFixedAspectRatio ssd_random_crop_fixed_aspect_ratio = 23;

You can see the details about each option in preprocessor.py. Arguments can be provided as key-value pairs.

  data_augmentation_options {
    ssd_random_crop {
    }
  }
  data_augmentation_options {
    random_pixel_value_scale {
      minval: 0.6
    }
  }

answered Oct 24 '17 at 01:59

Najih Km

1,011
9
6

8

Are all of these data augmentation operations done sequentially? Can we specify that they be done at random? – Jash Shah May 08 '18 at 09:11
Can you also tell us how to use the arguments for `ssd_random_crop_pad` operation? – Jash Shah Jun 10 '18 at 13:08
3

https://github.com/tensorflow/models/blob/master/research/object_detection/builders/preprocessor_builder_test.py, this file could help who want details of configuration – Raymond Cheng Sep 03 '18 at 09:47
4

Make sure to check out the preprocessor.proto link. There are a lot more options now. – Devin Haslam Oct 02 '19 at 18:27
One question: If I use data augmentation then should I have to take care about label file? Because if augmentation applies random crop or rotation to images then ground truth bounding box doesn't match. Is this handle by tensorflow object detection API? – Saurabh Chauhan Mar 10 '20 at 09:33
Thanks! Works! Perfectly! – ambigus9 Sep 11 '20 at 17:07
1

Object Detection API handles that internally. You don't have to worry about the labels. They are also transformed automatically when you apply data augmentation. @SaurabhChauhan – Matt May 11 '21 at 12:24

Nannigalaxy · Answer 2 · 2021-07-12T18:43:46.113

Adding to solution, here is the entire list added inside

data_augmentation_options {
 ...
  }

Augmentation options (found here):

normalize_image {
  original_minval: 0.0
  original_maxval: 255.0
  target_minval: -1.0
  target_maxval: 1.0
}


random_horizontal_flip {
  keypoint_flip_permutation: 1
  keypoint_flip_permutation: 0
  keypoint_flip_permutation: 2
  keypoint_flip_permutation: 3
  keypoint_flip_permutation: 5
  keypoint_flip_permutation: 4
  probability: 0.5
}


random_vertical_flip {
  keypoint_flip_permutation: 1
  keypoint_flip_permutation: 0
  keypoint_flip_permutation: 2
  keypoint_flip_permutation: 3
  keypoint_flip_permutation: 5
  keypoint_flip_permutation: 4
  probability: 0.5
}


random_rotation90 {
  keypoint_rot_permutation: 3
  keypoint_rot_permutation: 0
  keypoint_rot_permutation: 1
  keypoint_rot_permutation: 2
  probability: 0.5
}


random_pixel_value_scale {
  minval: 0.8
  maxval: 1.2
}


random_image_scale {
  min_scale_ratio: 0.8
  max_scale_ratio: 2.2
}


random_rgb_to_gray {
  probability: 0.8
}


random_adjust_brightness {
  max_delta: 0.2
}


random_adjust_contrast {
  min_delta: 0.7
  max_delta: 1.1
}


random_adjust_hue {
  max_delta: 0.01
}


random_adjust_saturation {
  min_delta: 0.75
  max_delta: 1.15
}


random_distort_color {
  color_ordering: 1
}


random_jitter_boxes {
  ratio: 0.1
  jitter_mode: SHRINK
}


random_crop_image {
  min_object_covered: 0.75
  min_aspect_ratio: 0.75
  max_aspect_ratio: 1.5
  min_area: 0.25
  max_area: 0.875
  overlap_thresh: 0.5
  clip_boxes: False
  random_coef: 0.125
}


random_pad_image {
}


random_absolute_pad_image {
  max_height_padding: 50
  max_width_padding: 100
}


random_crop_pad_image {
  min_object_covered: 0.75
  min_aspect_ratio: 0.75
  max_aspect_ratio: 1.5
  min_area: 0.25
  max_area: 0.875
  overlap_thresh: 0.5
  clip_boxes: False
  random_coef: 0.125
}


random_crop_pad_image {
  min_object_covered: 0.75
  min_aspect_ratio: 0.75
  max_aspect_ratio: 1.5
  min_area: 0.25
  max_area: 0.875
  overlap_thresh: 0.5
  clip_boxes: False
  random_coef: 0.125
  min_padded_size_ratio: 0.5
  min_padded_size_ratio: 0.75
  max_padded_size_ratio: 0.5
  max_padded_size_ratio: 0.75
}


random_crop_to_aspect_ratio {
  aspect_ratio: 0.85
  overlap_thresh: 0.35
  clip_boxes: False
}


random_black_patches {
  max_black_patches: 20
  probability: 0.95
  size_to_image_ratio: 0.12
}


random_jpeg_quality {
  random_coef: 0.5
  min_jpeg_quality: 40
  max_jpeg_quality: 90
}


random_downscale_to_target_pixels {
  random_coef: 0.5
  min_target_pixels: 200
  max_target_pixels: 900
}


random_patch_gaussian {
  random_coef: 0.5
  min_patch_size: 10
  max_patch_size: 300
  min_gaussian_stddev: 0.2
  max_gaussian_stddev: 1.5
}


autoaugment_image {
  policy_name: 'v0'
}


drop_label_probabilistically{
  label: 2
  drop_probability: 0.5
}


remap_labels{
  original_labels: 1
  original_labels: 2
  new_label: 3
}


random_resize_method {
  target_height: 75
  target_width: 100
}


resize_image {
  new_height: 75
  new_width: 100
  method: BICUBIC
}



random_self_concat_image {
  concat_vertical_probability: 0.5
  concat_horizontal_probability: 0.25
}


ssd_random_crop {
  operations {
    min_object_covered: 0.0
    min_aspect_ratio: 0.875
    max_aspect_ratio: 1.125
    min_area: 0.5
    max_area: 1.0
    overlap_thresh: 0.0
    clip_boxes: False
    random_coef: 0.375
  }
  operations {
    min_object_covered: 0.25
    min_aspect_ratio: 0.75
    max_aspect_ratio: 1.5
    min_area: 0.5
    max_area: 1.0
    overlap_thresh: 0.25
    clip_boxes: True
    random_coef: 0.375
  }
}


ssd_random_crop {
}


ssd_random_crop_pad {
  operations {
    min_object_covered: 0.0
    min_aspect_ratio: 0.875
    max_aspect_ratio: 1.125
    min_area: 0.5
    max_area: 1.0
    overlap_thresh: 0.0
    clip_boxes: False
    random_coef: 0.375
    min_padded_size_ratio: [1.0, 1.0]
    max_padded_size_ratio: [2.0, 2.0]
    pad_color_r: 0.5
    pad_color_g: 0.5
    pad_color_b: 0.5
  }
  operations {
    min_object_covered: 0.25
    min_aspect_ratio: 0.75
    max_aspect_ratio: 1.5
    min_area: 0.5
    max_area: 1.0
    overlap_thresh: 0.25
    clip_boxes: True
    random_coef: 0.375
    min_padded_size_ratio: [1.0, 1.0]
    max_padded_size_ratio: [2.0, 2.0]
    pad_color_r: 0.5
    pad_color_g: 0.5
    pad_color_b: 0.5
  }
}


ssd_random_crop_fixed_aspect_ratio {
  operations {
    min_object_covered: 0.0
    min_area: 0.5
    max_area: 1.0
    overlap_thresh: 0.0
    clip_boxes: False
    random_coef: 0.375
  }
  operations {
    min_object_covered: 0.25
    min_area: 0.5
    max_area: 1.0
    overlap_thresh: 0.25
    clip_boxes: True
    random_coef: 0.375
  }
  aspect_ratio: 0.875
}


ssd_random_crop_pad_fixed_aspect_ratio {
  operations {
    min_object_covered: 0.0
    min_aspect_ratio: 0.875
    max_aspect_ratio: 1.125
    min_area: 0.5
    max_area: 1.0
    overlap_thresh: 0.0
    clip_boxes: False
    random_coef: 0.375
  }
  operations {
    min_object_covered: 0.25
    min_aspect_ratio: 0.75
    max_aspect_ratio: 1.5
    min_area: 0.5
    max_area: 1.0
    overlap_thresh: 0.25
    clip_boxes: True
    random_coef: 0.375
  }
  aspect_ratio: 0.875
  min_padded_size_ratio: [1.0, 1.0]
  max_padded_size_ratio: [2.0, 2.0]
}


convert_class_logits_to_softmax {
    temperature: 2
}


random_square_crop_by_scale {
  scale_min: 0.25
  scale_max: 2.0
  num_scales: 8
}


adjust_gamma {
  gamma: 2.2
  gain: 2.0
}

does adding all these options increase processing time..does it also increase accuracy? — Apricot, Jul 22 '21 at 11:27
Yes, adding augmentation increases training time and also helps model to generalize to different variations. — Nannigalaxy, Jul 23 '21 at 12:04
how to find out which value is appropriate for a particular option? is there a good reference? thanks — Nguai al, Jan 06 '22 at 01:42
@Apricot - It can help with over fitting but it can also decrease accuracy. — Nguai al, Jan 10 '22 at 19:32

What are possible values for data_augmentation_options in the TensorFlow Object Detection pipeline configuration?

2 Answers2

Linked