g3doc/setup/your_own_dataset.md

Convert your own dataset for DeepLab2 framework

You may want to train DeepLab2 on your own dataset. Here, we provide some guidances and hopefully that will facillitate the preparation process.

1. Prepare your own dataset.

• Images should be stored either in jpg or png format.
• Annotations should be stored either in png or raw format. The DeepLab2 framework assumes the panoptic label format (i.e., panoptic_label = semantic_label * label_divisor + instance_id, where the label_divisor should be larger than the maximum number of instances per image). The raw format refers to the case where we could save the panoptic annotations in the int32 array (e.g., int32_numpy_array.tostring()).

2. Convert the dataset to TFRecord.

• Update our provided example code (e.g., build_step_data.py)to convert your dataset to TFRecord.

3. Modify the dataset.py (path: ${DEEPLAB2}/data/dataset.py) to provide your dataset information.

• Set the panoptic_label_divisor (i.e., the label_divisor above) correctly. Its value should be larger than the maximum number of instances that could appear per image in your dataset.
• Set the ignore_label properly. Pixels annotated with ignore_label are not used during both training and evaluation. If your dataset does not contain the ignore_label annotations, you could simply set it to be a large value (e.g., 255 as for Cityscapes).
• Set the class_has_instance_list properly. The variable specifies which class belongs to the thing class (i.e., countable objects such as people, cars).
• Set the colormap (for visualization) properly. You may also need to define your own colormap (see ${DEEPLAB2}/trainer/vis_utils.py).

4. Prepare the experiment config.

• Update our provided example configs (path: ${DEEPLAB2}/configs/${DATASET}/${MODEL}/${BACKBONE}) for your use case. A few things that may worth your attention:
  • Set the crop_size correctly for both training and evaluation. See Q2 in FAQ for more details.
  • Tune the config flags for your dataset (e.g., base_learning_rate, training_number_of_step, and so on).

Finally, if your dataset only contains semantic segmentation annotations, you could still use DeepLab2 framework with some minor changes:

1. Set panoptic_label_divisor=None in dataset.py (we also provide one example in dataset.py, where we only train with semantic segmentation on Cityscapes).
2. Have a config similar to ${DEEPLAB2}/configs/cityscapes/panoptic_deeplab/ resnet50_os32_semseg.textproto, where the instance branch is not initiated.

At this point, you are good to go! Enjoy training DeepLab2!