From 65331e8ac9c58bb263ddbba1f4226ca11bdc4d07 Mon Sep 17 00:00:00 2001 From: Om Doiphode Date: Wed, 14 Aug 2024 16:01:14 +0530 Subject: [PATCH 1/4] Added augmentations --- docs/augmentations.md | 184 ++++++++++++++++++++++++++++++++++++++++++ docs/index.rst | 1 + 2 files changed, 185 insertions(+) create mode 100644 docs/augmentations.md diff --git a/docs/augmentations.md b/docs/augmentations.md new file mode 100644 index 00000000..5bc13a1c --- /dev/null +++ b/docs/augmentations.md @@ -0,0 +1,184 @@ +# Augmentations + +## Overview + +DeepForest allows users to customize data augmentations by defining them in a configuration file (`deepforest_config.yml`). This configuration is then passed to the model using the `get_augmentations` function. This guide will walk you through the steps to add custom augmentations and apply them in your DeepForest model. + +## Configuration File (`deepforest_config.yml`) + +The `deepforest_config.yml` file is where you define the data augmentations you want to apply during training or validation. The augmentations are specified under the `train` and `validation` sections. + +### Example Configuration + +Below is an example configuration file with custom augmentations: + +```yaml +# Dataloaders +workers: 1 +devices: auto +accelerator: auto +batch_size: 1 + +# Model Architecture +architecture: 'retinanet' +num_classes: 1 +nms_thresh: 0.05 + +# Architecture specific params +retinanet: + score_thresh: 0.1 + +train: + csv_file: path/to/your/train/annotations.csv + root_dir: path/to/your/train/images/ + augmentations: + - type: RandomSizedBBoxSafeCrop + params: + height: 300 + width: 300 + erosion_rate: 0.0 + interpolation: 1 + always_apply: False + - type: PadIfNeeded + params: + always_apply: False + p: 1.0 + min_height: 1024 + min_width: 1024 + position: "center" + border_mode: 0 + value: 0 + - type: ToTensorV2 + - type: BBox + params: + format: pascal_voc + label_fields: [category_ids] + + default_augmentations: + - type: HorizontalFlip + params: + p: 0.5 + - type: VerticalFlip + params: + p: 0.5 + - type: RandomBrightnessContrast + params: + brightness_limit: 0.2 + contrast_limit: 0.2 + p: 0.5 + - type: RandomRotate90 + params: + p: 0.5 + - type: Normalize + params: + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + max_pixel_value: 255.0 + p: 1.0 + - type: ToTensorV2 + - type: BBox + params: + format: pascal_voc + label_fields: [category_ids] + +validation: + csv_file: path/to/your/val/annotations.csv + root_dir: path/to/your/val/images/ + augmentations: + - type: Normalize + params: + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + max_pixel_value: 255.0 + p: 1.0 + - type: ToTensorV2 + - type: BBox + params: + format: pascal_voc + label_fields: [category_ids] +``` +### Default Augmentations +If the `augmentations` key under `train` or `validation` is empty, DeepForest will automatically apply the `default_augmentations` specified in the configuration file. This ensures that some level of augmentation is always applied, even if no custom augmentations are defined. + +Example with empty augmentations: +```yaml +train: + csv_file: path/to/your/train/annotations.csv + root_dir: path/to/your/train/images/ + augmentations: # No custom augmentations + default_augmentations: + - type: HorizontalFlip + params: + p: 0.5 + - type: VerticalFlip + params: + p: 0.5 + - type: RandomBrightnessContrast + params: + brightness_limit: 0.2 + contrast_limit: 0.2 + p: 0.5 + - type: RandomRotate90 + params: + p: 0.5 + - type: Normalize + params: + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + max_pixel_value: 255.0 + p: 1.0 + - type: ToTensorV2 + +``` + +## Applying the Augmentations + +After defining the augmentations in your configuration file, you can apply them to your DeepForest model by using the `get_augmentations` function. + +### Example Usage + +```python +from deepforest.augmentations import get_augmentations +from deepforest import main + +# Initialize the model with custom augmentations +m = main.deepforest(transforms=get_augmentations) +``` + +### How to Add Custom Augmentations + +If you want to add a new augmentation, follow these steps: + +1. **Edit the `deepforest_config.yml` file**: + - Add your desired augmentation under the `augmentations` section in either `train` or `validation`. + - Specify the `type` of augmentation and the corresponding `params`. + +2. **Pass the `get_augmentations` function to your model**: + - As shown in the example above, initialize your model with the augmentations by passing the `get_augmentations` function. + +### Supported Augmentations + +Some of the supported augmentations in DeepForest include: + +- `HorizontalFlip`: Flips the image horizontally with a given probability. +- `VerticalFlip`: Flips the image vertically with a given probability. +- `RandomRotate90`: Rotates the image by 90 degrees randomly. +- `RandomCrop`: Randomly crops a part of the image. +- `RandomSizedBBoxSafeCrop`: Crops a part of the image ensuring bounding boxes remain within the cropped area. +- `PadIfNeeded`: Pads the image to a minimum height and width. +- `RandomBrightnessContrast`: Randomly adjusts the brightness and contrast. +- `Normalize`: Normalizes the image with the given mean and standard deviation. +- `ToTensorV2`: Converts the image and bounding boxes to PyTorch tensors. + +For a complete list of supported augmentations, refer to the [Albumentations documentation](https://albumentations.ai/docs/). + +## Troubleshooting + +- **Error: Missing key in the configuration file**: Ensure that all required keys and parameters are correctly defined in the `deepforest_config.yml` file. +- **Invalid Augmentation Type**: Double-check the spelling and case of the augmentation type in your configuration file. + +## Conclusion + +Customizing data augmentations in DeepForest is simple and flexible. By defining your desired augmentations in the `deepforest_config.yml` file and passing them to the model using `get_augmentations`, you can easily tailor the training and validation processes to suit your specific needs. + +--- \ No newline at end of file diff --git a/docs/index.rst b/docs/index.rst index bd8ff217..8447ad55 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -46,6 +46,7 @@ Get suggestions on how to improve a model by using the [discussion board](https: getting_started Reading_and_Writing ConfigurationFile + augmentations annotation training Evaluation From 75a7ec1273282b5f1cdc6a548568fdbbb0a4fe21 Mon Sep 17 00:00:00 2001 From: Om Doiphode Date: Wed, 14 Aug 2024 16:02:38 +0530 Subject: [PATCH 2/4] Added augmentations --- deepforest/augmentations.py | 145 ++++++++++++++++++++++++++ deepforest/data/deepforest_config.yml | 58 +++++++++++ deepforest_config.yml | 60 ++++++++++- 3 files changed, 262 insertions(+), 1 deletion(-) create mode 100644 deepforest/augmentations.py diff --git a/deepforest/augmentations.py b/deepforest/augmentations.py new file mode 100644 index 00000000..bd9e8036 --- /dev/null +++ b/deepforest/augmentations.py @@ -0,0 +1,145 @@ +import albumentations as A +from deepforest import utilities, get_data +import cv2 +from albumentations.pytorch import ToTensorV2 + +def get_augmentations(augment=True): + try: + config_path = get_data("deepforest_config.yml") + except Exception as e: + raise ValueError("No config file provided and deepforest_config.yml " + "not found either in local directory or in installed " + "package location. {}".format(e)) + + augmentations = [] + format_bbox='pascal_voc' + label_fields=['category_ids'] + try: + config = utilities.read_config(config_path) + augment_config = config["train" if augment else "validation"]["augmentations"] + except KeyError as e: + raise ValueError("Missing key in the configuration file: {}".format(e)) + + if not augment_config: + default_augmentations=config["train" if augment else "validation"]["default_augmentations"] + for augment in default_augmentations: + augment_config_type=augment.get('type') + params = augment.get('params', {}) + + if augment_config_type.lower()=='horizontalflip': + augmentations.append(A.HorizontalFlip(**params)) + + elif augment_config_type.lower()=='verticalflip': + augmentations.append(A.VerticalFlip(**params)) + + elif augment_config_type.lower()=='randomrotate90': + augmentations.append(A.RandomRotate90(**params)) + + elif augment_config_type.lower()=='randomcrop': + augmentations.append(A.RandomCrop(**params)) + + elif augment_config_type.lower()=='randombrightnesscontrast': + augmentations.append(A.RandomBrightnessContrast(**params)) + + elif augment_config_type.lower()=='normalize': + augmentations.append(A.Normalize(**params)) + + elif augment_config_type.lower()=='totensorv2': + augmentations.append(ToTensorV2()) + + elif augment_config_type.lower()=='bbox': + format_bbox=params['format'] + label_fields=params['label_fields'] + + transform = A.Compose( + augmentations, + bbox_params=A.BboxParams(format=format_bbox, label_fields=label_fields) + ) + + return transform + + for augment in augment_config: + augment_config_type=augment.get('type') + params = augment.get('params', {}) + if augment_config_type.lower() == "randomsizedbboxsafecrop": + try: + augmentations.append(A.RandomSizedBBoxSafeCrop(**params)) + except KeyError as e: + raise ValueError("Missing parameter for RandomSizedBBoxSafeCrop: {}".format(e)) + + elif augment_config_type.lower() == "padifneeded": + min_height = params.get('min_height') + pad_height_divisor = params.get('pad_height_divisor', 'None') + + try: + min_width = params['min_width'] + position = params.get('position', 'top_left') + border_mode = params.get('border_mode', cv2.BORDER_CONSTANT) + value = params.get('value', 0) + mask_value = params.get('mask_value', None) + always_apply = params.get('always_apply', False) + p = params.get('p', 1.0) + except KeyError as e: + raise ValueError("Missing parameter for PadIfNeeded: {}".format(e)) + + if min_height == 'None' and pad_height_divisor != 'None': + augmentations.append(A.PadIfNeeded( + min_height=None, + min_width=min_width, + pad_height_divisor=pad_height_divisor, + position=position, + border_mode=border_mode, + value=value, + mask_value=mask_value, + always_apply=always_apply, + p=p)) + elif min_height != "None" and pad_height_divisor == "None": + augmentations.append(A.PadIfNeeded( + min_height=min_height, + min_width=min_width, + position=position, + border_mode=border_mode, + value=value, + mask_value=mask_value, + always_apply=always_apply, + p=p)) + + elif augment_config_type.lower() == "totensorv2": + augmentations.append(ToTensorV2()) + + elif augment_config_type.lower()=='horizontalflip': + augmentations.append(A.HorizontalFlip(**params)) + + elif augment_config_type.lower()=='verticalflip': + augmentations.append(A.VerticalFlip(**params)) + + elif augment_config_type.lower()=='randomrotate90': + augmentations.append(A.RandomRotate90(**params)) + + elif augment_config_type.lower()=='randomcrop': + augmentations.append(A.RandomCrop(**params)) + + elif augment_config_type.lower()=='randombrightnesscontrast': + augmentations.append(A.RandomBrightnessContrast(**params)) + + elif augment_config_type.lower()=='normalize': + augmentations.append(A.Normalize(**params)) + + elif augment_config_type.lower()=='totensorv2': + augmentations.append(ToTensorV2()) + + elif augment_config_type.lower()=='bbox': + format_bbox=params['format'] + label_fields=params['label_fields'] + + transform = A.Compose( + augmentations, + bbox_params=A.BboxParams(format=format_bbox, label_fields=label_fields) + ) + + return transform + +if __name__ == "__main__": + augmentations = get_augmentations() + print("Augmentations created:", augmentations) + diff --git a/deepforest/data/deepforest_config.yml b/deepforest/data/deepforest_config.yml index 0a541578..9b3ac19d 100644 --- a/deepforest/data/deepforest_config.yml +++ b/deepforest/data/deepforest_config.yml @@ -20,6 +20,57 @@ retinanet: train: csv_file: root_dir: + augmentations: + - type: RandomSizedBBoxSafeCrop + params: + height: 300 + width: 300 + erosion_rate: 0.0 + interpolation: 1 + always_apply: False + - type: PadIfNeeded + params: + always_apply: False + p: 1.0 + min_height: 1024 + min_width: 1024 + pad_height_divisor: None + pad_width_divisor: None + position: "center" # Store as string and convert in code + border_mode: 0 # cv2.BORDER_CONSTANT as integer + value: 0 + - type: ToTensorV2 + - type: BBox + params: + format: pascal_voc + label_fields: [category_ids] + + default_augmentations: + - type: HorizontalFlip + params: + p: 0.5 + - type: VerticalFlip + params: + p: 0.5 + - type: RandomBrightnessContrast + params: + brightness_limit: 0.2 + contrast_limit: 0.2 + p: 0.5 + - type: RandomRotate90 + params: + p: 0.5 + - type: Normalize + params: + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + max_pixel_value: 255.0 + p: 1.0 + - type: ToTensorV2 + - type: BBox + params: + format: pascal_voc + label_fields: [category_ids] # Optimizer initial learning rate lr: 0.001 @@ -55,6 +106,13 @@ validation: # callback args csv_file: root_dir: + augmentations: + default_augmentations: + - type: ToTensorV2 + - type: BBox + params: + format: pascal_voc + label_fields: [category_ids] # Intersection over union evaluation iou_threshold: 0.4 val_accuracy_interval: 20 diff --git a/deepforest_config.yml b/deepforest_config.yml index 5f28694a..9b3ac19d 100644 --- a/deepforest_config.yml +++ b/deepforest_config.yml @@ -20,11 +20,62 @@ retinanet: train: csv_file: root_dir: + augmentations: + - type: RandomSizedBBoxSafeCrop + params: + height: 300 + width: 300 + erosion_rate: 0.0 + interpolation: 1 + always_apply: False + - type: PadIfNeeded + params: + always_apply: False + p: 1.0 + min_height: 1024 + min_width: 1024 + pad_height_divisor: None + pad_width_divisor: None + position: "center" # Store as string and convert in code + border_mode: 0 # cv2.BORDER_CONSTANT as integer + value: 0 + - type: ToTensorV2 + - type: BBox + params: + format: pascal_voc + label_fields: [category_ids] + + default_augmentations: + - type: HorizontalFlip + params: + p: 0.5 + - type: VerticalFlip + params: + p: 0.5 + - type: RandomBrightnessContrast + params: + brightness_limit: 0.2 + contrast_limit: 0.2 + p: 0.5 + - type: RandomRotate90 + params: + p: 0.5 + - type: Normalize + params: + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + max_pixel_value: 255.0 + p: 1.0 + - type: ToTensorV2 + - type: BBox + params: + format: pascal_voc + label_fields: [category_ids] # Optimizer initial learning rate lr: 0.001 scheduler: - type: + type: params: # Common parameters T_max: 10 @@ -55,6 +106,13 @@ validation: # callback args csv_file: root_dir: + augmentations: + default_augmentations: + - type: ToTensorV2 + - type: BBox + params: + format: pascal_voc + label_fields: [category_ids] # Intersection over union evaluation iou_threshold: 0.4 val_accuracy_interval: 20 From 5a326e82f29a8157531a889c7cac42c0a5d0036f Mon Sep 17 00:00:00 2001 From: Om Doiphode Date: Wed, 14 Aug 2024 17:24:23 +0530 Subject: [PATCH 3/4] Fixed formatting issues --- deepforest/augmentations.py | 166 ++++++++++++++++++------------------ 1 file changed, 81 insertions(+), 85 deletions(-) diff --git a/deepforest/augmentations.py b/deepforest/augmentations.py index bd9e8036..60b719bf 100644 --- a/deepforest/augmentations.py +++ b/deepforest/augmentations.py @@ -3,6 +3,7 @@ import cv2 from albumentations.pytorch import ToTensorV2 + def get_augmentations(augment=True): try: config_path = get_data("deepforest_config.yml") @@ -10,67 +11,68 @@ def get_augmentations(augment=True): raise ValueError("No config file provided and deepforest_config.yml " "not found either in local directory or in installed " "package location. {}".format(e)) - + augmentations = [] - format_bbox='pascal_voc' - label_fields=['category_ids'] + format_bbox = 'pascal_voc' + label_fields = ['category_ids'] try: config = utilities.read_config(config_path) augment_config = config["train" if augment else "validation"]["augmentations"] except KeyError as e: raise ValueError("Missing key in the configuration file: {}".format(e)) - + if not augment_config: - default_augmentations=config["train" if augment else "validation"]["default_augmentations"] + default_augmentations = config["train" if augment else "validation"][ + "default_augmentations"] for augment in default_augmentations: - augment_config_type=augment.get('type') + augment_config_type = augment.get('type') params = augment.get('params', {}) - - if augment_config_type.lower()=='horizontalflip': + + if augment_config_type.lower() == 'horizontalflip': augmentations.append(A.HorizontalFlip(**params)) - - elif augment_config_type.lower()=='verticalflip': + + elif augment_config_type.lower() == 'verticalflip': augmentations.append(A.VerticalFlip(**params)) - - elif augment_config_type.lower()=='randomrotate90': + + elif augment_config_type.lower() == 'randomrotate90': augmentations.append(A.RandomRotate90(**params)) - - elif augment_config_type.lower()=='randomcrop': + + elif augment_config_type.lower() == 'randomcrop': augmentations.append(A.RandomCrop(**params)) - - elif augment_config_type.lower()=='randombrightnesscontrast': + + elif augment_config_type.lower() == 'randombrightnesscontrast': augmentations.append(A.RandomBrightnessContrast(**params)) - - elif augment_config_type.lower()=='normalize': + + elif augment_config_type.lower() == 'normalize': augmentations.append(A.Normalize(**params)) - - elif augment_config_type.lower()=='totensorv2': + + elif augment_config_type.lower() == 'totensorv2': augmentations.append(ToTensorV2()) - - elif augment_config_type.lower()=='bbox': - format_bbox=params['format'] - label_fields=params['label_fields'] - - transform = A.Compose( - augmentations, - bbox_params=A.BboxParams(format=format_bbox, label_fields=label_fields) - ) - + + elif augment_config_type.lower() == 'bbox': + format_bbox = params['format'] + label_fields = params['label_fields'] + + transform = A.Compose(augmentations, + bbox_params=A.BboxParams(format=format_bbox, + label_fields=label_fields)) + return transform - + for augment in augment_config: - augment_config_type=augment.get('type') + augment_config_type = augment.get('type') params = augment.get('params', {}) if augment_config_type.lower() == "randomsizedbboxsafecrop": try: augmentations.append(A.RandomSizedBBoxSafeCrop(**params)) except KeyError as e: - raise ValueError("Missing parameter for RandomSizedBBoxSafeCrop: {}".format(e)) - + raise ValueError( + "Missing parameter for RandomSizedBBoxSafeCrop: {}".format(e)) + elif augment_config_type.lower() == "padifneeded": min_height = params.get('min_height') pad_height_divisor = params.get('pad_height_divisor', 'None') - + try: min_width = params['min_width'] position = params.get('position', 'top_left') @@ -81,65 +83,59 @@ def get_augmentations(augment=True): p = params.get('p', 1.0) except KeyError as e: raise ValueError("Missing parameter for PadIfNeeded: {}".format(e)) - + if min_height == 'None' and pad_height_divisor != 'None': - augmentations.append(A.PadIfNeeded( - min_height=None, - min_width=min_width, - pad_height_divisor=pad_height_divisor, - position=position, - border_mode=border_mode, - value=value, - mask_value=mask_value, - always_apply=always_apply, - p=p)) + augmentations.append( + A.PadIfNeeded(min_height=None, + min_width=min_width, + pad_height_divisor=pad_height_divisor, + position=position, + border_mode=border_mode, + value=value, + mask_value=mask_value, + always_apply=always_apply, + p=p)) elif min_height != "None" and pad_height_divisor == "None": - augmentations.append(A.PadIfNeeded( - min_height=min_height, - min_width=min_width, - position=position, - border_mode=border_mode, - value=value, - mask_value=mask_value, - always_apply=always_apply, - p=p)) - + augmentations.append( + A.PadIfNeeded(min_height=min_height, + min_width=min_width, + position=position, + border_mode=border_mode, + value=value, + mask_value=mask_value, + always_apply=always_apply, + p=p)) + elif augment_config_type.lower() == "totensorv2": augmentations.append(ToTensorV2()) - - elif augment_config_type.lower()=='horizontalflip': - augmentations.append(A.HorizontalFlip(**params)) - - elif augment_config_type.lower()=='verticalflip': + + elif augment_config_type.lower() == 'horizontalflip': + augmentations.append(A.HorizontalFlip(**params)) + + elif augment_config_type.lower() == 'verticalflip': augmentations.append(A.VerticalFlip(**params)) - - elif augment_config_type.lower()=='randomrotate90': + + elif augment_config_type.lower() == 'randomrotate90': augmentations.append(A.RandomRotate90(**params)) - - elif augment_config_type.lower()=='randomcrop': + + elif augment_config_type.lower() == 'randomcrop': augmentations.append(A.RandomCrop(**params)) - - elif augment_config_type.lower()=='randombrightnesscontrast': + + elif augment_config_type.lower() == 'randombrightnesscontrast': augmentations.append(A.RandomBrightnessContrast(**params)) - - elif augment_config_type.lower()=='normalize': + + elif augment_config_type.lower() == 'normalize': augmentations.append(A.Normalize(**params)) - - elif augment_config_type.lower()=='totensorv2': + + elif augment_config_type.lower() == 'totensorv2': augmentations.append(ToTensorV2()) - - elif augment_config_type.lower()=='bbox': - format_bbox=params['format'] - label_fields=params['label_fields'] - - transform = A.Compose( - augmentations, - bbox_params=A.BboxParams(format=format_bbox, label_fields=label_fields) - ) - - return transform -if __name__ == "__main__": - augmentations = get_augmentations() - print("Augmentations created:", augmentations) + elif augment_config_type.lower() == 'bbox': + format_bbox = params['format'] + label_fields = params['label_fields'] + + transform = A.Compose(augmentations, + bbox_params=A.BboxParams(format=format_bbox, + label_fields=label_fields)) + return transform From 047f29e2582a89a35d0405e4e5229e5e49b3f841 Mon Sep 17 00:00:00 2001 From: Om Doiphode Date: Wed, 14 Aug 2024 17:50:19 +0530 Subject: [PATCH 4/4] Fixed doc issue --- docs/augmentations.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/augmentations.md b/docs/augmentations.md index 5bc13a1c..b34e33f2 100644 --- a/docs/augmentations.md +++ b/docs/augmentations.md @@ -180,5 +180,3 @@ For a complete list of supported augmentations, refer to the [Albumentations doc ## Conclusion Customizing data augmentations in DeepForest is simple and flexible. By defining your desired augmentations in the `deepforest_config.yml` file and passing them to the model using `get_augmentations`, you can easily tailor the training and validation processes to suit your specific needs. - ---- \ No newline at end of file