Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add filter_boxes method #332

Merged
merged 8 commits into from
Jul 1, 2021
Merged

feat: add filter_boxes method #332

merged 8 commits into from
Jul 1, 2021

Conversation

charlesmindee
Copy link
Collaborator

@charlesmindee charlesmindee commented Jun 28, 2021

This PR adds a filter_boxes method in the core DetectionPostProcessor which performs the following:

  • NMS with a high threshold to eliminate boxes included in others. NMS is implemented in Numpy (Fast-RCNN implementation).
  • Merging for boxes with a consequent IOU, to avoid overlapping of letters which would appears in both words.

For now, only straight bounding boxes are supported, as the computation of NMS/boxes to merge is slightly harder when dealing with rotated boxes. In case of rotated boxes, it will simply return the array of boxes without filtering.

Any feeback is welcome!

@charlesmindee charlesmindee added type: enhancement Improvement module: models Related to doctr.models labels Jun 28, 2021
@charlesmindee charlesmindee added this to the 0.3.0 milestone Jun 28, 2021
@charlesmindee charlesmindee self-assigned this Jun 28, 2021
@codecov
Copy link

codecov bot commented Jun 28, 2021

Codecov Report

Merging #332 (400315c) into main (9bc289b) will increase coverage by 0.03%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #332      +/-   ##
==========================================
+ Coverage   94.69%   94.72%   +0.03%     
==========================================
  Files          83       83              
  Lines        3296     3335      +39     
==========================================
+ Hits         3121     3159      +38     
- Misses        175      176       +1     
Flag Coverage Δ
unittests 94.72% <100.00%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
doctr/documents/elements.py 98.93% <100.00%> (ø)
doctr/utils/geometry.py 100.00% <100.00%> (ø)
doctr/utils/metrics.py 96.59% <100.00%> (+0.87%) ⬆️
...dels/detection/differentiable_binarization/base.py 91.08% <0.00%> (-0.64%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9bc289b...400315c. Read the comment docs.

Copy link
Contributor

@fg-mindee fg-mindee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! I added a few improvement suggestions

doctr/models/detection/core.py Outdated Show resolved Hide resolved
doctr/models/detection/core.py Outdated Show resolved Hide resolved
doctr/models/detection/core.py Outdated Show resolved Hide resolved
doctr/models/detection/core.py Outdated Show resolved Hide resolved
@fg-mindee
Copy link
Contributor

Also, this PR modifies the default output behaviour (cf the failed unittest in the API)

@charlesmindee
Copy link
Collaborator Author

Changes:

  • nms is now implemented in metrics, but not used here.
  • the filter_boxes method uses box_ioa (which is implemented in metrics) to merge boxes above a threshold. This will remove boxes included in others or merge those who overlap significantly.
  • a resolve_enclosing_boxarray function is implemented in geometry.
  • added corresponding tests.

fg-mindee
fg-mindee previously approved these changes Jun 29, 2021
Copy link
Contributor

@fg-mindee fg-mindee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks!

For further reference, since this changes the default behaviour of the detection part, should we update the benchmark tables as well? (to ensure it's a positive change)

doctr/utils/geometry.py Outdated Show resolved Hide resolved
doctr/utils/geometry.py Outdated Show resolved Hide resolved
Copy link
Contributor

@fg-mindee fg-mindee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I had another thought of merging both bbox resolution function since they indeed have the same purpose. The rest looks good to me!

doctr/utils/geometry.py Outdated Show resolved Hide resolved
test/test_utils_geometry.py Outdated Show resolved Hide resolved
Copy link
Contributor

@fg-mindee fg-mindee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot!

@charlesmindee charlesmindee merged commit fb8f21b into main Jul 1, 2021
@charlesmindee charlesmindee deleted the nms branch July 1, 2021 13:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module: models Related to doctr.models type: enhancement Improvement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants