Skip to content

Commit

Permalink
add further benchmarks
Browse files Browse the repository at this point in the history
  • Loading branch information
@felixdittrich92
felixdittrich92 committed Aug 15, 2023
1 parent 74d2af0 commit 299b294
Show file tree
Hide file tree
Showing 3 changed files with 31 additions and 3 deletions.
32 changes: 29 additions & 3 deletions docs/source/using_doctr/using_models.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ For a comprehensive comparison, we have compiled a detailed benchmark on publicl
+----------------+---------------------------------+-----------------+--------------+------------+---------------+------------+---------------+--------------------+
| Tensorflow | master | (32, 128, 3) | 58.8 M | 87.44 | 88.21 | 93.83 | 94.25 | 22.3 |
+----------------+---------------------------------+-----------------+--------------+------------+---------------+------------+---------------+--------------------+
| TensorFlow | sar_resnet31 | (32, 128, 3) | 57.2 M | | | | | 7.1 |
| TensorFlow | sar_resnet31 | (32, 128, 3) | 57.2 M | 87.67 | 88.48 | 94.21 | 94.66 | 7.1 |
+----------------+---------------------------------+-----------------+--------------+------------+---------------+------------+---------------+--------------------+
| Tensorflow | vitstr_small | (32, 128, 3) | 21.4 M | 83.01 | 83.84 | 86.57 | 87.00 | 2.0 |
+----------------+---------------------------------+-----------------+--------------+------------+---------------+------------+---------------+--------------------+
Expand All @@ -153,9 +153,9 @@ For a comprehensive comparison, we have compiled a detailed benchmark on publicl
+----------------+---------------------------------+-----------------+--------------+------------+---------------+------------+---------------+--------------------+
| PyTorch | crnn_vgg16_bn | (32, 128, 3) | 15.8 M | 86.54 | 87.41 | 94.29 | 94.69 | 0.6 |
+----------------+---------------------------------+-----------------+--------------+------------+---------------+------------+---------------+--------------------+
| PyTorch | crnn_mobilenet_v3_small | (32, 128, 3) | 4.5 M | 87.25 | 87.99 | 93.91 | 94.34 | 0.05 |
| PyTorch | crnn_mobilenet_v3_small | (32, 128, 3) | 2.1 M | 87.25 | 87.99 | 93.91 | 94.34 | 0.05 |
+----------------+---------------------------------+-----------------+--------------+------------+---------------+------------+---------------+--------------------+
| PyTorch | crnn_mobilenet_v3_large | (32, 128, 3) | 2.1 M | 87.38 | 88.09 | 94.46 | 94.92 | 0.08 |
| PyTorch | crnn_mobilenet_v3_large | (32, 128, 3) | 4.5 M | 87.38 | 88.09 | 94.46 | 94.92 | 0.08 |
+----------------+---------------------------------+-----------------+--------------+------------+---------------+------------+---------------+--------------------+
| PyTorch | master | (32, 128, 3) | 58.7 M | | | | | 17.6 |
+----------------+---------------------------------+-----------------+--------------+------------+---------------+------------+---------------+--------------------+
Expand Down Expand Up @@ -218,10 +218,36 @@ For a comprehensive comparison, we have compiled a detailed benchmark on publicl
+----------------+----------------------------------------------------------+------------+---------------+------------+---------------+
| TensorFlow | db_resnet50 + crnn_vgg16_bn | 70.82 | 75.56 | 83.97 | 81.40 |
+----------------+----------------------------------------------------------+------------+---------------+------------+---------------+
| TensorFlow | db_resnet50 + crnn_mobilenet_v3_small | 69.63 | 74.29 | 81.08 | 78.59 |
+----------------+----------------------------------------------------------+------------+---------------+------------+---------------+
| TensorFlow | db_resnet50 + crnn_mobilenet_v3_large | | | | |
+----------------+----------------------------------------------------------+------------+---------------+------------+---------------+
| TensorFlow | db_resnet50 + sar_resnet31 | 69.42 | 74.04 | 80.67 | 78.21 |
+----------------+----------------------------------------------------------+------------+---------------+------------+---------------+
| TensorFlow | db_resnet50 + master | 68.75 | 73.76 | 78.56 | 76.24 |
+----------------+----------------------------------------------------------+------------+---------------+------------+---------------+
| TensorFlow | db_resnet50 + vitstr_small | 64.58 | 68.91 | 74.66 | 72.37 |
+----------------+----------------------------------------------------------+------------+---------------+------------+---------------+
| TensorFlow | db_resnet50 + vitstr_base | | | | |
+----------------+----------------------------------------------------------+------------+---------------+------------+---------------+
| TensorFlow | db_resnet50 + parseq | | | | |
+----------------+----------------------------------------------------------+------------+---------------+------------+---------------+
| PyTorch | db_resnet50 + crnn_vgg16_bn | 67.82 | 73.35 | 84.84 | 83.27 |
+----------------+----------------------------------------------------------+------------+---------------+------------+---------------+
| PyTorch | db_resnet50 + crnn_mobilenet_v3_small | 67.89 | 74.01 | 84.43 | 82.85 |
+----------------+----------------------------------------------------------+------------+---------------+------------+---------------+
| PyTorch | db_resnet50 + crnn_mobilenet_v3_large | 68.45 | 74.63 | 84.86 | 83.27 |
+----------------+----------------------------------------------------------+------------+---------------+------------+---------------+
| PyTorch | db_resnet50 + sar_resnet31 | | | | |
+----------------+----------------------------------------------------------+------------+---------------+------------+---------------+
| PyTorch | db_resnet50 + master | | | | |
+----------------+----------------------------------------------------------+------------+---------------+------------+---------------+
| PyTorch | db_resnet50 + vitstr_small | | | | |
+----------------+----------------------------------------------------------+------------+---------------+------------+---------------+
| PyTorch | db_resnet50 + vitstr_base | | | | |
+----------------+----------------------------------------------------------+------------+---------------+------------+---------------+
| PyTorch | db_resnet50 + parseq | | | | |
+----------------+----------------------------------------------------------+------------+---------------+------------+---------------+
| None | Gvision text detection | 59.50 | 62.50 | 75.30 | 59.03 |
+----------------+----------------------------------------------------------+------------+---------------+------------+---------------+
| None | Gvision doc. text detection | 64.00 | 53.30 | 68.90 | 61.10 |
Expand Down
1 change: 1 addition & 0 deletions scripts/evaluate.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ def main(args):
args.recognition,
pretrained=True,
reco_bs=args.batch_size,
preserve_aspect_ratio=False,
assume_straight_pages=not args.rotation,
)

Expand Down
1 change: 1 addition & 0 deletions scripts/evaluate_kie.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ def main(args):
args.recognition,
pretrained=True,
reco_bs=args.batch_size,
preserve_aspect_ratio=False,
assume_straight_pages=not args.rotation,
)

Expand Down

0 comments on commit 299b294

Please sign in to comment.