Deepstream Inference

deploy/Deepstream/README.md

@@ -0,0 +1,63 @@
+# YOLOv6 in Deepstream
+
+## Dependencies
+- Deepstream6.0
+- TensorRT-8.0
+
+## Step 1: Get the TensorRT 
+Please Follow the file [TensorRT README](../TensorRT/README.md)  to get TensorRT engine file `yolov6n.trt`.
+
+```shell
+python ./deploy/ONNX/export_onnx.py \
+    --weights yolov6n.pt \
+    --img 640 \
+    --batch 1
+```
+
+```shell
+python3 onnx_to_tensorrt.py --fp16 --int8 -v \
+        --max_calibration_size=${MAX_CALIBRATION_SIZE} \
+        --calibration-data=${CALIBRATION_DATA} \
+        --calibration-cache=${CACHE_FILENAME} \
+        --preprocess_func=${PREPROCESS_FUNC} \
+        --explicit-batch \
+        --onnx ${ONNX_MODEL} -o ${OUTPUT}
+```
+### Example
+```shell
+python3 onnx_to_tensorrt.py --fp16 --onnx ${ONNX_MODEL} -o ${OUTPUT}
+```
+
+## Step 2: Build the So file 
+Execute the following Command, get the libnvdsinfer_custom_impl_Yolov6.so file.
+```shell
+cd nvdsparsebbox_YoloV6
+export CUDA_VER=11.4  # for dGPU  
+make
+```
+
+## Step 3: Run the Demo
+```shell
+mv yolov6n.trt Deepstream # move yolov6n.trt file to Deepstream folder
+deepstream-app -c ds_app_config_yoloV6.txt 
+```
+
+## Additional Experimental Results   
+
+**The TensorRT performance of Yolov6n was tested on COCO2017 val datasets, as shown below.**
+
+---------------------
+|Yolov6N(640) FP16 |     AP       | area | maxDet |
+|-----------------|------------------------|-------------|-----------------------|
+|Average Precision|  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.357 |
+|Average Precision|  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.512 |
+|Average Precision|  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.383 |
+|Average Precision|  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.170 |
+|Average Precision|  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.398 |
+|Average Precision|  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.511 |
+|Average Recall   |  (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.299 |
+|Average Recall   |  (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.490 |
+|Average Recall   |  (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.538 |
+|Average Recall   |  (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.296 |
+|Average Recall   |  (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.605 |
+|Average Recall   |  (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.729 |

deploy/Deepstream/config_infer_prinary_yoloV6.txt

@@ -0,0 +1,94 @@
+################################################################################
+# Copyright (c) 2019-2021, NVIDIA CORPORATION. All rights reserved.
+#
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+# DEALINGS IN THE SOFTWARE.
+################################################################################
+
+# Following properties are mandatory when engine files are not specified:
+#   int8-calib-file(Only in INT8), model-file-format
+#   Caffemodel mandatory properties: model-file, proto-file, output-blob-names
+#   UFF: uff-file, input-dims, uff-input-blob-name, output-blob-names
+#   ONNX: onnx-file
+#
+# Mandatory properties for detectors:
+#   num-detected-classes
+#
+# Optional properties for detectors:
+#   cluster-mode(Default=Group Rectangles), interval(Primary mode only, Default=0)
+#   custom-lib-path
+#   parse-bbox-func-name
+#
+# Mandatory properties for classifiers:
+#   classifier-threshold, is-classifier
+#
+# Optional properties for classifiers:
+#   classifier-async-mode(Secondary mode only, Default=false)
+#
+# Optional properties in secondary mode:
+#   operate-on-gie-id(Default=0), operate-on-class-ids(Defaults to all classes),
+#   input-object-min-width, input-object-min-height, input-object-max-width,
+#   input-object-max-height
+#
+# Following properties are always recommended:
+#   batch-size(Default=1)
+#
+# Other optional properties:
+#   net-scale-factor(Default=1), network-mode(Default=0 i.e FP32),
+#   model-color-format(Default=0 i.e. RGB) model-engine-file, labelfile-path,
+#   mean-file, gie-unique-id(Default=0), offsets, process-mode (Default=1 i.e. primary),
+#   custom-lib-path, network-mode(Default=0 i.e FP32)
+#
+# The values in the config file are overridden by values set through GObject
+# properties.
+
+[property]
+gpu-id=0
+net-scale-factor=0.0039215697906911373
+maintain-aspect-ratio=1
+symmetric-padding=1
+scaling-filter=1
+scaling-compute-hw=0
+#0=RGB, 1=BGR
+model-color-format=0
+
+model-engine-file=yolov6n.trt
+force-implicit-batch-dim=1
+batch-size=1
+
+labelfile-path=labels.txt
+# int8-calib-file=yolov3-calibration.table.trt7.0
+## 0=FP32, 1=INT8, 2=FP16 mode
+network-mode=2
+num-detected-classes=80
+gie-unique-id=1
+# Integer 0: Detector  1: Classifier
+network-type=0
+# is-classifier=0
+## 1=DBSCAN, 2=NMS, 3= DBSCAN+NMS Hybrid, 4 = None(No clustering)
+cluster-mode=4
+# lib path
+parse-bbox-func-name=NvDsInferParseCustomYoloV6
+custom-lib-path=nvdsparsebbox_YoloV6/libnvdsinfer_custom_impl_Yolov6.so
+# engine-create-func-name=NvDsInferYoloCudaEngineGet
+#scaling-filter=0
+#scaling-compute-hw=0
+
+[class-attrs-all]
+nms-iou-threshold=0.3
+pre-cluster-threshold=0.7

deploy/Deepstream/ds_app_config_yoloV6.txt

@@ -0,0 +1,136 @@
+################################################################################
+# Copyright (c) 2019-2020, NVIDIA CORPORATION. All rights reserved.
+#
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+# DEALINGS IN THE SOFTWARE.
+################################################################################
+
+[application]
+enable-perf-measurement=1
+perf-measurement-interval-sec=5
+#gie-kitti-output-dir=streamscl
+
+[tiled-display]
+enable=1
+rows=1
+columns=1
+width=1920
+height=1080
+gpu-id=0
+#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
+#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla
+#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla
+#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla
+#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
+nvbuf-memory-type=0
+
+[source0]
+enable=1
+#Type - 1=CameraV4L2 2=URI 3=MultiURI
+type=3
+uri=file://./sample_1080p_h264.mp4
+num-sources=1
+gpu-id=0
+# (0): memtype_device   - Memory type Device
+# (1): memtype_pinned   - Memory type Host Pinned
+# (2): memtype_unified  - Memory type Unified
+cudadec-memtype=0
+
+[sink0]
+enable=1
+#Type - 1=FakeSink 2=EglSink 3=File
+type=3
+sync=0
+source-id=0
+gpu-id=0
+nvbuf-memory-type=0
+
+container=1
+codec=1
+bitrate=4000000
+iframeinterval=30
+output-file=output_yolov6.mp4
+
+[osd]
+enable=1
+gpu-id=0
+border-width=1
+text-size=15
+text-color=1;1;1;1;
+text-bg-color=0.3;0.3;0.3;1
+font=Serif
+show-clock=0
+clock-x-offset=800
+clock-y-offset=820
+clock-text-size=12
+clock-color=1;0;0;0
+nvbuf-memory-type=0
+
+[streammux]
+gpu-id=0
+##Boolean property to inform muxer that sources are live
+live-source=0
+batch-size=1
+##time out in usec, to wait after the first buffer is available
+##to push the batch even if the complete batch is not formed
+batched-push-timeout=40000
+## Set muxer output width and height
+width=1920
+height=1080
+##Enable to maintain aspect ratio wrt source, and allow black borders, works
+##along with width, height properties
+enable-padding=0
+nvbuf-memory-type=0
+
+# config-file property is mandatory for any gie section.
+# Other properties are optional and if set will override the properties set in
+# the infer config file.
+[primary-gie]
+enable=1
+gpu-id=0
+#model-engine-file=model_b1_gpu0_int8.engine
+labelfile-path=labels.txt
+batch-size=1
+#Required by the app for OSD, not a plugin property
+bbox-border-color0=1;0;0;1
+bbox-border-color1=0;1;1;1
+bbox-border-color2=0;0;1;1
+bbox-border-color3=0;1;0;1
+interval=2
+gie-unique-id=1
+nvbuf-memory-type=0
+config-file=config_infer_prinary_yoloV6.txt
+
+[tracker]
+enable=1
+# For NvDCF and DeepSORT tracker, tracker-width and tracker-height must be a multiple of 32, respectively
+tracker-width=640
+tracker-height=384
+ll-lib-file=/opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_nvmultiobjecttracker.so
+# ll-config-file required to set different tracker types
+# ll-config-file=/opt/nvidia/deepstream/deepstream-6.0//samples/configs/deepstream-app/config_tracker_IOU.yml
+ll-config-file=/opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml
+# ll-config-file=/opt/nvidia/deepstream/deepstream-6.0//samples/configs/deepstream-app/config_tracker_NvDCF_accuracy.yml
+# ll-config-file=/opt/nvidia/deepstream/deepstream-6.0//samples/configs/deepstream-app/config_tracker_DeepSORT.yml
+gpu-id=0
+enable-batch-process=1
+enable-past-frame=1
+display-tracking-id=1
+
+[tests]
+file-loop=0

deploy/Deepstream/labels.txt

@@ -0,0 +1,80 @@
+person
+bicycle
+car
+motorbike
+aeroplane
+bus
+train
+truck
+boat
+traffic light
+fire hydrant
+stop sign
+parking meter
+bench
+bird
+cat
+dog
+horse
+sheep
+cow
+elephant
+bear
+zebra
+giraffe
+backpack
+umbrella
+handbag
+tie
+suitcase
+frisbee
+skis
+snowboard
+sports ball
+kite
+baseball bat
+baseball glove
+skateboard
+surfboard
+tennis racket
+bottle
+wine glass
+cup
+fork
+knife
+spoon
+bowl
+banana
+apple
+sandwich
+orange
+broccoli
+carrot
+hot dog
+pizza
+donut
+cake
+chair
+sofa
+pottedplant
+bed
+diningtable
+toilet
+tvmonitor
+laptop
+mouse
+remote
+keyboard
+cell phone
+microwave
+oven
+toaster
+sink
+refrigerator
+book
+clock
+vase
+scissors
+teddy bear
+hair drier
+toothbrush