DALI v1.5.0
Key Features and Enhancements
This DALI release includes the following key features and enhancements.
- Extended
decoders.image
to support WebP decoding (#3206) - Added indexing (NumPy-like) API for tensor slicing (#3200 and #3195)
- Extended
external_source
to supportsource
argument in TensorFlow DALI Dataset (#3215, #3193, #3177 and #3176) - Added examples:
Fixed issues
This DALI release includes the following fixes:
- Fixed include paths that prevented including some parts of DALI in other C/C++ projects (#3210)
- Fixed a crash when only anchors and no shapes were provided in
multi_paste
(#3166) - In the
spectrogram
operator, extracted windows are now correctly centered before FFT calculation, when thenfft
argument is bigger than length of the window. (#3180) - Fixed a minor memory leak in
decoders.image
(#3148)
Improvements
- Add documentation for indexing. (#3200)
- Move to CUDA 11.4U1 (#3213)
- Add WebP support to image decoder (#3206)
- libtar API implementation (#3198)
- Tensor indexing (#3195)
- Make TF graph-mode tests faster (#3204)
- Add support for ES
source
in TF DALI Dataset (#3177) - Add tensorflow YOLOv4 example (#2883)
- Refactor Python External Source code (#3176)
- Update third party dependencies to latest release versions (#3184)
- Add deferred deallocation to
cuda_vm_resource
. (#3154) - Adjust test scripts and section header for webadataset notebook (#3162)
- Add Webdataset-ExternalSource Jupyter notebook (#3153)
- Update PR template (#3150)
- Update PR template (#3129)
Bug Fixes
- Fix failing TarArchive tests (#3226)
- Build custom libtar in conda (#3223)
- Improve validation in DALIDataset (#3215)
- Update DALI_DEPS_VERSIOn to include NVIDIA/DALI_deps#19 (#3224)
- Fix identity check in _is_generator_function which. Add test. (#3216)
- Fix unused imports in test_utils.py (#3214)
- Remove the usage of ManagedMemory from the OpticalFlow tests (#3211)
- Suppress test using unified memory when it is not supported (#3209)
- Remove include prefix from include paths (#3210)
- Fix CVE-2021-3246 in libsnd (#3208)
- Fix pytorch-lighting test (#3196)
- Fix coverity issues + skip tests involving managed memory when not supported. (#3190)
- Disable NVJPEG HW decoder for driver < 455 due to performance reason (#3189)
- Fix compilation with newer GCC (#3188)
- Disallow some types of sources for parallel ES explicitly (#3193)
- Center windows when extracting windows to a bigger output window (#3180)
- Add a compute cap value before running the GDS test (#3185)
- MultiPaste to adjust the region shape to cover up to the end of the input shape (#3166)
- Fix wording in docs (#3165)
- Fix image decode (#3148)
- Fix LastBatchPolicy doc and update Parallel ES wording (#3152)
- Fix some errors (#3147)
Breaking API changes
There are no breaking changes in this DALI release.
Deprecated features
There are no deprecated features in this DALI release.
Known issues:
- The video loader operator requires that the key frames occur at a minimum every 10 to 15 frames of the video stream. If the key frames occur at a lesser frequency, then the returned frames may be out of sync.
- The DALI TensorFlow plugin might not be compatible with TensorFlow versions 1.15.0 and later.
To use DALI with the TensorFlow version that does not have a prebuilt plugin binary shipped with DALI, make sure that the compiler that is used to build TensorFlow exists on the system during the plugin installation. (Depending on the particular version, use GCC 4.8.4, GCC 4.8.5, or GCC 5.4.) - Due to some known issues with meltdown/spectra mitigations and DALI, DALI shows best performance when run in Docker with escalated privileges, for example:
- privileged=yes in Extra Settings for AWS data points
- --privileged or --security-opt seccomp=unconfined for bare Docker
Binary builds
Install via pip for CUDA 10.2:
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-cuda102==1.5.0
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-tf-plugin-cuda102==1.5.0
or for CUDA 11:
CUDA 11.0 build uses CUDA toolkit enhanced compatibility. It is built with the latest CUDA 11.x toolkit
while it can run on the latest, stable CUDA 11.0 capable drivers (450.80 or later).
Using the latest driver may enable additional functionality.
More details can be found in enhanced CUDA compatibility guide.
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-cuda110==1.5.0
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-tf-plugin-cuda110==1.5.0
Or use direct download links (CUDA 10.2):
- https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda102/nvidia_dali_cuda102-1.5.0-2725759-py3-none-manylinux2014_x86_64.whl
- https://developer.download.nvidia.com/compute/redist/nvidia-dali-tf-plugin-cuda102/nvidia-dali-tf-plugin-cuda102-1.5.0.tar.gz
Or use direct download links (CUDA 11.0):
- https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda110/nvidia_dali_cuda110-1.5.0-2725760-py3-none-manylinux2014_x86_64.whl
- https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda110/nvidia_dali_cuda110-1.5.0-2725760-py3-none-manylinux2014_aarch64.whl
- https://developer.download.nvidia.com/compute/redist/nvidia-dali-tf-plugin-cuda110/nvidia-dali-tf-plugin-cuda110-1.5.0.tar.gz
FFmpeg source code:
Libsndfile source code: