Skip to content

Releases: NVIDIA/DALI

DALI v0.27.0

29 Oct 16:32
@klecki klecki
v0.27.0
511c22e
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
DALI v0.27.0

Key Features and Enhancements

This DALI release includes the following key features and enhancements.

  • New operators:
    • CoordTransform Operator for applying a linear transformation to points or vectors (#2288)
    • GaussianBlur Gpu Operator (#2314, #2311, #2254)
    • Nemo ASR Reader (#2234)
    • Resize 3D - operator can now process 3D inputs (#2226)
    • Add Translate affine transform generator (#2297) - in the next release it will be moved to a dedicated module.
  • Use true scalars (except in classification readers) - 0-dim Tensors represent scalar values (#2318)
  • Adjust documentation after review (#2175)
  • Support for ZSTD compression for TIFF files (#2273)
  • Support for Run-Length Encodings and Pixelwise Masks in COCO Reader (#2248)
  • Support more types in Lookup table (#2290)

Bug fixes

  • Fixes crash in RandomBBoxCrop when no labels are provided (#2265)
  • Fix minor issues reported by static analysis (#2276)
  • Fix detection pipeline test on Ampere (#2304)
  • Fix BUILD_LIBSND=OFF build (#2316)
  • Fix build for LMDB disabled (#2319)

Improvements

  • Update build and test deps to the latest version (#2250)
  • Resize 3D + resize tests (#2226)
  • Allow passing a <= 0 values in the file list to allow more flexible frame indexing (#2264)
  • Extend host decoder to support jpeg2000 (#2270)
  • Add file_list argument support to the Numpy reader operator (#2274)
  • Allow Slice to silently assume absolute anchor and shape when those are represented by an integer (#2282)
  • TransformPoints kernel (#2287)
  • Add inline to LookaheadParser methods (#2289)
  • Add deprecation handling in backend (#2279)
  • Support more types in Lookup table (#2290)
  • Adjust documentation after review (#2175)
  • Transform points op (#2288)
  • Support for ZSTD compression for TIFF files (#2273)
  • Support for Run-Length Encodings and Pixelwise Masks in COCO Reader (#2248)
  • Extract a DecodeAudio implementation from Audio decoder operator (#2294)
  • Extend test_RN50_data_pipeline.py test (#2295)
  • Add ConvolutionGPU kernel based on CUTLASS (#2254)
  • Add Translate affine transform generator (#2297)
  • Add *.cuh and *.inl to list of headers to bundle (#2307)
  • Add Nemo ASR reader (#2234)
  • Add SeprableConvolutionGPU kernel (#2311)
  • Add GaussianBlur Gpu Operator (#2314)
  • Use true scalars (except in classification readers) + bug fixes (#2318)
  • Add nvjpeg2k support to GPU Image Decoder. Extend nvjpeg memory pool to support nvjpeg2k allocators.
  • Adds a separate option to preallocate nvjPEG2k memory (#2347)
  • Due to some decoding problems disable nvJPEG2K support for now by the default

Breaking API changes

Deprecated feature

Known issues:

  • The video loader operator requires that the key frames occur at a minimum every 10 to 15 frames of the video stream. If the key frames occur at a lesser frequency, then the returned frames may be out of sync.
  • The DALI TensorFlow plugin might not be compatible with TensorFlow versions 1.15.0 and later.
    To use DALI with the TensorFlow version that does not have a prebuilt plugin binary shipped with DALI, make sure that the compiler that is used to build TensorFlow exists on the system during the plugin installation. (Depending on the particular version, use GCC 4.8.4, GCC 4.8.5, or GCC 5.4.)
  • Due to some known issues with meltdown/spectra mitigations and DALI, DALI shows best performance when run in Docker with escalated privileges, for example:
    • privileged=yes in Extra Settings for AWS data points
    • --privileged or --security-opt seccomp=unconfined for bare Docker

Binary builds

Install via pip for CUDA 10
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-cuda100==0.27.0
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-tf-plugin-cuda100==0.27.0

or for CUDA 11:
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-cuda110==0.27.0
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-tf-plugin-cuda110==0.27.0

Or use direct download links (CUDA 10.0):

Or use direct download links (CUDA 11.0):

FFmpeg source code:

  • This software uses code of FFmpeg licensed under the LGPLv2.1 and its source can be downloaded here

Libsndfile source code:

Assets 2
Loading

DALI 0.25.1

11 Sep 11:22
@klecki klecki
v0.25.1
db2c06f
Compare
Choose a tag to compare
DALI 0.25.1

Key Features and Enhancements

This is a patch release that contains only fixes.

Bug fixes

  • Fixed a crash that occurred when DALI CUDA 11 runs on pre 450.x driver with the compatibility layer (#2208, #2230).

Known issues

  • The video loader operator requires that the key frames occur at a minimum every 10 to 15 frames of the video stream. If the key frames occur at a lesser frequency, then the returned frames may be out of sync.
  • The DALI TensorFlow plugin might not be compatible with TensorFlow versions 1.15.0 and later.
    To use DALI with the TensorFlow version that does not have a prebuilt plugin binary shipped with DALI, make sure that the compiler that is used to build TensorFlow exists on the system during the plugin installation. (Depending on the particular version, use GCC 4.8.4, GCC 4.8.5, or GCC 5.4.)
  • Due to some known issues with meltdown/spectra mitigations and DALI, DALI shows best performance when run in Docker with escalated privileges, for example:
    • privileged=yes in Extra Settings for AWS data points
    • --privileged or --security-opt seccomp=unconfined for bare Docker

Binary builds

Install via pip for CUDA 10
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-cuda100==0.25.1
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-tf-plugin-cuda100==0.25.1

or for CUDA 11:
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-cuda110==0.25.1
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-tf-plugin-cuda110==0.25.1

Or use direct download links (CUDA 10.0):

Or use direct download links (CUDA 11.0):

SBSA aarch64 CUDA 11.0 direct download link:

FFmpeg source code:

  • This software uses code of FFmpeg licensed under the LGPLv2.1 and its source can be downloaded here

Libsndfile source code:

Assets 2
Loading

DALI v0.26.0

29 Sep 12:30
@klecki klecki
v0.26.0
1e91967
Compare
Choose a tag to compare
DALI v0.26.0

Key Features and Enhancements

This DALI release includes the following key features and enhancements.

  • New operators:
    • Add PeekShape operator to learn the decoded image shape (#2205)
  • Add an ability to run DALI without GPU (#2165)
  • Optimize single-channel audio resampling with SSE2 (#2240)
  • Add ability to pass DALI TensorList or a list of DALI Tensors to exernal source (#2244)
  • Enhance error messages in case of not supported data types in operators (#2211)
  • Add a more verbose message about unsupported videos (#2203)
  • Use copy kernel when making a contiguous batch during ShareUserData, if user requested it (#2200)

Bug fixes

  • Fix typo in VERSION
  • Fix lack of input type checking in GPU variant of Spectrogram operator (#2192)
  • Fix TensorListView::to_static (#2216)
  • Temporarily freeze protobuf packages versions in Conda (#2222)
  • Fix VideoReader error checking when opening files (#2223)
  • Fix NVTX annotations (#2215)
  • Fix docker/build.sh to use Python 3 for TF plugin (#2214)
  • Fix hw_decoder_load=0.0 for ImageDecoder related tests that require deterministic results (#2232)
  • Fix a memory leak in the audio decoder (#2235)
  • Fix for TF nightly container (#2236)
  • Fix wrong jupyter execution syntax (#2241)
  • Fix TL1_ssd_training test (#2243)

Improvements

  • Use copy kernel when making a contiguous batch during ShareUserData, if user requested it (#2200)
  • Update ExternalSource documentation (#2201)
  • Use NVCC to detect cuda release version (#2194)
  • DALI TF stop requiring DALI to be installed before build_ext step (#2204)
  • Add PeekShape operator to learn the decoded image shape (#2205)
  • Remove dummy package (#2207)
  • Add a more verbose message about unsupported videos (#2203)
  • Enhance error messages in case of not supported data types in operators (#2211)
  • Add more supported types to SliceBase (#2210)
  • Add an ability to run DALI without GPU (#2165)
  • Add CUTLASS to third party with an initial code layout (#2237)
  • Make the CUTLASS template files pass lint check (#2238)
  • Use SSE2 for single-channel audio resampling (#2240)
  • Add nvidia-tensorflow to DALI tests (#2075)
  • Update APEX version to the latest stable and tested version (#2246)
  • Fuzzing targets (#2219)
  • Add ability to pass DALI TensorList or a list of DALI Tensors to exernal source (#2244)
  • 3D resampling (#1489)
  • Skip VP9 tests instead of failing if codec is not supported. (#2251)

Breaking API changes

Deprecated feature

Known issues:

  • The video loader operator requires that the key frames occur at a minimum every 10 to 15 frames of the video stream. If the key frames occur at a lesser frequency, then the returned frames may be out of sync.
  • The DALI TensorFlow plugin might not be compatible with TensorFlow versions 1.15.0 and later.
    To use DALI with the TensorFlow version that does not have a prebuilt plugin binary shipped with DALI, make sure that the compiler that is used to build TensorFlow exists on the system during the plugin installation. (Depending on the particular version, use GCC 4.8.4, GCC 4.8.5, or GCC 5.4.)
  • Due to some known issues with meltdown/spectra mitigations and DALI, DALI shows best performance when run in Docker with escalated privileges, for example:
    • privileged=yes in Extra Settings for AWS data points
    • --privileged or --security-opt seccomp=unconfined for bare Docker

Binary builds

Install via pip for CUDA 10
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-cuda100==0.26.0
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-tf-plugin-cuda100==0.26.0

or for CUDA 11:
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-cuda110==0.26.0
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-tf-plugin-cuda110==0.26.0

Or use direct download links (CUDA 10.0):

Or use direct download links (CUDA 11.0):

FFmpeg source code:

  • This software uses code of FFmpeg licensed under the LGPLv2.1 and its source can be downloaded here

Libsndfile source code:

Assets 2
Loading

DALI v0.25.0

28 Aug 20:02
@klecki klecki
v0.25.0
5f0c12c
Compare
Choose a tag to compare
DALI v0.25.0

Key Features and Enhancements

This DALI release includes the following key features and enhancements.

  • Added support for aarch64 Server Base System Architecture (#2110) - we provide a build for CUDA 11 that can be installed following the installation guide.
  • New operators:
    • Normal Distribution GPU Operator (#2125)
    • Video reader resize (#2097)
  • Improvements to ExternalSource Op:
    • Added the no_copy option, which allows DALI to borrow a user's memory instead of copying it (#2024).
    • Removed the redundant copy in the ExternalSource operator (#2124)
  • Reworked the Resize operator family, including video, channel-first, RoI, and multiple-type support (#2164) with the new Resize tutorial (#2189).
  • Bundled all python versions into one wheel (#2096).
    • One DALI wheel can be used with all supported Python versions, including 3.5, 3.6, 3.7 and 3.8.
  • Improved error messages and added information about the Operator of origin (#2065).
  • Extended the following C APIs to copy output and input samples:
    • daliOutputCopy (#2145) and daliOutputCopySamples (#2161, #2186).
    • These APIs allow you to use the copy kernel and reduce the amount of copied memory and to use the copy kernel in ShareUserData (#2200).
  • Performance improvements:
    • Arithmetic Ops GPU (#2137)
    • Priorities in CPU thread pool allowing for better load balancing with uneven samples (#2092, #2102)

Bug fixes

  • Fix aarch64 builds that are still gcc 5.x based (#2099)
  • Fix conda build after the new build of libprotobuf was released (#2101)
  • Fix the lack of setting the right device in the ExternalSource (#2112)
  • Fix lack of a proper include to set CUDART_VERSION inside nvml.h and nvml_wrap.h (#2113)
  • Fix layout propagation in Gaussian Blur (#2118)
  • Fix layout propagation in Erase (#2133)
  • Fix TF dataset notebook (#2135)
  • Fix lack of MXNet plugin docs generation (#2146)
  • Fix TL3_RN50_convergence test for PaddlePaddle (#2159)
  • Workaround a bug in compiler, magically converting instance call to static call. (#2162)
  • Fix the need to have a numpy installed when test_utils.py is just imported (#2166)
  • Fix missing layouts in operators (#2136)
  • Fix QNX build (#2199)

Improvements

  • Update to CUDA 11 GA toolkit (#2094)
  • Allow nvJPEG to pre-allocate pinned and device buffers during construction (#2081)
  • Add zero-copy to the ExternalSource operator (#2024)
  • Introduce priorities in ThreadPool (#2092)
  • Video reader resize (#2097)
  • Detect version of CUDA based on libcudart.so.* name (#2105)
  • Add Operator origin information to most errors (#2065)
  • Enhance Pad documentation (#2098)
  • Bundle all python versions into one wheel (#2096)
  • Use new nvmlDeviceGetCpuAffinityWithinScope API for thread binding (#2093)
  • Use new ThreadPool API to post work with priority (#2102)
  • TensorListView generalized reshape and reinterpret (#2108)
  • Update aarch64_linux build to Jetpack 4.4 and CUDA 10.2 (#2107)
  • Renable VP9 video tests after driver update (#2117)
  • Remove usage of future from DALI (#2119)
  • Removes redundant copy in ExternalSource operator (#2124)
  • Add more verbose info when HwDecoderUtilizationTest is skipped (#2106)
  • Per-stream/per-device object pool. (#2127)
  • Fix PaddlePaddle test broken by rarfile update not compatible with Python 3.5 (#2130)
  • Add missing and a partial check in linter for this include file. (#2131)
  • Add libprotobuf-static as DALI conda build dependency (#2132)
  • Auto apply dataset options (#1963)
  • Add an option to use a copy kernel to feed external input (#2122)
  • Adjust mel filter test to librosa change (#2144)
  • Add dependency to dali_kernels to dali lib (#2143)
  • Tune Arithmetic Op launch specification (#2137)
  • Add daliOutputCopy (#2145)
  • Reduce memory usage in VideoReadeResize test (#2149)
  • Normal Distribution GPU Operator (#2125)
  • Remove pinning of numba version as librosa 0.8.0 has been released (#2151)
  • Add an ability to suppress _iterator_deprecation_warning (#2154)
  • Span-of-arrays flattening + minor layout utils (#2156)
  • Remove deprecated use of ltrb in BboxRandomCrop (#2141)
  • Improve PyTorch and MXNet ExternalSource examples (#2147)
  • Enable DALI build and tests for SBSA (#2110)
  • Add --disable-mmap flag to RN50 data pipeline test (#2163)
  • Make TF dataset build for 2.3.0 (#2160)
  • Enforce recordio indices are not empty (#2157)
  • Add daliOutputCopySamples (#2161)
  • Use TIFFGetFieldDefaulted and remove warning about falling back to GenericImage decoder (#2153)
  • Add an information about the faulty image to CreateImage invocation in nvjpeg_decoder_decoupled_api.h (#2174)
  • Add proper error handling where there are no valid sequences in the VideoReader (#2180)
  • Update instruction how run ResNet50 example for PyTorch (#2170)
  • Add the possibility to skip individual samples when using daliOutputCopySamples (#2186)
  • Change DALI build command to use minor CUDA version as well (#2155)
  • Reworked Resize operator family - video, channel-first, RoI and multiple type support (#2164)
  • Move to Update 1 release of CUDA 11 toolkit (#2188)
  • Make the test deterministically pick video files. (#2190)
  • Resize tutorial (#2189)
  • Use copy kernel when making a contiguous batch during ShareUserData, if user requested it (#2200)

Breaking API changes

  • Remove deprecated use of ltrb in BboxRandomCrop (#2141)

Deprecated feature

Known issues:

  • The video loader operator requires that the key frames occur at a minimum every 10 to 15 frames of the video stream. If the key frames occur at a lesser frequency, then the returned frames may be out of sync.
  • The DALI TensorFlow plugin might not be compatible with TensorFlow versions 1.15.0 and later.
    To use DALI with the TensorFlow version that does not have a prebuilt plugin binary shipped with DALI, make sure that the compiler that is used to build TensorFlow exists on the system during the plugin installation. (Depending on the particular version, use GCC 4.8.4, GCC 4.8.5, or GCC 5.4.)
  • Due to some known issues with meltdown/spectra mitigations and DALI, DALI shows best performance when run in Docker with escalated privileges, for example:
    • privileged=yes in Extra Settings for AWS data points
    • --privileged or --security-opt seccomp=unconfined for bare Docker

Binary builds

Install via pip for CUDA 10
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-cuda100==0.25.0
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-tf-plugin-cuda100==0.25.0

or for CUDA 11:
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-cuda110==0.25.0
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-tf-plugin-cuda110==0.25.0

Or use direct download links (CUDA 10.0):

Or use direct download links (CUDA 11.0):

FFmpeg source code:

  • This software uses code of FFmpeg licensed under the LGPLv2.1 and its source can be downloaded here

Libsndfile source code:

Assets 2
Loading

DALI v0.24.0

29 Jul 17:20
@klecki klecki
v0.24.0
81f94e0
Compare
Choose a tag to compare
DALI v0.24.0

Key Features and Enhancements

This DALI release includes the following key features and enhancements.

  • New Operators:
  • Operator Improvements:
    • Extended the Slice and Crop family of operators with out-of-bounds policies, which provides support for padding and trimming to existing shape (#2000, #2056, #2044).
    • Moved the memory hint allocation in the Resize to the build phase (#2033).
    • Optimized the Transpose GPU operator to improve the performance on non-uniform data batches (#2011, #2032).
  • Support for GPU data input data in the ExternalSource operator (#1997).
    • Added built-in support for GPU CuPy and PyTorch tensors in ExternalSource (#2050).
    • Added the ability to provide an external stream, stream 0, or automatic stream selection for GPU data access (#2050).
    • Added DLPack input support to the ExternalSource operator (#2023).
  • Add an ability to dump info about operator output buffer size (#2039)
  • Improved error checking with external libraries (#2062, #2063).

Bug fixes

  • Fix semantics of the masks_meta field (#2029)
  • Fix shape comparison in C API tests. (#2045)
  • Fix conda build after TensorFlow 2.2 release (#2048)
  • Fix Slice pad support when last dimension is padded (#2056)
  • Fix TL1_jupyter_conda test (#2058)
  • Fix CropMirrorNormalize benchmark (#2059)
  • Fix epoch_size method in the pipeline (#2071)
  • Undefined name: RuntimeErrorError --> RuntimeError (#2076)
  • Use ==/!= to compare constant literals (str, bytes, int, float, tuple) (#2078)
  • Fix Assertion is always true in Python tests (#2077)
  • Fix undefined name errors in Python reshape tests (#2079)
  • Fix conda build after the new build of libprotobuf was released (#2101)

Improvements

  • Add Convolution CPU kernel (#1987)
  • Lock numba version to 0.49 when librosa is used (#2016)
  • Add a deprecation warning for python 3.5 (#2021)
  • Change locked version of numba to at most 0.49, as 0.47 is the last release for py35 (#2020)
  • Add Preemphasis GPU operator (#2025)
  • Add out-of-bounds-policy (including pad support) to Slice/Crop (#2000)
  • Change from a custom manylinux3 to prebuild and upstream manylinux2014 (#1965)
  • Enable python ExternalSource operator for the GPU data (#1997)
  • Batched GPU transposition (#2011)
  • Move memory hint allocation in the Resize to the build phase (#2033)
  • Replace cuTT in Transpose operator with DALI kernel + move permute to core. (#2032)
  • Separable convolution (#2009)
  • Build DALI with OpenMP SIMD (#1992)
  • Use empty tensors for DL FW plugins instead of zeroed one (#2030)
  • Lanczos3 downscale + interp type notebook. (#2041)
  • Update docs layout template after sphinx_rtd_theme update (#2046)
  • Makes TF RN50 TL3 test to compile ahead of time (#2028)
  • Add an ability to dump info about operator output buffer size (#2039)
  • Add Gaussian window calculation for Gaussian Op (#2053)
  • Remove cuda 9 related packages from tests, update cupy to 7.5 (#2049)
  • Use Slice kernel to implement Pad operator (instead of SliceFlipNormalizePermutePad) (#2057)
  • Add PyTorch support in ExternalSource + fix handling of CUDA streams in Python frontend (#2050)
  • Add GaussianBlur CPU Op (#2038)
  • HW Decoder utilization test (#2022)
  • Add DLPack input support to the ExternalSource operator (#2023)
  • Add better return value/error status checks (#2062)
  • Check libjpeg return codes (#2063)
  • CropMirrorNormalize full pad support (#2044)
  • Remove confusing main from nosetest files (#2083)
  • Update to CUDA 11 GA toolkit (#2094)
  • Detect version of CUDA based on libcudart.so.* name (#2105)
  • Reduce Paddle RN50 test gpu mem fraction to 80% (#2152)

Breaking API changes

Deprecated feature

  • Added a deprecation warning for Python 3.5 (#2021).
  • Deprecated output_dtype and use dtype (#2051).
  • Added an argument deprecation mechanism and deprecated "image_type" in Crop, Slice, and CropMirrorNormalize (#2061).

Known issues:

  • The video loader operator requires that the key frames occur at a minimum every 10 to 15 frames of the video stream. If the key frames occur at a lesser frequency, then the returned frames may be out of sync.
  • The DALI TensorFlow plugin might not be compatible with TensorFlow versions 1.15.0 and later.
    To use DALI with the TensorFlow version that does not have a prebuilt plugin binary shipped with DALI, make sure that the compiler that is used to build TensorFlow exists on the system during the plugin installation. (Depending on the particular version, use GCC 4.8.4, GCC 4.8.5, or GCC 5.4.)
  • Due to some known issues with meltdown/spectra mitigations and DALI, DALI shows best performance when run in Docker with escalated privileges, for example:
    • privileged=yes in Extra Settings for AWS data points
    • --privileged or --security-opt seccomp=unconfined for bare Docker

Binary builds

Install via pip for CUDA 10
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-cuda100==0.24.0
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-tf-plugin-cuda100==0.24.0

or for CUDA 11:
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-cuda110==0.24.0
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-tf-plugin-cuda110==0.24.0

Or use direct download links (CUDA 10.0):

https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda100/nvidia_dali_cuda100-0.24.0-1446725-cp35-cp35m-manylinux2014_x86_64.whl
https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda100/nvidia_dali_cuda100-0.24.0-1446725-cp36-cp36m-manylinux2014_x86_64.whl
https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda100/nvidia_dali_cuda100-0.24.0-1446725-cp37-cp37m-manylinux2014_x86_64.whl
https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda100/nvidia_dali_cuda100-0.24.0-1446725-cp38-cp38-manylinux2014_x86_64.whl
https://developer.download.nvidia.com/compute/redist/nvidia-dali-tf-plugin-cuda100/nvidia-dali-tf-plugin-cuda100-0.24.0.tar.gz

Or use direct download links (CUDA 11.0):

https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda110/nvidia_dali_cuda110-0.24.0-1472979-cp35-cp35m-manylinux2014_x86_64.whl
https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda110/nvidia_dali_cuda110-0.24.0-1472979-cp36-cp36m-manylinux2014_x86_64.whl
https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda110/nvidia_dali_cuda110-0.24.0-1472979-cp37-cp37m-manylinux2014_x86_64.whl
https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda110/nvidia_dali_cuda110-0.24.0-1472979-cp38-cp38-manylinux2014_x86_64.whl
https://developer.download.nvidia.com/compute/redist/nvidia-dali-tf-plugin-cuda110/nvidia-dali-tf-plugin-cuda110-0.24.0.tar.gz

FFmpeg source code:

  • This software uses code of FFmpeg licensed under the LGPLv2.1 and its source can be downloaded here

Libsndfile source code:

Assets 2
Loading

DALI v0.23.0

29 Jun 18:01
@klecki klecki
v0.23.0
d6f581b
Compare
Choose a tag to compare
DALI v0.23.0

Key Features and Enhancements

This DALI release includes the following key features and enhancements.

  • DALI packages name add -cuda110 and -cuda100 suffixes to indicate CUDA version and allow hosting all packages under single pip index. This is important only for installation, the DALI module in Python is still nvidia.dali regardless of CUDA version. See the https://docs.nvidia.com/deeplearning/dali/user-guide/docs/installation.html installation guide for details.
  • New and improved Operators:
    • Normalize Operator for GPU (#1974, #1981, #1986).
    • Support for epsilon and delta degrees of freedom arguments for Normalize Operator (#1964).
    • SequenceRearrange Operator (#465).
    • Erase Operator for GPU (#1971).
  • Improve how iterators count padded samples based on the reader (#1831) - the provided iterators can now query reader for the epoch size and sharding and handle the shard size changing from epoch-to-epoch when it's not evenly divisible by number of shards (rank) and batch size. More details can be found in https://docs.nvidia.com/deeplearning/dali/user-guide/docs/advanced_topics.html#sharding
  • CUDA 11 build scripts for DALI were added (#2008).

Bug fixes

  • Fix out-of-source build (#1975)
  • Fix typo in installation documentation (#1976)
  • Fix reference counting issue in the PythonFunction operator (#1978)
  • Fix the wording for preset OF argument (#1994)
  • Fix generation of Erase Region in kernel test (#1996)
  • Fix GPU spectrogram when window_length != nfft (#1999)
  • Fix MelFilterBank bug: setup block descriptors when changing shape between iterations. (#2001)
  • Change locked version of numba to at most 0.49, as 0.47 is the last release for py35 (#2016, #2020)

Improvements

  • Mean and Standard Deviation GPU kernels (#1919)
  • Linter script change: from CMake to Python (#1951)
  • Update links to the new location, remove deprecated installation guide (#1955)
  • Adding more Numpy data types (#1961)
  • Extend HSV example with RandomGrayscale implementation (#1962)
  • Add workaround for the problem with patchelf changing TLS alignment (#1952)
  • Add epsilon and ddof (delta degrees of freedom) arguments to Normalize. (#1964)
  • Small docs improvements (#1970)
  • Add Sequence Rearrange Op (#465)
  • Add a helper class for fast unsigned division, usable on host and device. (#1967)
  • Fix documentation drop-down menu and other links (#1972)
  • Erase GPU operator (#1971)
  • Update TF versions supported (#1973)
  • Add -cudaXXX to dali package name (#1948)
  • Add more error checking (#1979)
  • Make DALI test to be fPIE (#1980)
  • Normalize GPU kernel (#1974)
  • Normalize GPU - pImpl + Bessel's corrections (#1981)
  • Slice CPU kernel pad support (#1977)
  • Makes GTest and Google Benchmark fPIE, DALI binaries as dynamically relocatable (#1982)
  • Add more error checking in TensorFlow DALI integration (#1991)
  • Normalize operator for GPU backend (#1986)
  • Slice GPU kernel with multi-channel pad support (#1983)
  • Split Slice benchmarks into CPU and GPU (#1995)
  • Improve how iterators count padded samples based on the reader (#1831)
  • Remove boost from the dependencies as it is no longer used anyway (#2006)
  • Enable file path arguments (#2002)
  • Enable CUDA 11 builds (#2008)
  • Silence CUDA 11 compute 35 and 50 deprecation warning (#2010)
  • Drop CUDA 9 from docs (#2012)

Breaking API changes

  • DALI packages name add -cuda110 and -cuda100 suffixes to indicate CUDA version and allow hosting all packages under single pip index.
  • CUDA 9 is no longer supported. DALI 0.22.0 was the last release that provides CUDA 9 build.

Deprecated feature

Known issues:

  • The video loader operator requires that the key frames occur at a minimum every 10 to 15 frames of the video stream. If the key frames occur at a lesser frequency, then the returned frames may be out of sync.
  • DALI TensorFlow plugin may not be compatible with TensorFlow versions 1.15.0 and/or later. If the user wants to use DALI with TensorFlow version which doesn’t have prebuilt plugin binary shipped with DALI it requires the gcc compiler that matches the one used to build TensorFlow (gcc 4.8.4 or gcc, 4.8.5 or 5.4, depending on the particular version) is present on the system.
  • Due to some known issues with meltdown/spectra mitigations and DALI, DALI shows best performance when run in Docker with escalated privileges, for example:
    • privileged=yes in Extra Settings for AWS data points
    • --privileged or --security-opt seccomp=unconfined for bare Docker

Binary builds

Install via pip for CUDA 10
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-cuda100==0.23.0
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-tf-plugin-cuda100

or for CUDA 11:
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-cuda110==0.23.0
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/ nvidia-dali-tf-plugin-cuda110

Or use direct download links (CUDA 10.0):

https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda100/nvidia_dali_cuda100-0.23.0-1396139-cp35-cp35m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda100/nvidia_dali_cuda100-0.23.0-1396139-cp36-cp36m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda100/nvidia_dali_cuda100-0.23.0-1396139-cp37-cp37m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda100/nvidia_dali_cuda100-0.23.0-1396139-cp38-cp38-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/nvidia-dali-tf-plugin-cuda100/nvidia-dali-tf-plugin-cuda100-0.23.0.tar.gz

Or use direct download links (CUDA 11.0):

https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda110/nvidia_dali_cuda110-0.23.0-1396141-cp35-cp35m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda110/nvidia_dali_cuda110-0.23.0-1396141-cp36-cp36m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda110/nvidia_dali_cuda110-0.23.0-1396141-cp37-cp37m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/nvidia-dali-cuda110/nvidia_dali_cuda110-0.23.0-1396141-cp38-cp38-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/nvidia-dali-tf-plugin-cuda110/nvidia-dali-tf-plugin-cuda110-0.23.0.tar.gz

FFmpeg source code:

  • This software uses code of FFmpeg licensed under the LGPLv2.1 and its source can be downloaded here

Libsndfile source code:

Assets 2
Loading

DALI v0.22.0

09 Jun 08:57
@klecki klecki
v0.22.0
4070053
Compare
Choose a tag to compare
DALI v0.22.0

Key Features and Enhancements

This DALI release includes the following key features and enhancements.

  • DALI now supports CUDA 11:
    • DALI builds for CUDA 11 are now available.
    • CUDA 9 support has been deprecated.
    • DALI 0.22.0 is the final release that provides a CUDA 9 build.
  • Support is now available for the Ampere Hardware JPEG decoder.
  • The following new operators are now available:
    • NumpyReader, which allows you to read standard .npy (NumPy) files (#1858).
    • CoordFlip for CPU and GPU (#1894, #1895).
  • Readers can be set to read files directly instead of using mmap, which improves network filesystems performance (#1909).
  • DALI can be built as a CMake subproject (#1924).

Bug fixes

  • Fix TL1_tensorflow-dali_test (#1869)
  • Hotfix of external_source.py (#1878)
  • Build fix for aarch64 (incorrect cmake dependency) (#1883)
  • Fix TL1_ssd_training test by freezing apex version (#1898)
  • Fix support for dynamic per-sample shape in Warp operators (#1911)
  • Remove Optical flow test bug (#1902)
  • Fix jitter operator illegal memory access (#1914)
  • Fix setup_packages.py after pip update to 20.1 version (#1916)
  • Fix TL1_python-nvjpeg_test test dependency (#1926)
  • L1 test fix for Xavier (#1936)
  • Fix tensorflow_dataset test to run on any power of 2 number of GPUs (#1935)
  • Fix a race condition in ExternalSourceTest test (#1943)

Improvements

  • Add support for array and cuda_array interface for DALI tensor (#1857)
  • Add collapse_dim and collapse_dims for TensorListShape. (#1862)
  • Add support for TensorFlow 2.2.0rc2 (#1860)
  • Add ExternalSource to "C API" (#1865)
  • Numpy reader (#1858)
  • Add TensorGPU and TensorListGPU constructors based on CUDA array interface (#1868)
  • Bump up OpenCV version to 4.3.0, libturbo-jpeg to 2.0.4, libtiff to 4.1.0, FFmpeg to 4.2.2 (#1783)
  • Add "no exec check" to SmallVector to prevent warnings in host-only functions. (#1870)
  • Allow for a separate dali_tf_plugin pip wheel step (#1856)
  • QA tests: Fix nvidia-dali-tf-plugin to uninstall weekly and nightly packages (#1877)
  • make install target for installing DALI on system where it's build (#1854)
  • Allow RandomBBoxCrop thresholds to refer to relative overlap alternatively to IoU (#1874)
  • Add a link to release notes in the docs (#1881)
  • Operator diagnostics mechanism (#1880)
  • Reductions: position-dependent preprocessing, kernels for unhandled edge cases (#1884)
  • Update Horovod in Tensorflow test (#1887)
  • Add an ability to strip DALI whl binary from debug symbols (#1897)
  • Extend conda testing (#1784)
  • Copy out core* files if the test_body fails (#1890)
  • Make volume return 1 for 0-dim shape. (#1906)
  • Update DALI PyTorch RN50 example to the latest AMP version (#1888)
  • Add a specialized TF dataset for conda (#1910)
  • Deserialize pipeline in python API (#1912)
  • Add CoordFlip CPU operator (#1894)
  • Restore an ability to use direct read of files instead of mmap (#1909)
  • Use only ImportError in setup_packages (#1922)
  • Collect exit code from test_body (#1923)
  • Coordinate Flip GPU operator (#1895)
  • DALI as a git submodule (#1924)
  • Add Erase GPU Kernel (#1903)
  • C API ExternalSource for GPU input (#1892)
  • Fix warning condition in ExternalSource (#1934)
  • Reduce GPU - kernel frontend (#1882)
  • Add checking alignment argument for 0 in the pad operator (#1937)
  • Move from http://xiph.org to GitHub for libflac, libvorbis and libogg (#1938)
  • C API function: inherit parameters from serialized pipeline (#1932)
  • Use LinearTransformation kernel in ColorTwist GPU Op (#1918)
  • Adjust test sizes for Erase GPU Kernel (#1939)
  • Use user stream by default in copy_to_external/feed_ndarray (#1921)
  • Move to TensorFlow 2.2.0 from 2.2.0-RC2 (#1946)
  • Add support for random_shuffle argument in test_RN50_data_pipeline (#1945)
  • Proper DALI initialization in process & daliInitialize function (#1929)
  • Update clang version to 8.0.1 in deps image (#1949)
  • Add support for nvjpeg HW decoder, including rework to accommodate different decoding methods in one batch
  • Fix "hw_decoder_load" handling for slice/cropImageDecoder for nvJPEG
  • Move HW decoding to separate stream
  • Fix linter in nvjpeg HW decoder
  • Deprecate CUDA 9
  • Add CUDA 11 to the installation guide and build.sh

Breaking API changes

None

Deprecated feature

  • CUDA 9 support is deprecated. DALI 0.22.0 is the last release that provides CUDA 9 build.

Known issues:

  • The video loader operator requires that the key frames occur at a minimum every 10 to 15 frames of the video stream. If the key frames occur at a lesser frequency, then the returned frames may be out of sync.
  • DALI TensorFlow plugin may not be compatible with TensorFlow versions 1.15.0 and/or later. If the user wants to use DALI with TensorFlow version which doesn’t have prebuilt plugin binary shipped with DALI it requires the gcc compiler that matches the one used to build TensorFlow (gcc 4.8.4 or gcc, 4.8.5 or 5.4, depending on the particular version) is present on the system.
  • Due to some known issues with meltdown/spectra mitigations and DALI, DALI shows best performance when run in Docker with escalated privileges, for example:
    • privileged=yes in Extra Settings for AWS data points
    • --privileged or --security-opt seccomp=unconfined for bare Docker

Binary builds

Install via pip for CUDA 9:
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/cuda/9.0 nvidia-dali==0.22.0
or for CUDA 10:
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/cuda/10.0 nvidia-dali==0.22.0
or for CUDA 11:
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/cuda/11.0 nvidia-dali==0.22.0

Or use direct download links (CUDA 9.0):
https://developer.download.nvidia.com/compute/redist/cuda/9.0/nvidia-dali/nvidia_dali-0.22.0-1313462-cp35-cp35m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/9.0/nvidia-dali/nvidia_dali-0.22.0-1313462-cp36-cp36m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/9.0/nvidia-dali/nvidia_dali-0.22.0-1313462-cp37-cp37m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/9.0/nvidia-dali/nvidia_dali-0.22.0-1313462-cp38-cp38-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/9.0/nvidia-dali-tf-plugin/nvidia-dali-tf-plugin-0.22.0.tar.gz

Or use direct download links (CUDA 10.0):

https://developer.download.nvidia.com/compute/redist/cuda/10.0/nvidia-dali/nvidia_dali-0.22.0-1313464-cp35-cp35m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/10.0/nvidia-dali/nvidia_dali-0.22.0-1313464-cp36-cp36m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/10.0/nvidia-dali/nvidia_dali-0.22.0-1313464-cp37-cp37m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/10.0/nvidia-dali/nvidia_dali-0.22.0-1313464-cp38-cp38-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/10.0/nvidia-dali-tf-plugin/nvidia-dali-tf-plugin-0.22.0.tar.gz

Or use direct download links (CUDA 11.0):

https://developer.download.nvidia.com/compute/redist/cuda/11.0/nvidia-dali/nvidia_dali-0.22.0-1313465-cp35-cp35m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/11.0/nvidia-dali/nvidia_dali-0.22.0-1313465-cp36-cp36m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/11.0/nvidia-dali/nvidia_dali-0.22.0-1313465-cp37-cp37m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/11.0/nvidia-dali/nvidia_dali-0.22.0-1313465-cp38-cp38-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/11.0/nvidia-dali-tf-plugin/nvidia-dali-tf-plugin-0.22.0.tar.gz

FFmpeg source code:

  • This software uses code of FFmpeg licensed under the LGPLv2.1 and its source can be downloaded here

Libsndfile source code:

Assets 2
Loading

DALI v0.21.0

28 Apr 16:39
@klecki klecki
v0.21.0
10e4af7
Compare
Choose a tag to compare
DALI v0.21.0 Pre-release
Pre-release

Key Features and Enhancements

This DALI release includes the following key features and enhancements.

  • Introduced experimental Functional API (#1598):
    • Operators can be used directly with a single call, no need to create an instance with a constructor
    • DALI pipeline can be used in Context Manager
    • There is no need to subclass Pipeline
  • Simplified usage of ExternalSource (#1598, #1832) - it accepts callbacks or generators as a parameter.
  • Added Python 3.8 build and support (#1782)
  • Allowed seed to be set for serialized pipeline (#1844)

New operators:

  • ToDecibels GPU operator (#1837)
  • One hot encoding CPU operator (#1807)

Bug fixes

  • Fix positional argument propagation in TF Dataset (#1798)
  • Fix parameter name in data_node._check. (#1816)
  • Fix Transpose bugs - degenerate dims and non-uniform GPU (#1817)
  • Fix sharding.png image link in multigpu example (#1821)
  • Fix collecting vector arguments in rotate_params. (#1841)
  • Fix a leak of the last created DALI pipeline instance (#1845)
  • Remove of usage of internal Sphinx _MockImporter method (#1861)
  • Make SSDRandomCrop calculate crop window in double precision (#1848)

Improvements

  • Move RNNT test to Torch specific tests (#1805)
  • Propagate layout in cast operator (#1801)
  • Add proper type info for optional arguments in schema (#1769)
  • Add missing new line for section anchor in rst (#1808)
  • Add missing #include <cstdint> to util and math_util. (#1810)
  • Update file_list argument description in FileReader (#1779)
  • Functional API + improved ExternalSource + improved Pipeline (#1598)
  • GPU reduction kernels part 1 - non-directional batched and global reductions (#1806)
  • Enable NVTX profiling information for CUDA 10 by default (#1793)
  • Make read function provided to FFmpeg return AVERROR_EOF for EOF (#1814)
  • Make DALI buildable for Python 3.8 (#1782)
  • Allow empty arrays in MXNet iterator (#1815)
  • Ignore VS Code settings directory in Git (#1826)
  • Reworks setup_packages script (#1820)
  • Add one hot encoding operator (CPU backend) (#1807)
  • New page layout of Supported Operations & "How to verify DALI build" description in compilation tutorial (#1722)
  • Generator support in ExternalSource (#1832)
  • 3d RandomBboxCrop (#1785)
  • Update TF RN50 performance test threshold to make it pass on dgx1v32GB (#1838)
  • ToDecibels GPU kernel (#1836)
  • Add ReduceAllGPU kernel (#1839)
  • Directional reduction CUDA kernels (#1840)
  • Rename CPU reductions; separate reduction functors from kernels. (#1846)
  • ToDecibels GPU operator (#1837)
  • Allow seed to be set for serialized pipeline (#1844)
  • Change StrictVersion to LooseVersion in TensorFlow plugin (#1851)
  • Make reader respect shard_id pipeline argument in tf.data.Dataset with multiple GPUs example (#1850)

Breaking API changes

None

Deprecated feature

  • CUDA 9 support will end in several releases (#1684)

Known issues:

  • The video loader operator requires that the key frames occur at a minimum every 10 to 15 frames of the video stream. If the key frames occur at a lesser frequency, then the returned frames may be out of sync.
  • DALI TensorFlow plugin may not be compatible with TensorFlow versions 1.15.0 and/or later. If the user wants to use DALI with TensorFlow version which doesn’t have prebuilt plugin binary shipped with DALI it requires the gcc compiler that matches the one used to build TensorFlow (gcc 4.8.4 or gcc, 4.8.5 or 5.4, depending on the particular version) is present on the system.
  • Due to some known issues with meltdown/spectra mitigations and DALI, DALI shows best performance when run in Docker with escalated privileges, for example:
    • privileged=yes in Extra Settings for AWS data points
    • --privileged or --security-opt seccomp=unconfined for bare Docker

Binary builds

Install via pip for CUDA 9:
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/cuda/9.0 nvidia-dali==0.21.0
or for CUDA 10
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/cuda/10.0 nvidia-dali==0.21.0

Or use direct download links (CUDA 9.0):
https://developer.download.nvidia.com/compute/redist/cuda/9.0/nvidia-dali/nvidia_dali-0.21.0-1239037-cp35-cp35m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/9.0/nvidia-dali/nvidia_dali-0.21.0-1239037-cp36-cp36m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/9.0/nvidia-dali/nvidia_dali-0.21.0-1239037-cp37-cp37m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/9.0/nvidia-dali/nvidia_dali-0.21.0-1239037-cp38-cp38-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/9.0/nvidia-dali-tf-plugin/nvidia-dali-tf-plugin-0.21.0.tar.gz

Or use direct download links (CUDA 10.0):

https://developer.download.nvidia.com/compute/redist/cuda/10.0/nvidia-dali/nvidia_dali-0.21.0-1239036-cp35-cp35m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/10.0/nvidia-dali/nvidia_dali-0.21.0-1239036-cp36-cp36m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/10.0/nvidia-dali/nvidia_dali-0.21.0-1239036-cp37-cp37m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/10.0/nvidia-dali/nvidia_dali-0.21.0-1239036-cp38-cp38-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/10.0/nvidia-dali-tf-plugin/nvidia-dali-tf-plugin-0.21.0.tar.gz

FFmpeg source code:

  • This software uses code of FFmpeg licensed under the LGPLv2.1 and its source can be downloaded here

Libsndfile source code:

Assets 2
Loading

DALI v0.20.0

27 Mar 16:56
@klecki klecki
v0.20.0
c8de5f3
Compare
Choose a tag to compare
DALI v0.20.0 Pre-release
Pre-release

Key Features and Enhancements

This DALI release includes the following key features and enhancements.

Added operators:

  • Spectrogram for GPU (#1786)
  • MelFilterBank for GPU (#1796)

Allow align-only behavior in Pad operator by treating shape argument as minimum shape (#1764)
Added data_ptr method to Tensor and TensorList (#1773) - it enables __array_interface__ and __cuda_array_interface__ support.
Extended shape support in DALI Dataset for TensorFlow (#1723)
Documentation improvements: layouts, Python API.
Added Gluon iterator plugin (#1683)

Bug fixes

  • Fix bug in TransposeCPU & ToDecibels operators (#1729)
  • Fix BBFlip issues (#1738)
  • Fix build without NVJPEG (#1739)
  • Fix precision loss in CropWindowGenerator (#1735)
  • Fix warnings reported by static analysis tool: (#1734)
  • Fixed the test failure on Power and x86 (#1752)
  • Fix out of range detection in get_item for TensorList (#1758)
  • Fix a race condition in AsyncPipelinedExecutor destructor and WorkerThread (#1757)
  • Fix bug in the COCOReader with masks (#1724)
  • Fix test_plugin_manager (#1749)
  • Fix typo in TensorListGPU docs, show getitem docs (#1746)
  • Fix SSD type mismatch (#1767)
  • Fix TF dataset build (#1792)
  • Fix DALI TF plugin build (#1789)
  • Fix positional argument propagation in TF Dataset (#1798)

Improvements

  • Add Gluon iterator plugin (#1683)
  • Adjust mxnet DALIClassificationIterator doc (#1718)
  • Change default value in ToDecibels, add one test (#1720)
  • Add error handling when trying to serialize Python Operators (#1730)
  • Use CMake's CUDA language support (#1733)
  • Allow 1 and 2 dimmensional input for Slice (#1741)
  • Specialize mul artihm op for bool (#1737)
  • Optical flow test against ground truth. (#1753)
  • Add /usr/local/cuda/bin to PATH in the main Dockerfile (#1756)
  • Add an ability to read noncontinuous RecordIO and TFRecord files (#1747)
  • Allow align-only behavior in Pad operator by treating shape argument as minimum shape (#1764)
  • Enable XLA for TensorFlow RN50 tests and use passthrough ImageNet for MXNet (#1760)
  • Add Reinterpret operator as a flavor of Reshape (#1768)
  • Short-time Fourier transform for GPU (#1721)
  • Adds data_ptr method to Tensor and TensorList (#1773)
  • Correct COCOReader mask doc (#1772)
  • Add GPU variant of Spectrogram operator (#1786)
  • Extend shape support in DALI Dataset for TF (#1723)
  • MelFIlterBank GPU kernel (#1787)
  • MelFilterBank GPU operator (#1796)
  • Test for RNNT data pipeline (CPU) (#1745)
  • Add data layout documentation and input layout expectations in operator's documentation (#1766)
  • Move RNNT test to Torch specific tests (#1805)

Breaking API changes

None

Deprecated feature

  • CUDA 9 support will end in several releases (#1684)

Known issues:

  • The video loader operator requires that the key frames occur at a minimum every 10 to 15 frames of the video stream. If the key frames occur at a lesser frequency, then the returned frames may be out of sync.
  • DALI TensorFlow plugin may not be compatible with TensorFlow versions 1.15.0 and/or later. If the user wants to use DALI with TensorFlow version which doesn’t have prebuilt plugin binary shipped with DALI it requires the gcc compiler that matches the one used to build TensorFlow (gcc 4.8.4 or gcc, 4.8.5 or 5.4, depending on the particular version) is present on the system.
  • Due to some known issues with meltdown/spectra mitigations and DALI, DALI shows best performance when run in Docker with escalated privileges, for example:
    • privileged=yes in Extra Settings for AWS data points
    • --privileged or --security-opt seccomp=unconfined for bare Docker

Binary builds

Install via pip for CUDA 9:
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/cuda/9.0 nvidia-dali==0.20.0
or for CUDA 10
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/cuda/10.0 nvidia-dali==0.20.0

Or use direct download links (CUDA 9.0):

Or use direct download links (CUDA 10.0):

FFmpeg source code:

  • This software uses code of FFmpeg licensed under the LGPLv2.1 and its source can be downloaded here

Libsndfile source code:

Assets 2
Loading

DALI v0.19.0

02 Mar 17:02
@JanuszL JanuszL
v0.19.0
6d532d4
Compare
Choose a tag to compare
DALI v0.19.0 Pre-release
Pre-release

Bug fixes

  • Update examples with COCO data set and fix reader behavior for padding (#1557)
  • Fix TensorFlow dataset test (#1641)
  • Fix typo in QNX cmake files (#1648)
  • Remove allocation-dependent test assert (#1650)
  • Fix several explicit "something is implicitly deleted" warnings (#1652)
  • Fix formatting of the example in the FW iterators docs (#1649)
  • Fix hang in decoder benchmark (#1672)
  • Fix error message (#1680)
  • Fix torch stream initialization in TorchPythonFunction (#1681)
  • Fix multi-channel fill value check in Erase operator (#1675)
  • Tests fix after examples refactor (#1687)
  • Fix Reshape docstring typo (#1691)
  • Add synchronization to read/write operations in image decoder cache (#1702)
  • Fix Buffer linkage and Reshape bug (#1714)
  • Fix TL1 tests (#1710)
  • Fix Pad operator bug (#1713)

Improvements

  • Allow Crop and CropMirrorNormalize to crop sequences as if they were volumetric images (#1605)
  • Erase CPU operator (#1609)
  • Improved Reshape (#1634)
  • Add GetDimIndices utility to tensor_layout.h (#1640)
  • Add example with booleans, comparisons, bitwise and muxing (#1631)
  • Remove unimplemented scale parameter in ops.VideoReader. (#1658)
  • Change ambiguous here in docs developer version (#1657)
  • Docs layout and navigation changes (#1635)
  • GPU PythonFunction operator (#1655)
  • Rename Tensor to TensorList in Supported Ops doc (#1661)
  • Add Pad CPU operator (including aligned padded shape support) (#1642)
  • Remove the ColorTwist deprecation message (#1646)
  • Change PipelineAPIType to Enum (#1636)
  • Directional reductions (for CPU) - mean standard deviation, sum, mean square; with tree reduction. (#1653)
  • Add support to UINT8 data type in SequenceWrapper (#1643)
  • Moving operators around. (#1667)
  • Normalize CPU vol 2 (#1666)
  • GPU PyTorch operator (#1662)
  • Proposing new structure of DALI examples (#1540)
  • VideoReader example (#1612)
  • MovingMeanSquared kernel (#1668)
  • Allow extra dimensions with extent 1 in Spectrogram operator & AudioDecoder changes (#1679)
  • Make DataIter a base class for MXNet DALIGenericIterator (#1669)
  • Add Transpose CPU Operator (#1677)
  • Remove not supported python versions from manylinux build (#1694)
  • Add deprecation message about CUDA 9 (#1684)
  • Mitigate the OS file-max limit in the VideoReader (#1659)
  • Adds support to StopIteration raised inside framework iterators (#1625)
  • Enable FFTS builds for ARM (Xavier, QNX) (#1686)
  • Normalize operator for CPU backend (#1670)
  • Python operator notebook (#1685)
  • Change backend_impl at to getitem - return TensorXPU (#1682)
  • Normalize tutorial (#1697)
  • Adjust setup_packages.py to the latest pip version (#1698)
  • Remove gif as supported extension (#1700)
  • Making "Supported backend" title in docs appear correctly
  • Update supported TF versions, update setup_packages.py (#1693)
  • Add pass-through info to OpSchema to add shared data to stage outputs. (#1707)
  • Nonsilence operator (#1701)
  • Constant operator and Python wrapper. (#1699)
  • Add support in CropMirrorNormalize for uneven sizes of mean and std (#1708)
  • Shrink host buffers (#1712)
  • Move pipeline ownership from Dataset to Iterator (#1704)
  • Align Rn50 data processing pipeline for TensorFlow with upstream examples (#1706)
  • Add a note how to set DALI_EXTRA_PATH to run jupyter examples (#1703)
  • Gpu python operator notebook (#1715)
  • Update Memory consumption and Custom operator docs sections (#1719)
  • Use prebuild cupy for TL0_jupyter test (#1728)

Breaking API changes

None

Deprecated feature

  • CUDA 9 support will end in several releases (#1684)
  • Access to Tensors of TensorListCPU and TensorListGPU with at was replaced by array subscript operator. (#1682)

Known issues:

  • The video loader operator requires that the key frames occur at a minimum every 10 to 15 frames of the video stream. If the key frames occur at a lesser frequency, then the returned frames may be out of sync.
  • DALI TensorFlow plugin may not be compatible with TensorFlow versions 1.15.0 and/or later. If the user wants to use DALI with TensorFlow version which doesn’t have prebuilt plugin binary shipped with DALI it requires the gcc compiler that matches the one used to build TensorFlow (gcc 4.8.4 or gcc, 4.8.5 or 5.4, depending on the particular version) is present on the system.
  • Due to some known issues with meltdown/spectra mitigations and DALI, DALI shows best performance when run in Docker with escalated privileges, for example:
    • privileged=yes in Extra Settings for AWS data points
    • --privileged or --security-opt seccomp=unconfined for bare Docker

Binary builds

Install via pip for CUDA 9:
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/cuda/9.0 nvidia-dali==0.19.0
or for CUDA 10
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/cuda/10.0 nvidia-dali==0.19.0

Or use direct download links (CUDA 9.0):

Or use direct download links (CUDA 10.0):

FFmpeg source code:

  • This software uses code of FFmpeg licensed under the LGPLv2.1 and its source can be downloaded here

Libsndfile source code:

Assets 2
Loading