Skip to content

DALI v0.21.0

Pre-release
Pre-release
Compare
Choose a tag to compare
@klecki klecki released this 28 Apr 16:39
v0.21.0
10e4af7

Key Features and Enhancements

This DALI release includes the following key features and enhancements.

  • Introduced experimental Functional API (#1598):
    • Operators can be used directly with a single call, no need to create an instance with a constructor
    • DALI pipeline can be used in Context Manager
    • There is no need to subclass Pipeline
  • Simplified usage of ExternalSource (#1598, #1832) - it accepts callbacks or generators as a parameter.
  • Added Python 3.8 build and support (#1782)
  • Allowed seed to be set for serialized pipeline (#1844)

New operators:

  • ToDecibels GPU operator (#1837)
  • One hot encoding CPU operator (#1807)

Bug fixes

  • Fix positional argument propagation in TF Dataset (#1798)
  • Fix parameter name in data_node._check. (#1816)
  • Fix Transpose bugs - degenerate dims and non-uniform GPU (#1817)
  • Fix sharding.png image link in multigpu example (#1821)
  • Fix collecting vector arguments in rotate_params. (#1841)
  • Fix a leak of the last created DALI pipeline instance (#1845)
  • Remove of usage of internal Sphinx _MockImporter method (#1861)
  • Make SSDRandomCrop calculate crop window in double precision (#1848)

Improvements

  • Move RNNT test to Torch specific tests (#1805)
  • Propagate layout in cast operator (#1801)
  • Add proper type info for optional arguments in schema (#1769)
  • Add missing new line for section anchor in rst (#1808)
  • Add missing #include <cstdint> to util and math_util. (#1810)
  • Update file_list argument description in FileReader (#1779)
  • Functional API + improved ExternalSource + improved Pipeline (#1598)
  • GPU reduction kernels part 1 - non-directional batched and global reductions (#1806)
  • Enable NVTX profiling information for CUDA 10 by default (#1793)
  • Make read function provided to FFmpeg return AVERROR_EOF for EOF (#1814)
  • Make DALI buildable for Python 3.8 (#1782)
  • Allow empty arrays in MXNet iterator (#1815)
  • Ignore VS Code settings directory in Git (#1826)
  • Reworks setup_packages script (#1820)
  • Add one hot encoding operator (CPU backend) (#1807)
  • New page layout of Supported Operations & "How to verify DALI build" description in compilation tutorial (#1722)
  • Generator support in ExternalSource (#1832)
  • 3d RandomBboxCrop (#1785)
  • Update TF RN50 performance test threshold to make it pass on dgx1v32GB (#1838)
  • ToDecibels GPU kernel (#1836)
  • Add ReduceAllGPU kernel (#1839)
  • Directional reduction CUDA kernels (#1840)
  • Rename CPU reductions; separate reduction functors from kernels. (#1846)
  • ToDecibels GPU operator (#1837)
  • Allow seed to be set for serialized pipeline (#1844)
  • Change StrictVersion to LooseVersion in TensorFlow plugin (#1851)
  • Make reader respect shard_id pipeline argument in tf.data.Dataset with multiple GPUs example (#1850)

Breaking API changes

None

Deprecated feature

  • CUDA 9 support will end in several releases (#1684)

Known issues:

  • The video loader operator requires that the key frames occur at a minimum every 10 to 15 frames of the video stream. If the key frames occur at a lesser frequency, then the returned frames may be out of sync.
  • DALI TensorFlow plugin may not be compatible with TensorFlow versions 1.15.0 and/or later. If the user wants to use DALI with TensorFlow version which doesn’t have prebuilt plugin binary shipped with DALI it requires the gcc compiler that matches the one used to build TensorFlow (gcc 4.8.4 or gcc, 4.8.5 or 5.4, depending on the particular version) is present on the system.
  • Due to some known issues with meltdown/spectra mitigations and DALI, DALI shows best performance when run in Docker with escalated privileges, for example:
    • privileged=yes in Extra Settings for AWS data points
    • --privileged or --security-opt seccomp=unconfined for bare Docker

Binary builds

Install via pip for CUDA 9:
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/cuda/9.0 nvidia-dali==0.21.0
or for CUDA 10
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/cuda/10.0 nvidia-dali==0.21.0

Or use direct download links (CUDA 9.0):
https://developer.download.nvidia.com/compute/redist/cuda/9.0/nvidia-dali/nvidia_dali-0.21.0-1239037-cp35-cp35m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/9.0/nvidia-dali/nvidia_dali-0.21.0-1239037-cp36-cp36m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/9.0/nvidia-dali/nvidia_dali-0.21.0-1239037-cp37-cp37m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/9.0/nvidia-dali/nvidia_dali-0.21.0-1239037-cp38-cp38-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/9.0/nvidia-dali-tf-plugin/nvidia-dali-tf-plugin-0.21.0.tar.gz

Or use direct download links (CUDA 10.0):

https://developer.download.nvidia.com/compute/redist/cuda/10.0/nvidia-dali/nvidia_dali-0.21.0-1239036-cp35-cp35m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/10.0/nvidia-dali/nvidia_dali-0.21.0-1239036-cp36-cp36m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/10.0/nvidia-dali/nvidia_dali-0.21.0-1239036-cp37-cp37m-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/10.0/nvidia-dali/nvidia_dali-0.21.0-1239036-cp38-cp38-manylinux1_x86_64.whl
https://developer.download.nvidia.com/compute/redist/cuda/10.0/nvidia-dali-tf-plugin/nvidia-dali-tf-plugin-0.21.0.tar.gz

FFmpeg source code:

  • This software uses code of FFmpeg licensed under the LGPLv2.1 and its source can be downloaded here

Libsndfile source code:

Assets 2
Loading