v8.0.14: New activation functions, bug fixes and more

✨ New features and improvements

• Add new activation functions: ClippedLinear.v1, Gelu.v1, HardSigmoid.v1, HardSwish.v1, HardSwishMobilenet.v1, HardTanh.v1, ReluK.v1, and Swish.v1.
• Automatically set the GPU allocator to PyTorch when PyTorch models are loaded through PyTorchWrapper on GPU to avoid memory contention between CuPy and PyTorch.
• Support big endian platforms through thinc-bigendian-ops and consistently serialize model data with little endian byte order.
• Add Softmax.v2 with support for softmax with temperature and optional normalization.
• Add CategoricalCrossentropy.v3 and SequenceCategoricalCrossentropy.v3 with support for label smoothing.
• Speed up CupyOps.maxout by exploiting GPU parallelism better.
• Support sequence lengths in the NumpyOps.seq2col and CupyOps.seq2col implementations of Ops.seq2col to determine padding.
• Improve performance of Ragged.
• Support Ragged arrays in expand_window.v1.

🔴 Bug fixes

• Fix issue #552: Do not backpropagate Inf/NaN out of PyTorch layers when using mixed-precision training.
• Fix issue #578: Correctly cast the threshold argument of CupyOps.mish and correct an equation in Ops.backprop_mish.
• Fix issue #587: Correct invariant checks in CategoricalCrossentropy.get_grad.
• Fix issue #592: Update murmurhashrequirement.
• Fix issue #594: Do not sort positional arguments in Config.

⚠️ Backwards incompatibilities

• The out keyword argument of Ops.mish and Ops.backprop_mish is replaced by inplace for consistency with other activations.

📖Documentation and examples

• Update example Jupyter notebooks for the current Thinc version.

👥 Contributors

@adrianeboyd, @andrewsi-z, @danieldk, @honnibal, @ines, @Jette16, @kadarakos, @kianmeng, @polm, @svlandeg, @thatbudakguy