PaddlePaddle 2.2.2: Mish Activation, TensorRT Upgrades, and Critical Bug Fixes

by

PaddlePaddle 2.2.2 is a maintenance release focused on refining the training framework and enhancing inference capabilities. The update addresses several performance bottlenecks and functional regressions introduced in the previous version, while adding support for new activation functions and expanding TensorRT integration.

Training Framework Updates

The release introduces several new APIs and enhancements to existing operations:

  • Mish Activation: The framework now includes paddle.nn.Mish and paddle.nn.functional.mish to support the Mish activation function, which is defined as x * tanh(softplus(x)).
  • PReLU Enhancements: The PReLU class and its functional variants now accept a data_format parameter to specify the input data layout.
  • Float16 Support: The paddle.index_select operation has been updated to support float16 data types.
  • Quantization Improvements: The PostTrainingQuantization class now accepts a data_loader parameter, allowing users to pass paddle.io.DataLoader objects or Python generators.

Bug Fixes and Stability

A significant portion of the release notes is dedicated to resolving stability issues across the training pipeline:

  • Core Operations: Fixes were applied to paddle.max and paddle.min to correct incorrect results on CPU when the axis parameter is a list type. Memory leaks associated with tensor index assignment have also been addressed.
  • Dynamic-to-Static Graph: The conversion process has been refined to handle initialization APIs, code comments, and loops (e.g., for ... zip ...) more accurately.
  • Quantization: Issues with redundant nodes in quantized dynamic graph models and compatibility problems with Paddle Lite have been resolved.

Paddle Inference Enhancements

For deployment scenarios, the update focuses on backend optimization and bug correction:

  • TensorRT Expansion: The TensorRT subgraph engine now supports a wider range of operators, including relu, gelu, swish, prelu, and mish. This allows for more efficient inference on NVIDIA hardware.
  • API Improvements: The C API has been updated to handle std::string types. Additionally, the CreatePredictor interface is now safe for use in multi-threaded environments.
  • Model-Specific Fixes: Bugs causing hangs in the ERNIE model on TensorRT 8 and incompatibilities with the roi_align operator have been patched.