PaddlePaddle 2.3.2 Release: CUDA 11.7 Compatibility, Quantization Enhancements, and Inference Optimizations

by

PaddlePaddle 2.3.2 Release: CUDA 11.7 Compatibility, Quantization Enhancements, and Inference Optimizations

PaddlePaddle has released version 2.3.2, a maintenance update focusing on bug fixes and minor feature additions across the training framework and deployment components.

Training Framework Updates

  • Profiler Improvements: The performance analyzer now collects input tensor shapes for operators in both static and dynamic graphs.
  • CUDA 11.7 Fixes: The release resolves unit test failures occurring in CUDA 11.7 environments.
  • Quantization: Issues with scale naming inconsistencies in post-training quantization (PTQ) and quantization-aware training (QAT) have been resolved. The update also adds support for new PACT quantization formats and enables Reduce Max operator quantization.
  • Custom Operators: Fixes address stream inconsistencies in custom operators and input attribute type parsing errors in the new dynamic graph mode.

Paddle Inference Enhancements

  • Operator Loading: A new option allows operators to be loaded on-demand using the environment variable export CUDA_MODULE_LOADING=LAZY.
  • ONNX Runtime: The backend now supports conversion for partial_sum, partial_concat, bilinear_interp, conv3d, pool3d, and multiclass_nms. Memory consumption is reduced through Clone interface support.
  • TensorRT: New conversion support is added for squeeze2, unsqueeze2, cast, and slice.
  • Transformer: FusedMultiTransformer gains a normalize_before=False option to support layer_norm placement after attention and feed-forward layers.