PaddlePaddle 2.3.1: CUDA 11.6 Support, Quantization Enhancements, and Performance Gains

PaddlePaddle released version 2.3.1, building upon the 2.3 baseline with bug fixes and the addition of CUDA 11.6 support. The update focuses on refining the training framework, particularly around quantization and distributed training, while also expanding Paddle Inference capabilities.

Training Framework Updates

The core training framework received several refinements. The paddle.io.DataLoader now fully supports the prefetch_factor parameter to cache pre-fetched data, aiming to reduce I/O blocking during large batch processing.

API changes include paddle.nn.initializer.KaimingUniform and KaimingNormal now supporting multiple activation function types. A new static graph executor was introduced with delayed thread pool initialization to avoid overhead for one-off programs like save or load.

Quantization and Mixed Precision

Quantization workflows were significantly updated. The framework now supports rounding to nearest ties to even in model quantization APIs and expands the quantization range to [-128, 127]. Quantization-aware training (QAT) now supports AMP (Automatic Mixed Precision) training.

Dynamic graph quantization training added support for conv and bn fusion, while static graph offline quantization gained a skip_tensor_list option to exclude specific layers.

Performance Improvements

PaddlePaddle reports performance gains in specific models. The fused_attention and fused_feedforward operators saw a 7.7% performance boost in CAE models after adding an add_residual property. The linspace operator was optimized to initialize inputs on CPU, resulting in a 6% performance increase for SOLOv2.

Paddle Inference and Deployment

On the deployment side, the ONNX Runtime backend on CPUs now supports PaddleSlim quantized models. TensorRT integration was expanded to include bilinear_interp_v2 and new fusion passes like matmul_scale_fuse_pass and multihead_matmul_fuse_pass_v3.

Hardware and Environment

Version 2.3.1 includes official verification and precompiled binaries for CUDA 11.6, including a fix for compilation errors on Windows. New hardware support was added for Cambricon MLU (Profiler) and GraphCore IPU (compilation progress visualization).