PaddlePaddle, the Chinese industrial-grade deep learning platform, has released version 3.0.0. The update focuses on simplifying large model training and enhancing performance through architectural upgrades, including a new automatic parallelism system and an upgraded neural network compiler.
Automatic Parallelism for Large Models
A major focus of PaddlePaddle 3.0 is reducing the complexity of distributed training. The framework introduces “dynamic/static unified automatic parallelism.” Users can mark tensor splits on a single card, and the system automatically derives distributed strategies, adds communication operators, and optimizes scheduling.
The release notes highlight that this architecture supports common large model scenarios, including dense models, Mixture-of-Experts (MoE) models, and multimodal models. A new paddle.distributed.parallel interface allows users to configure data, model, and pipeline parallelism without modifying the model definition code. The vendor claims this system has been validated on Llama series models and achieves performance on par with or exceeding manual parallelism.
Neural Network Compiler (CINN) Upgrades
The integrated CINN neural network compiler has received comprehensive optimizations. The release notes state that over 60% of models see significant performance improvements, with an average gain of 27.4%. Key optimizations include:
- Automatic Re-Compute for backward computation graphs.
- Pass performance optimization and symbol derivation upgrades.
- Enhanced operator fusion and schedule strategies.
- Improved subscript expression simplification.
Scientific Computing and High-Order Differentiation
PaddlePaddle 3.0 introduces high-order automatic differentiation capabilities for scientific computing. According to the vendor, testing against 41 equations in Nvidia Modulus showed that PaddlePaddle is 115% faster than PyTorch with compiler optimization enabled.
Additionally, the framework now natively supports complex number operations, which is significant for applications in weather forecasting and aerodynamic analysis. The release also introduces toolkits like PaddleScience and PaddleHelix to support general mathematical and biological computing tasks.
Unified Training and Inference
PaddlePaddle continues to support its “dynamic/static unified, train/inference unified” design philosophy. The framework claims a dynamic-to-static export success rate of 95%, compared to 62% for PyTorch. This allows developers to reuse model definition code between training and inference, streamlining the deployment pipeline. The release notes also mention support for the DeepSeek-R1 full version on single machines with doubled throughput.
Breaking Changes and Improvements
Implicit Type Promotion: To address stability issues in mixed precision (bf16/fp16) calculations, PaddlePaddle 3.0 clarifies implicit type promotion rules. This ensures consistency between dynamic and static graphs, operator overloading, and commutativity.
0-Dim Tensor: The framework has finalized the distinction between 0-dim Tensors and 1-dim Tensors containing a single element. A deprecated flag that incorrectly treated 0-dim Tensors as 1-dim has been removed.
Heterogeneous Hardware: The release emphasizes a robust ecosystem for hardware adaptation. The framework has attracted 4,001 pull requests and 26,584 commits from hardware vendors, enabling new devices to be integrated with minimal user configuration.