PaddlePaddle 3.2.2 Release: Enhanced Distributed Training and HuggingFace Interoperability

by

PaddlePaddle has released version 3.2.2, focusing on distributed parallelism, operator mechanisms, and hardware adaptation. According to the vendor, this update features multiple optimizations and upgrades to further enhance the framework's performance and stability.

Distributed Training Updates

The release prioritizes improvements in distributed training workflows:

  • FlexCheckpoint Optimization: The communication process for re-sharding in FlexCheckpoint has been optimized.
  • Interoperability: The framework now supports loading Checkpoints in the HuggingFace open-source format.
  • Model Parameter Access: A new full interface has been added to paddle.nn.Layer to return complete model parameters.
  • Sharded Optimizers: The sharded_state_dict function has been introduced for the group_sharded_optimizer_stage2 optimizer.
  • Bug Fixes: An issue with the device_id parameter and core dumps when loading safetensor files via paddle.load has been resolved.
  • Memory Management: A new PipelineDatasetPreprocessor mechanism has been introduced to eliminate potential memory leak issues in pipeline parallelism strategies.

Operator Mechanism Fixes

A precision issue affecting the to_tensor operation in BFloat16 list scenarios has been fixed.

Hardware Adaptation

The independent XPU memory monitoring module has been modified to ensure consistency with the latest memory monitoring logic.