RESEARCH · RESEARCH · #2553
PPO vs. SAC compared for cooperative motion control of dual-arm space robots (arXiv:2610.32034v1)
This arXiv preprint develops a centralized RL framework for cooperative motion planning of a fixed-base dual-arm space manipulator using a multi-objective reward (target accuracy, orientation alignment, smoothness, synchronization, collision avoidance) and compares on-policy PPO with off-policy SAC across five random seeds. In the simulated tasks SAC substantially outperforms PPO: SAC reaches an 80% success threshold in every seed (median 1.6M environment steps) and achieves a higher mean success rate (78.6% vs. 7.4%), lower position/orientation errors, and tighter bimanual synchronization; PPO fails to reach the threshold within a 4M-step budget and exhibits a reward-loitering local-optimum failure mode under the shaped reward. Both methods degrade on unseen boundary targets, but SAC retains most performance while PPO largely collapses.
KEY POINTS
- This arXiv preprint develops a centralized RL framework for cooperative motion planning of a fixed-base dual-arm space manipulator using a multi-objective reward (target accuracy, orientation alignment, smoothness, synchronization, collision avoidance) and compares on-policy PPO with off-policy SAC across five random seeds.
- In the simulated tasks SAC substantially outperforms PPO: SAC reaches an 80% success threshold in every seed (median 1.6M environment steps) and achieves a higher mean success rate (78.6% vs.
- 7.4%), lower position/orientation errors, and tighter bimanual synchronization; PPO fails to reach the threshold within a 4M-step budget and exhibits a reward-loitering local-optimum failure mode under the shaped reward.
WHY IT MATTERS
The paper shows off-policy SAC is more sample-efficient and robust than on-policy PPO for constrained dual-arm space-manipulator tasks and highlights PPO's sensitivity to reward shaping, which matters for RL applied to on-orbit servicing and debris removal.