Can treating obstacles like electric charges teach a robot to steer?
Adding reward terms modelled on electric attraction to the goal and repulsion from obstacles helped a learning robot reach goals more often with fewer collisions, in simulation and on a real robot.
Source
Coulomb force-guided deep reinforcement learning for effective and explainable robotic motion planning
Study at a glance
- Design
- Computational / modelling — Reward-shaping ablation: five TD3 agents differing only in reward terms, trained in Gazebo and tested in simulated scenes (static, dynamic, cluttered) and two real-world scenes.
- N
- No single N: five agent variants evaluated over multiple rollouts in several simulated test scenes and two real test scenes; exact rollout counts are in tables not included in the extracted text.
- Population
- Simulated and real TurtleBot3 Burger mobile robots with 2-D LiDAR
- Outcome
- Success rate, collision ratio per 1000 steps, and average final distance to goal
Structured fields used in claim comparison tables when every cited study has a complete layer.
What they did
The authors trained a TD3 deep reinforcement learning agent to drive a small wheeled robot using LiDAR. They compared five versions whose rewards differed: a baseline with a simple too-close penalty, and versions adding a Coulomb-force reward (goal attracts, obstacles repel with inverse-square strength), a 'vision' reward that encourages heading past obstacle edges before collision is imminent, or combinations. Agents were trained in the Gazebo simulator and then tested in new simulated scenes and on a real TurtleBot3.
What they found
The baseline agent did worst everywhere, with the lowest success rate and most collisions. Adding the Coulomb reward consistently improved success, collisions and final goal distance and sped up training, and adding the vision reward improved things further. In simulation the model with all terms was best, but on the real robot the Coulomb-plus-vision model without the extra proximity penalty did best, because the full model took longer detours that raised its failure risk.
The limits
What it doesn't show
The extracted text lacks the results tables, so the size of the improvements and the number of test rollouts cannot be checked here, and no statistical tests or multiple training seeds are described. Real-world testing used only two simple scenes with a few static obstacles and a single small robot. The best model differed between simulation and reality, showing that simulation rankings do not fully transfer. 'Explainability' here means the reward terms have physical interpretations, not that the learned policy itself was analysed.
Key terms
- Reward shaping
- Adding extra reward terms to guide a reinforcement learning agent towards good behaviour faster than the task reward alone would.
- TD3 (Twin Delayed DDPG)
- An off-policy actor-critic algorithm for continuous actions that uses two critics and delayed policy updates to reduce overestimation and instability.
- Artificial potential field
- A classic planning method where the goal attracts and obstacles repel the robot; it can get stuck in local minima where forces cancel.
- Sim-to-real gap
- The drop or change in performance when a policy trained in simulation is run on a physical robot.
- Ablation study
- An experiment that adds or removes one component at a time to see how much each contributes.
Flashcards
0 of 10 answers reviewed
Research intelligence for this paper
See its role on concept claims, tensions it is part of, placement history, and related discoveries.
Quiz yourself
What is the core idea of the proposed method?
Common questions
How is this different from a classic potential-field planner?
The Coulomb forces are not used to steer directly; they are folded into the reward, and the agent learns a policy from them, which the authors argue avoids the local-minimum traps of hand-built potential fields.
Why did the best model change between simulation and the real robot?
The model with every reward term tended to make larger detours on the real robot, and longer paths gave more chances to collide, lose the goal or get stuck.
What does the vision reward add?
It uses obstacle edges from LiDAR and the estimated time to collision to reward heading past obstacles early, rather than reacting only when very close.
More on Robot learning