Robot learning
Can treating obstacles like electric charges teach a robot to steer?
Open access · cc by · source: Europe PMC
Adding reward terms modelled on electric attraction to the goal and repulsion from obstacles helped a learning robot reach goals more often with fewer collisions, in simulation and on a real robot.
Study at a glance
- Design
- Computational / modelling — Reward-shaping ablation: five TD3 agents differing only in reward terms, trained in Gazebo and tested in simulated scenes (static, dynamic, cluttered) and two real-world scenes.
- N
- No single N: five agent variants evaluated over multiple rollouts in several simulated test scenes and two real test scenes; exact rollout counts are in tables not included in the extracted text.
- Population
- Simulated and real TurtleBot3 Burger mobile robots with 2-D LiDAR
- Outcome
- Success rate, collision ratio per 1000 steps, and average final distance to goal
Structured fields used in claim comparison tables when every cited study has a complete layer.
Key findings
The baseline agent did worst everywhere, with the lowest success rate and most collisions. Adding the Coulomb reward consistently improved success, collisions and final goal distance and sped up training, and adding the vision reward improved things further. In simulation the model with all terms was best, but on the real robot the Coulomb-plus-vision model without the extra proximity penalty did best, because the full model took longer detours that raised its failure risk.
Methodology
The authors trained a TD3 deep reinforcement learning agent to drive a small wheeled robot using LiDAR. They compared five versions whose rewards differed: a baseline with a simple too-close penalty, and versions adding a Coulomb-force reward (goal attracts, obstacles repel with inverse-square strength), a 'vision' reward that encourages heading past obstacle edges before collision is imminent, or combinations. Agents were trained in the Gazebo simulator and then tested in new simulated scenes and on a real TurtleBot3.
Limitations
The extracted text lacks the results tables, so the size of the improvements and the number of test rollouts cannot be checked here, and no statistical tests or multiple training seeds are described. Real-world testing used only two simple scenes with a few static obstacles and a single small robot. The best model differed between simulation and reality, showing that simulation rankings do not fully transfer. 'Explainability' here means the reward terms have physical interpretations, not that the learned policy itself was analysed.
How this study connects
Role on claims
Each row is a claim on a concept or method page where this paper supports, challenges, or qualifies the statement. Roles are hand-checked — not a model guess.
What you reward, and what the robot senses, shapes what it learns.
Shaping the reward with extra structured signals speeds and improves learning: a Coulomb-force reward improved success, collisions and goal distance for a navigating manipulator, and adding nociceptive (damage-sensing) inputs to reward gave the lowest positioning error in a simulated arm.
Evidence for the claim as stated.
Simulation rankings do not fully carry to real robots: the full Coulomb-plus-vision-plus-proximity model was best in simulation, but on the real robot the model without the proximity penalty did best.
Evidence for the claim as stated.
'Interpretable' means different things: point scores are interpretable by simplicity (not measured with clinicians), knowledge-guided networks by structure, and a robot navigation paper uses 'explainable' to mean reward terms have physical meaning, not that the policy was analysed.
Evidence for the claim as stated.
Open questions
Tensions this paper is part of
From concept pages' “where studies disagree.” Disagreement means the same question; scope means different assays, populations, or outcomes.
Simulation rankings do not fully carry to real robots: the full Coulomb-plus-vision-plus-proximity model was best in simulation, but on the real robot the model without the proximity penalty did best.
'Interpretable' means different things: point scores are interpretable by simplicity (not measured with clinicians), knowledge-guided networks by structure, and a robot navigation paper uses 'explainable' to mean reward terms have physical meaning, not that the policy was analysed.
Related papers in this topic
Same topic cluster — not a recommendation engine.