Robot learning
Can a robot adapt one demonstration to new situations?
Open access · cc by · source: Europe PMC
Mixing evolutionary exploration with policy-gradient learning, guided by a single demonstration, let a simulated robot arm open doors in positions it was never shown, where copying the demonstration failed.
Study at a glance
- Design
- Computational / modelling — New RL algorithm evaluated in simulation: OpenAI Gym continuous-control tasks with dense and sparse rewards (vs ES and DPPO), and a Sawyer door-opening task with one demonstration and changed door configurations (vs behaviour cloning and BC + DPPO).
- N
- No participant or dataset N; Gym curves averaged over six repeated runs and door-task success rates over five repeated runs, each scored on 100 test episodes.
- Population
- Simulated robots (Reacher, Hopper, HalfCheetah, Swimmer; Sawyer arm)
- Outcome
- Average cumulative reward; door-opening success rate in new configurations
Structured fields used in claim comparison tables when every cited study has a complete layer.
Key findings
On dense-reward tasks EPG matched or beat the baselines, notably escaping a local optimum that trapped PPO in Swimmer. With sparse rewards all methods did worse, but EPG was the most sample-efficient. In the door task, plain behaviour cloning failed in the new configurations, while EPG reached high success rates and generalised better than behaviour cloning followed by DPPO. Results are reported as curves in figures, without numerical tables in the text.
Methodology
The authors built Evolutionary Policy Gradient (EPG): it creates a population of policies by adding noise to network weights, keeps the best-scoring ones, improves each with PPO, and merges them by training a child policy to imitate an ensemble of the elites, with an extra term pulling it toward the demonstration. They first tested exploration without demonstrations on OpenAI Gym robots, including versions modified to give only sparse rewards, against evolution strategies (ES) and distributed PPO. They then gave a simulated Sawyer arm one hand-engineered door-opening demonstration and tested it on doors with changed position, angle and handle placement.
Limitations
All experiments are in simulation, so nothing is shown about real robots, wear or safety. Results are reported mainly as learning curves over five or six repeats, with no statistical tests or numeric success rates in the text, making effect sizes hard to judge. The method is compared to a small set of baselines and only one manipulation task, and the demonstration-weight schedule must be tuned to how different the new environment is.
How this study connects
Role on claims
Each row is a claim on a concept or method page where this paper supports, challenges, or qualifies the statement. Roles are hand-checked — not a model guess.
Copying demonstrations alone breaks when the scene changes.
Demonstrations help but need adaptation to new situations: plain behaviour cloning failed in new door configurations while evolutionary policy gradient using demonstrations generalised better, and a hybrid imitation-plus-planner navigator kept high success across four simulated environments.
Evidence for the claim as stated.
Simulation rankings do not fully carry to real robots: the full Coulomb-plus-vision-plus-proximity model was best in simulation, but on the real robot the model without the proximity penalty did best.
Same question, contrary or null result.
Open questions
Tensions this paper is part of
From concept pages' “where studies disagree.” Disagreement means the same question; scope means different assays, populations, or outcomes.
Simulation rankings do not fully carry to real robots: the full Coulomb-plus-vision-plus-proximity model was best in simulation, but on the real robot the model without the proximity penalty did best.
Related papers in this topic
Same topic cluster — not a recommendation engine.