Concept
Robot learning from rewards, demonstrations and humans
9 studiesEvidence last moved Sep 27, 2026
Robot learning covers how robots acquire skills from reward signals, human demonstrations, human ratings or even evolutionary inheritance rather than hand-coded control. This page also covers robot motor learning; the evidence is mostly simulation studies plus small real-robot demonstrations and user studies with 8–26 sessions.
Students often picture robots learning from a clean reward. In practice, rewards are shaped, humans provide noisy signals, and simulation results don't always transfer, which these studies show concretely.
Studies
9
Findings
5
10 supporting · 0 challenging · 0 qualifying citations
Open tensions
1
Latest change
Concept page published
Robot learning from rewards, demonstrations and humans
Currently
What we know
- What you reward, and what the robot senses, shapes what it learns.
- Negative reward is not a free safety signal.
- Copying demonstrations alone breaks when the scene changes.
- People can be the reward, but how they give it decides the outcome.
- Curiosity, inheritance and few-shot examples are viable but lightly tested.
Largest unresolved question
Simulation rankings do not fully carry to real robots: the full Coulomb-plus-vision-plus-proximity model was best in simulation, but on the real robot the model without the proximity penalty did best.
Common misconceptions
Results in simulation show a method works on robots.
Most studies here (EPG, navigation, Lamarckian evolution, nociception) were simulation-only, and the one sim-to-real comparison found the best model changed.
A non-significant difference between human ratings and a cost function proves they're equivalent.
The user study's authors note small groups; lack of a significant difference is not proof of equivalence.
Related
Claim ledger
What the evidence shows
Drawn from 9 studies in this library. Mix labels say which citation roles are present; they are not a strength score. Supports means evidence for a finding; Challenges means evidence against a stated position; Qualifies marks scope.
What you reward, and what the robot senses, shapes what it learns.
Shaping the reward with extra structured signals speeds and improves learning: a Coulomb-force reward improved success, collisions and goal distance for a navigating manipulator, and adding nociceptive (damage-sensing) inputs to reward gave the lowest positioning error in a simulated arm.
- Can treating obstacles like electric charges teach a robot to steer?
- Do pain signals or punishments help a robot learn to reach?
Study Role Design N Population Outcome Can treating obstacles like electric charges teach a robot to steer? Supports Computational / modellingReward-shaping ablation: five TD3 agents differing only in reward terms, trained in Gazebo and tested in simulated scenes (static, dynamic, cluttered) and two real-world scenes. No single N: five agent variants evaluated over multiple rollouts in several simulated test scenes and two real test scenes; exact rollout counts are in tables not included in the extracted text. Simulated and real TurtleBot3 Burger mobile robots with 2-D LiDAR Success rate, collision ratio per 1000 steps, and average final distance to goal Do pain signals or punishments help a robot learn to reach? Supports Computational / modellingSimulated 2-joint arm reaching task learned with CACLA+var; hyperparameters evolved by a genetic algorithm per condition, then best settings retrained from 10 random initialisations. No single N: four conditions, each with its best hyperparameter set trained 10 times; the genetic search used 32 individuals per generation for 50 generations. Simulated two-degree-of-freedom robot arm learning inverse kinematics Final positioning error, perceived nociception (potential for joint damage) and number of steps to reach the target Negative reward is not a free safety signal.
Explicit punishment can hurt: in the simulated arm, adding punishment slowed convergence and raised potential damage during learning by almost 50%, and the parameter search pushed the punishment weight to its minimum.
Copying demonstrations alone breaks when the scene changes.
Demonstrations help but need adaptation to new situations: plain behaviour cloning failed in new door configurations while evolutionary policy gradient using demonstrations generalised better, and a hybrid imitation-plus-planner navigator kept high success across four simulated environments.
- Can a robot adapt one demonstration to new situations?
- Can a robot learn a driving style from a few human demos?
Study Role Design N Population Outcome Can a robot adapt one demonstration to new situations? Supports Computational / modellingNew RL algorithm evaluated in simulation: OpenAI Gym continuous-control tasks with dense and sparse rewards (vs ES and DPPO), and a Sawyer door-opening task with one demonstration and changed door configurations (vs behaviour cloning and BC + DPPO). No participant or dataset N; Gym curves averaged over six repeated runs and door-task success rates over five repeated runs, each scored on 100 test episodes. Simulated robots (Reacher, Hopper, HalfCheetah, Swimmer; Sawyer arm) Average cumulative reward; door-opening success rate in new configurations Can a robot learn a driving style from a few human demos? Supports Computational / modellingSimulated wheeled robot in Gazebo and Unity; a small CNN picks one of nine local waypoints; 50 trials per model per navigation task in training and unseen test environments. No single N: 200 planner demonstrations and 12 human demonstrations per training environment; evaluation used six navigation tasks per simulator with 50 trials each. Simulated Clearpath Husky robot in two Gazebo and two Unity environments Navigation success rate and similarity (inverse modified Hausdorff distance) of trajectories to held-out human demonstrations People can be the reward, but how they give it decides the outcome.
Human feedback can substitute for an engineered cost function but its quality matters: human ratings produced success rates not significantly different from a camera cost function, but raters who scored relative to the previous attempt caused premature convergence; with EEG-based feedback, robot errors correlated strongly with decoder misclassifications.
- Can ordinary people teach a robot a skill just by rating its tries?
- How do human-side errors affect a robot learning from brain signals?
Study Role Design N Population Outcome Can ordinary people teach a robot a skill just by rating its tries? Supports Human experimentHumanoid robot Pepper optimised a ball-in-cup movement using either naive users' star ratings as the cost or a tuned camera-based cost function; success measures compared between setups and across users' rating strategies. N=26 · 26 non-expert participants each taught one session of 82 rated movements; compared with 15 camera-optimised sessions. Adult non-experts aged 19-70 recruited around Bielefeld University; Pepper humanoid robot. Final hit/miss and ball-cup distance, number of hits, roll-outs until first hit; correlation of user ratings with camera-measured distance. How do human-side errors affect a robot learning from brain signals? Supports Human experimentSecondary analysis of a human-robot interaction experiment: a robot arm learned gesture-to-action mappings with LinUCB, using EEG error-related potentials as reward, under warm-start (pre-trained) and cold-start conditions N=8 · 8 human subjects, each contributing several online learning datasets per condition (one subject had only one cold-start dataset) Eight healthy adult volunteers interacting with a robot arm via hand gestures while EEG was recorded Robot mapping errors (wrong gesture-action choices); ErrP classification errors (false positives/negatives); gesture recognition errors Curiosity, inheritance and few-shot examples are viable but lightly tested.
Alternative learning routes also work in constrained settings: inheriting learned controllers (Lamarckian evolution) produced faster simulated robots under small learning budgets, a curiosity-driven humanoid explored more evenly than random agents, and a neuromorphic robot learned to turn from ten positive examples.
- Should newborn robots inherit what their parents learned?
- Can curiosity drive a humanoid robot to learn how its body moves?
- Can a robot learn from its own lucky accidents?
Study Role Design N Population Outcome Should newborn robots inherit what their parents learned? Supports Computational / modellingSimulated evolution of modular robot bodies and CPG/CPPN controllers in Revolve/Gazebo; Darwinian vs Lamarckian controller inheritance compared over five lineages and three HyperNEAT learning budgets. Five lineages per condition, each with 20 robots per generation for 10 generations (200 robots per lineage); no single analytic N. Simulated modular robots built from RoboGen-style components Locomotion speed (fitness) after lifetime learning; parent-offspring morphological similarity; retention of parental controller material Can curiosity drive a humanoid robot to learn how its body moves? Supports Computational / modellingTwo online experiments on a real iCub robot: one curious learner for an arm compared with random and least-tried exploration; three parallel curious learners for both arms and torso at a table. No sample size in the usual sense: Experiment 1 used an arm model with 1296 state-actions and ran for 4000 transitions; Experiment 2 ran three learners for 18,000 transitions. A single physical iCub humanoid robot (arm and torso joints) Exploration coverage of states and state-actions, uniformity of visits, decay of intrinsic reward, emergent behaviour Can a robot learn from its own lucky accidents? Supports Computational / modellingSpiking-network controller on SpiNNaker hardware driving a tracked robot in a T-maze, plus abstract simulations varying signal strength and number of positive examples. No single N; robot tasks were trained from ten positive examples each, and simulation points were averaged over repeated random trials. A small mobile robot controlled by a neural network built with the Neural Engineering Framework, and simulated sensory signals. Similarity between network output and the desired action (normalised dot product); robot turning behaviour in a T-maze with and without a mirror.
Debates
Tensions and limits
Some items are genuine disagreements on the same question. Others mark different assays, populations, or outcomes.
Simulation rankings do not fully carry to real robots: the full Coulomb-plus-vision-plus-proximity model was best in simulation, but on the real robot the model without the proximity penalty did best.
PaperFren reads this as a limit on how far one study travels — different assays, populations, or outcomes — not a forced fight between papers.
Timeline
How understanding moved
Study years are when the paper was published. Evidence edits are dated changes to this page's claims. Explanations are when PaperFren added a Discovery — not a claim that the science happened that day.
2026
Concept page published
Robot learning from rewards, demonstrations and humans
Change log
What changed
Dated edits to this page's evidence: studies added or removed from a claim, claims added or withdrawn, and new explanations tagged here. Rewordings are not listed.
- Concept page published
Papers
9 studies in this library bear on Robot learning from rewards, demonstrations and humans, ordered by citations.
- Can curiosity drive a humanoid robot to learn how its body moves?
Rewarding a real humanoid robot for improving its own model of how its movements turn out made it explore more efficiently than random or least-tried exploration, and led it to discover and probe a table on its own.
- Should newborn robots inherit what their parents learned?
Letting simulated robots inherit the gait controllers their parents had learned made offspring walk faster than starting from inherited-but-unlearned controllers, especially when learning time was short.
- Can a robot learn from its own lucky accidents?
A neural robot controller learned new behaviours, like turning a particular way when facing a mirror, simply by being retrained on runs where it had happened to do the right thing.
- How do human-side errors affect a robot learning from brain signals?
A robot learning from people's brain-signal feedback made fewer mistakes when given a little prior training, and its errors tracked how often the brain-signal decoder misread the feedback.
- Do pain signals or punishments help a robot learn to reach?
Giving a learning robot arm pain-like sensor inputs made it more accurate and safer, while subtracting punishment from its reward made learning worse.
- Can a robot learn a driving style from a few human demos?
Adding a small imitation-learning module between a classical path planner and motion controller let a robot copy a human's navigation style while still reliably reaching its goal.
- Can treating obstacles like electric charges teach a robot to steer?
Adding reward terms modelled on electric attraction to the goal and repulsion from obstacles helped a learning robot reach goals more often with fewer collisions, in simulation and on a real robot.
- Can a robot adapt one demonstration to new situations?
Mixing evolutionary exploration with policy-gradient learning, guided by a single demonstration, let a simulated robot arm open doors in positions it was never shown, where copying the demonstration failed.
Show 1 more studiesShow fewer studies
- Can ordinary people teach a robot a skill just by rating its tries?
Non-expert users giving simple one-to-five star ratings taught a robot the ball-in-cup trick about as well as a carefully engineered camera-based cost function.
Compare studies
Select 2–10 studies. Design and N are labels, not a ranking.
Nothing selected yet.
Questions
What is still open
Simulation rankings do not fully carry to real robots: the full Coulomb-plus-vision-plus-proximity model was best in simulation, but on the real robot the model without the proximity penalty did best.
Ask PaperFren about Robot learning from rewards, demonstrations and humans
Study this conceptflashcards and short-answer questions
Why can human-provided feedback lead to poor robot learning?
In a ball-in-cup study, participants who rated attempts relative to the previous one caused premature convergence, while those rating distance to the cup succeeded. In EEG-based feedback, robot errors tracked decoder misclassifications, and one subject's learning failed due to poor decoding. So the feedback channel and rating strategy directly shape learning.
Give one piece of evidence that negative signals help and one that they hurt.
Adding nociceptive sensory inputs to the reward-only arm gave the lowest positioning error. But adding explicit punishment slowed convergence and raised potential damage by almost 50% during learning. Both findings are from one simple two-joint simulation with 10 runs per condition.
Flashcards
0 of 1 answers reviewed