Can ordinary people teach a robot a skill just by rating its tries?
Non-expert users giving simple one-to-five star ratings taught a robot the ball-in-cup trick about as well as a carefully engineered camera-based cost function.
Source
A User Study on Robot Skill Learning Without a Cost Function: Optimization of Dynamic Movement Primitives via Naive User Feedback
Study at a glance
- Design
- Human experiment — Humanoid robot Pepper optimised a ball-in-cup movement using either naive users' star ratings as the cost or a tuned camera-based cost function; success measures compared between setups and across users' rating strategies.
- N
- N=26 · 26 non-expert participants each taught one session of 82 rated movements; compared with 15 camera-optimised sessions.
- Population
- Adult non-experts aged 19-70 recruited around Bielefeld University; Pepper humanoid robot.
- Outcome
- Final hit/miss and ball-cup distance, number of hits, roll-outs until first hit; correlation of user ratings with camera-measured distance.
Structured fields used in claim comparison tables when every cited study has a complete layer.
What they did
A Pepper humanoid robot learned the ball-in-cup game using a standard movement representation (dynamic movement primitives) optimised by a simple evolution strategy (CMA-ES), starting from an imperfect demonstrated throw. In one condition a tuned two-camera system measured ball-to-cup distance as the cost; in the other, 26 naive participants rated each of the robot's attempts from one to five stars on its tablet, with no instructions about how to rate. The authors compared five success measures between setups and related participants' self-reported rating strategies and rating accuracy to learning success.
What they found
None of the five success measures differed significantly between human ratings and the camera cost function. Of 26 user sessions, 17 converged successfully, 6 converged prematurely and 3 failed; the camera condition also had 2 failures out of 15. Most participants rated by how close the ball came to the cup, and all of them were in successful sessions, while all participants who rated each attempt relative to the previous one produced premature convergence. Ratings correlated with camera-measured distance at about 0.72 on average, and this correlation was significantly higher in successful than in failed sessions.
The limits
What it doesn't show
Only one task on one robot was tested, so it is unknown whether rating-based learning works for other skills. The comparison is a non-significant difference with small groups, which is not proof that the two setups are equivalent. Users were in a lab with an experimenter present, not at home, and several found the near-identical attempts within a batch confusing or gave identical scores that carry no learning signal. The strategy-outcome links rest on self-report and very small subgroups.
Key terms
- Dynamic movement primitives (DMPs)
- A way of representing a robot movement as a stable spring-damper system plus a learnable forcing term that shapes the trajectory.
- CMA-ES
- A black-box optimisation method that samples variations around a mean, ranks them by cost, and shifts the sampling distribution toward better ones.
- Cost function
- A numeric score the learner tries to minimise; here either measured ball-cup distance or an inverted user star rating.
- Programming by demonstration
- Teaching a robot by showing it a movement, which it then reproduces and may refine.
- Premature convergence
- When an optimiser stops exploring and settles on a solution before reaching the goal.
Flashcards
0 of 11 answers reviewed
Research intelligence for this paper
See its role on concept claims, tensions it is part of, placement history, and related discoveries.
Quiz yourself
What did naive users provide to the learning algorithm?
Common questions
Why use people instead of a camera to score attempts?
Designing a reliable cost function and sensor setup is hard even for experts and impractical in homes; users already know what success looks like.
Which rating strategy worked best?
Rating how close the ball came to the cup; comparing each try only to the previous one led to premature convergence.
Were human ratings noisy?
Somewhat, yet the unmodified algorithm still learned; ratings tracked true distance fairly well on average.
More on Robot learning