Reinforcement learning
Can curiosity drive a humanoid robot to learn how its body moves?
Open access · cc by · source: Europe PMC
Rewarding a real humanoid robot for improving its own model of how its movements turn out made it explore more efficiently than random or least-tried exploration, and led it to discover and probe a table on its own.
Study at a glance
- Design
- Computational / modelling — Two online experiments on a real iCub robot: one curious learner for an arm compared with random and least-tried exploration; three parallel curious learners for both arms and torso at a table.
- N
- No sample size in the usual sense: Experiment 1 used an arm model with 1296 state-actions and ran for 4000 transitions; Experiment 2 ran three learners for 18,000 transitions.
- Population
- A single physical iCub humanoid robot (arm and torso joints)
- Outcome
- Exploration coverage of states and state-actions, uniformity of visits, decay of intrinsic reward, emergent behaviour
Structured fields used in claim comparison tables when every cited study has a complete layer.
Key findings
The curious agent explored best: it reached novel states more aggressively and spread its visits and action tries more evenly than the random and least-tried agents. It learned that the robot's cable-driven mechanics make motions unreliable in places, which the authors argue makes plan-then-act planners unsuitable. With three learners, the arms and torso ended up coordinating to lean forward and touch the table repeatedly, because actions affected by the other learners stayed surprising longest.
Methodology
The authors divided the iCub robot's joint space into a grid of states, where each action aims to move to a neighbouring state, and learned the probability of where each action actually ends up. The only reward was intrinsic: how much a new observation changed the learned transition model, measured with KL divergence. In one experiment a single learner controlled four arm joints and was compared with random and least-tried exploration; in a second, three independent curious learners controlled the torso and both arms while the robot sat at a table.
Limitations
All evidence comes from one robot with coarse state grids (three positions per joint), and comparisons are shown mainly through figures and qualitative observation rather than statistical tests or repeated runs. The 'table exploration' is an emergent behaviour described by the authors, not measured against a baseline. Learning took many hours on hardware, and the authors note that finer state spaces would make it slower still; the planner was not evaluated on external goal-reaching tasks.
How this study connects
Role on claims
Each row is a claim on a concept or method page where this paper supports, challenges, or qualifies the statement. Roles are hand-checked — not a model guess.
Curiosity, inheritance and few-shot examples are viable but lightly tested.
Alternative learning routes also work in constrained settings: inheriting learned controllers (Lamarckian evolution) produced faster simulated robots under small learning budgets, a curiosity-driven humanoid explored more evenly than random agents, and a neuromorphic robot learned to turn from ten positive examples.
Evidence for the claim as stated.
Related papers in this topic
Same topic cluster — not a recommendation engine.