MIT Robot Learns the Game of Jenga

MIT news writes that teaching a robot to play Jenga with traditional machine learning schemes require 10s of thousands of tested attempts — not too bad until you consider the number of human hours it would take to build 54 blocks in perfect perpendicular 3 X 3 stacks 10s of thousands of times. however, the engineers at MIT have found a short cut around all that.

The robot, developed by MIT engineers, is equipped with a soft-pronged gripper, a force-sensing wrist cuff, and an external camera, all of which it uses to see and feel the tower and its individual blocks.

As the robot carefully pushes against a block, a computer takes in visual and tactile feedback from its camera and cuff and compares these measurements to moves that the robot previously made. It also considers the outcomes of those moves — specifically, whether a block, in a certain configuration and pushed with a certain amount of force, was successfully extracted or not. In real-time, the robot then “learns” whether to keep pushing or move to a new block, in order to keep the tower from falling.

Details of the Jenga-playing robot are published today in the journal Science Robotics. Alberto Rodriguez, the Walter Henry Gale Career Development Assistant Professor in the Department of Mechanical Engineering at MIT, says the robot demonstrates something that’s been tricky to attain in previous systems: the ability to quickly learn the best way to carry out a task, not just from visual cues, as it is commonly studied today, but also from tactile, physical interactions.

Robot Setup

(A) Physical setup consisting of the robot, Jenga tower, Intel RealSense D415 camera, and ATI Gamma force/torque sensor (mounted at the wrist). (B) Machine intelligence architecture with the learned physics model.

In all, it took the robot just 300 tries to learn the game. granted, it’s poking and prying the blocks and there are many unreasonable friends out there who might call this technique cheating. official rules of jenga says otherwise. this robot is completely in its right to poke and test blocks for ease of extraction before stacking.