Google’s latest AI model teaches robots to throw away trash. It’s an essential step toward making robots more helpful and responsive to the needs of people in their environments.
While robots can perform basic tasks that they’re programmed to do, they struggle to adapt to changes and deal with new situations that they haven’t been specifically trained for. Traditional robot control systems use complex stacks of high-level reasoning and low-level manipulation systems that play an imperfect game of telephone to translate human commands into the robots’ movements.
Only now, teaching robots to perform general tasks has been a time-consuming and costly process. This is because robots require massive training data to learn how to recognize the many different objects and scenarios they might encounter. In addition to learning how to identify objects, robots also need to learn how to move and pick up items mechanically and distinguish between different types of trash.
In a blog post, Google introduced the Robotics Transformer 2 (RT-2), an innovative artificial intelligence model that trains robots to perform real-world actions. RT-2 is a vision-language-action model that expertly analyzes text and images to understand the meaning of human instructions. Then, it transfers concepts embedded in language and vision training data to direct robots on performing specific actions—even for tasks the robot has never been explicitly trained for.
To demonstrate the capabilities of RT-2, researchers used a robot from Everyday Robots, another company under Google’s umbrella, to conduct several experiments with the public. In one demonstration, they asked the robot to decide whether an improvised hammer was a rock or a nail and then choose the appropriate tool to fulfill that request. The robot accurately identified the improvised tool as a rock and correctly chose the correct tool to pick it up. In another experiment, the robot was asked to move a Coke can toward a picture of Taylor Swift—it did so successfully.
While these experiments offer promising glimpses into the future of robotics, it’s worth noting that RT-2 isn’t yet ready for commercial deployment. The system must adjust to different languages and environments before it can be widely adopted. This process will be accelerated by introducing a neural network capable of understanding and processing a more comprehensive range of inputs. The technology is a significant step forward from the system developed by Google’s parent company, Alphabet, which merged its language model LLM PaLM with physical robotics to create the awkwardly named PaLM-SayCan system. The ability of robots to better interpret human language and visual information could open up a wide variety of possibilities for the next generation of home and business automation. The future of robotics is looking brighter than ever.
