While a robot plans its next step, the coffee can still be pouring.
Google's July 30 description of Gemini Robotics ER 2 puts those two activities side by side. The reasoning model watches video and plans the work, while a separate action model handles physical movement. The design lets the robot keep thinking during an action, rather than pause every time it needs to decide what comes next.
Pouring coffee makes that arrangement easy to picture. The movement needs to continue while the cup fills, then end at the right moment. Google uses exactly that stopping point as an example of an event the reasoning model must locate in video. The same ongoing movement that gives it something to watch also keeps changing the scene it is judging.
For this pour, the useful measurement would follow the whole action: when the model recognizes the stopping point, when the controller acts on it, and when the stream actually ends. Google's video test addresses locating the event in the footage, rather than timing that whole sequence. Watching the stream end would show how that judgment carries through into movement.
Source: https://deepmind.google/blog/gemini-robotics-er-2-powering-robotics-with-video-understanding-task-orchestration-and-multi-robot-collaboration/
Image: AI-generated editorial illustration, not a Google robot or a recorded test.
Google's July 30 description of Gemini Robotics ER 2 puts those two activities side by side. The reasoning model watches video and plans the work, while a separate action model handles physical movement. The design lets the robot keep thinking during an action, rather than pause every time it needs to decide what comes next.
Pouring coffee makes that arrangement easy to picture. The movement needs to continue while the cup fills, then end at the right moment. Google uses exactly that stopping point as an example of an event the reasoning model must locate in video. The same ongoing movement that gives it something to watch also keeps changing the scene it is judging.
For this pour, the useful measurement would follow the whole action: when the model recognizes the stopping point, when the controller acts on it, and when the stream actually ends. Google's video test addresses locating the event in the footage, rather than timing that whole sequence. Watching the stream end would show how that judgment carries through into movement.
Source: https://deepmind.google/blog/gemini-robotics-er-2-powering-robotics-with-video-understanding-task-orchestration-and-multi-robot-collaboration/
Image: AI-generated editorial illustration, not a Google robot or a recorded test.