Seeing someone use Gemini to plan a hike, being advised to bring “much less than what’s actually needed” for food and water, and then eventually getting trapped in the mountains awaiting rescue—I didn’t just laugh at the user. I actually got chills.

After writing code for so many years, we know all too well the issues with these models: the more confident the tone, the more wildly wrong the answer. Before, users blamed the search engine for serving the wrong page; now they might blame you for a list that isn’t enough to keep them alive.

A product can’t just know how to answer—it also needs to know when to shut up. Those are two different things.