Here is a fun math problem. Google processes more than five trillion searches per year. Its AI Overviews feature now sits at the top of most results pages, confidently summarizing the internet for you. A new analysis commissioned by The New York Times found that AI Overviews gives the wrong answer roughly 10% of the time. Do the multiplication and you land somewhere in the neighborhood of tens of millions of incorrect answers flowing out of Google every single day.

The testing was done by Oumi, an AI startup, using OpenAI's SimpleQA benchmark. They ran 4,326 questions through AI Overviews and found an 85% accuracy rate when Google was running Gemini 2. After the Gemini 3 upgrade, that improved to 91%. Progress? Sure. But a 9% error rate at Google's scale is not a rounding error. It is an industrial-grade misinformation machine.

The examples are almost comedic. When asked what date Bob Marley's former home became a museum, AI Overviews cited three pages, two of which said nothing about the date, and then confidently picked the wrong year from the one that did. When asked about Yo Yo Ma's Classical Music Hall of Fame induction, it cited the organization's own website and then claimed the hall of fame does not exist.

Google's response tells you everything. Spokesperson Ned Adriance told the Times the study "has serious holes" and "doesn't reflect what people are actually searching on Google." Translation: please do not test our product with standardized tests because the results are embarrassing.

Here is the deeper problem. Google does not even run its best model for most searches. AI Overviews uses faster, cheaper Gemini Flash models most of the time because running the full Gemini 3.1 Pro on every query would be too slow and too expensive. So the feature that replaced the blue links people trusted for 25 years is running on the budget model. And it shows.

The uncomfortable truth is that 90% accuracy sounds good until you remember what search engines are for. People do not go to Google for entertainment. They go for facts. A search engine that is wrong 10% of the time is like a doctor who is wrong 10% of the time. You would not keep going to that doctor. But Google has no competition, so you keep going to Google.

The real question nobody at Google wants to answer: at what accuracy threshold does an AI summary become a net negative? If AI Overviews confidently serves wrong answers to hundreds of millions of people per day, and those people trust it because it looks authoritative, Google has not improved search. It has made the internet dumber. At scale.