Disclaimer: This article is based on actual news from the real world – honestly! However, it has been sprinkled with a healthy dose of satire.
MOUNTAIN VIEW, Calif. — Google’s AI Overviews, the Gemini-powered summary pinned to the top of every search result page, correctly answers questions 91 percent of the time, according to a New York Times analysis conducted with the AI startup Oumi. This represents a six-point improvement over last year’s 85 percent accuracy and, extrapolated across Google’s daily query volume, amounts to roughly tens of millions of incorrect answers per day. Or, if you prefer, one very large reference book being written in real time by someone who has not read it. No word on how many Rhode Islands this constitutes.
We're no medical experts, but we're quite certain this is a 'hallucination'. (Matt Rose/YouTube)
Asked when Bob Marley’s former home became a museum, AI Overviews cited three sources. Two of them did not mention the date at all. The third, Wikipedia, listed two contradictory years, and AI Overviews selected the wrong one with the calm authority of someone who has never been uncertain about anything in their life. Asked when Yo-Yo Ma was inducted into the Classical Music Hall of Fame, the model cited the hall’s own website, confirming Ma’s induction, then informed the user that the Classical Music Hall of Fame does not exist.
Google spokesperson Ned Adriance told the Times the study “has serious holes” and “doesn’t reflect what people are actually searching on Google.” Asked what people are actually searching on Google, Adriance cited three pages, two of which did not contain the answer and one of which contained two contradictory answers. He then selected the wrong one with great confidence.
A spokesman clarified that the best model existed but could not be deployed at scale for economic reasons. The cheaper model, he noted, was only wrong about a billion times a week, which under current accounting rules is classified as overhead. The difference is recovered in ad revenue stolen and then misinterpreted from online sources Google no longer sends traffic to, presumably.
Every overview concludes with a disclaimer reading “AI can make mistakes, so double-check responses,” a sentence Google considers legally sufficient and somehow operationally acceptable.
At press time, AI Overviews was asked whether AI Overviews makes mistakes. It cited the disclaimer, summarized the disclaimer incorrectly, and offered to refine its answer incorrectly. Its findings will be summarized by AI Overviews. Purple monkey dishwasher.
This story is based on fully factual news, but if we got it wrong, blame these guys, we’re just here to make it funny.
