The concern whether or not AI powered search is going to be reliable started with Google’s embarrassing fail in one of their first demos of its AI driven upgrade to Google Search, called Bard. While the initial concern has been around the large language models and the new AI technology in general, the attention has turned to the current state of Google and the litany of inaccuracies floating through search results search results. We have a theory of what is going on and what is to blame, and that is Google Assistant.
Google’s Overreliance of Rich Snippets
One common thread that we have seen in this are examples showing rich snippets. Rich snippets are structured data that Google extracts from web pages and displays in a special format on the search results page. Google Assistant, uses rich snippets to provide quick and concise voice search answers to users’ queries. For example, if you ask Google Assistant “What is the abbreviation for Nevada?”, it will show you a rich snippet that says “NV”. Or that’s what you would expect to see…
The reality is that up until a few days ago half after several people on Twitter pointed it out, if you were to search or ask Google Assistant “What is the abbreviation for Nevada?”, it would show you a rich snippet that says “GA”.
In fact, if you go through this thread you will see the litany of errors Google would give to this. I did a few screenshots of my results.
You’ll see that it’s not just limited to Nevada. If you did a search for Colorado Wyoming and more it would fall apart. In the example of Colorado, it would go with the 1874 state abbreviation and not the modern “CO” abbreviation. Both the CO and GA examples are quite egregious in that it is querying government websites. It shows that it doesn’t have a clear understanding of what data it’s looking at and just spits out information seemingly at random. One possible explanation is that Google Assistant is using a poor source of information for its rich snippets. For example, it appears that the FAA web page that Google gets the incorrect Nevada snippet from has a table of state abbreviations, but the table is formatted in such a way that it could cause data parsing errors.
In those screenshots, you’ll also notice the difference in what Microsoft is providing. They don’t allow for ambiguity and have made a partnership with Wikipedia as well as pulling from Census data to get proper abbreviations within their results along with state flags.
Quality of Search Results vs. Overreach
There has long been a debate on how much search engine providers like Google and Microsoft usurp search results to provide a definitive answer at the very top of the search results page, but I don’t think those arguments are in basic data like state abbreviations. There’s no debate that should come up in a search for what the two-letter abbreviation is for and about, there’s only one answer, what is the abbreviation. There have been abbreviations used in the past but that’s not what the search intent was. So that the plate becomes somewhat moot when search data is used to produce voice search answers for something like Google Assistant. Either way, the result is that Google Assistant is providing wrong information to users, which can have negative consequences for their voice search experience. For example, users might:
- Lose trust in Google Assistant and voice search in general, and switch to other sources of information or methods of searching.
- Make wrong decisions or take wrong actions based on the incorrect information, such as booking a flight to the wrong destination or sending a letter to the wrong address.
- Miss out on relevant or useful information that Google Assistant could have provided, such as the history, culture, or attractions of Nevada.
- Waste time or money trying to correct or verify the incorrect information, such as calling customer service or visiting another website.
Therefore, it is important for Google and other search providers to ensure that their rich snippets are accurate, relevant, and up-to-date. They should also provide users with ways to report or flag incorrect or misleading information, and to access alternative or additional sources of information. Otherwise, voice search might lose its appeal and credibility, and users might lose their confidence and satisfaction.
