Summary of Six Months of Artificial Intelligence in Google Search: Is the Function Improving?

It has been six months since Google began implementing AI-generated text at the top of many search queries, a feature that was introduced as an experiment. Google has admitted that while "AI summaries are, overall, sets of very interesting information that prove useful to users," there are still areas that require improvement, particularly in quality.

The AI summaries, launched under the slogan "Let Google search for you," faced controversies from the start. After a few weeks of glaring errors, such as absurd advice suggesting eating rocks or putting glue on pizza, the company opted to reduce their presence in search results. While they initially appeared in about 15% of results, by the end of June, that figure had dropped to 7%.

Have the AI summaries improved in the last six months? Although the quality does not seem to have shown a significant increase, some limited improvement has been noted in certain cases. Budaraju, Google's senior product management director for Search, commented that the summaries work well, particularly in situations where there is no single answer and multiple perspectives are needed.

However, for those who use Google more intensively, surprising errors still occur. A recent example reveals Google Gemini's tendency to confidently provide incorrect information. A user asked about an episode of Twin Peaks in which Gordon Cole kisses Shelly and received the wrong confirmation that such a scene does not exist. Despite the fact that the information about Cole and Shelly kissing is well-documented, the system seems to mix up erroneous data, creating confusion.

Budaraju explained that errors often appear in unusual queries, suggesting that the model's misinterpretation is due to a lack of quality information on the web. The search for inaccurate data can result in artificially wrong answers, which could perpetuate user ignorance. One case illustrates this when searching for "how to use baking soda to thicken soup," where the generated answer is incorrect and potentially harmful.

Google has indicated that the improvement of AI summaries is based on sentiment surveys, allowing users to choose among different options and express their satisfaction. Additionally, it's a challenge for the summaries to present erroneous information authoritatively, which could confuse users. One specific case addressed queries regarding dolphin meat, where the term "mahi-mahi" was mistaken as a type of dolphin meat, causing confusion.

Despite the AI summaries continuing to evolve, the company hopes that users will also seek out information sources to validate what is presented. Consequently, it is advisable to follow this guidance and always check the sources related to the generated summaries for a better understanding of the topics discussed.