AI Chatbot Recommendations Vary, Urging User Caution Before Purchases
Translated & summarized from Bizportal by baba
Consumers are increasingly using AI chatbots for purchase recommendations, but a study found these suggestions can vary significantly even for identical queries. Researchers analyzed responses from ChatGPT, Gemini, and Google AI, noting inconsistencies in product lists and cited sources. While AI chatbots use personal language to build trust, users are advised to critically evaluate recommendations, verify information, and compare options to avoid making uninformed purchases.
The story in 5 lines · by baba
- AI chatbot product recommendations are inconsistent, with the same query yielding different results.
- A study found ChatGPT had the lowest recommendation overlap, while Google's summaries had the highest.
- ChatGPT frequently uses first-person language like "I recommend" in its product suggestions.
- The AI's decision-making process for recommendations is not fully transparent to users.
- Users are advised to critically evaluate AI suggestions and verify information before purchasing.
Consumers increasingly rely on AI chatbots like ChatGPT and Gemini for product recommendations before making significant purchases, a trend that has become almost second nature. These tools offer quick comparisons and justifications, often stating what they "would buy." However, a recent study highlights that these recommendations are not consistent; the same query can yield entirely different product lists and justifications on subsequent attempts.
A September 2026 study by researchers from Maastricht, Utrecht, and Zurich universities analyzed 1,536 responses to 117 user queries sent three times each to ChatGPT, Gemini, and Google's AI summaries. The experiment revealed significant variation, with ChatGPT showing the lowest overlap in recommendations and Google's summaries the highest. Even the sources cited alongside the answers changed, meaning users might miss out on suitable products if they don't re-query or explore alternatives.
The study also noted differences in how recommendations were phrased. ChatGPT used first-person language like "I recommend" in 79% of its product recommendations, compared to 7% for Gemini and 2% for Google. While this personal tone can make the interaction feel more human and trustworthy, researchers caution that it doesn't indicate the AI has personally tested the product. The AI's goal is to simulate a personal conversation, making users feel more comfortable asking follow-up questions.
Researchers found that the AI's selection process is not fully transparent. While sources are provided, the exact algorithms and weighting of factors influencing the final recommendation remain unclear. The study did not prove that recommendations are sponsored or definitively rank product quality. However, it suggests that users should critically evaluate AI suggestions, ask for detailed justifications, verify information, and compare alternatives, especially regarding pricing, features, and local purchasing conditions like official versus parallel import in Israel.