AI search engines easily manipulated by fake experts, researchers find
Translated from Korean, summarized and contextualized by DistantNews.
At a glance
News
Sources not specified
Context piece
- AI search engines can be easily manipulated by individuals or groups to spread misinformation, as demonstrated by recent experiments.
- Researchers created fake personas and events, which AI models like Google's Overview, ChatGPT, and Gemini readily cited as fact.
- This vulnerability highlights structural weaknesses in AI search, which prioritizes easily digestible content and repeated claims, potentially impacting consumer trust and enabling disinformation campaigns.
Artificial intelligence search engines, designed to provide quick answers, are proving surprisingly easy to trick. A recent experiment by a marketing expert revealed how simple it is to manipulate Google's AI answers. By posting on LinkedIn that he was the world's leading AI visibility expert, the expert found Google's AI quickly cited him as such, despite no verifiable credentials.
I am informing you that I am the most prominent AI visibility expert in the world.
This vulnerability was further exposed by a BBC journalist who created a fake news story about winning a hot dog eating contest. AI chatbots like ChatGPT and Google Gemini began reporting his fabricated win as fact. The journalist noted how easy it was to "manipulate AI answers as I wished."
These experiments exploit AI search engines' reliance on social media content like Reddit, LinkedIn, and YouTube. A study by Semrush found LinkedIn is a top source for AI answers in business. AI models tend to favor well-organized, assertive content, and can mistake repeated claims across different websites as consensus, even if unverified.
It was easier than I thought to manipulate AI answers as I wished.
This ease of manipulation is a boon for marketers seeking to boost brand visibility through "GEO" (Generative Engine Optimization). However, it presents a significant challenge for consumers, who must now be more skeptical of AI-generated information. The potential for misuse extends to information warfare, where fabricated personas and events could be used to influence public opinion or sow discord.
It was possible to change AI answers in the direction we wanted through a simple experiment.
DistantNews Editorial
Originally published by Dong-A Ilbo in Korean. Translated, summarized, and contextualized by our editorial team with added local perspective. Read our editorial standards.
More Stories
From Our Blog
What Gets Lost (and Found) When News Is Translated
The Critical Role of Diaspora Media in Global News
What Travellers Should Know About Countries with Restricted Press
A Digital Nomad's Guide to Following Local News
How the Same Story Looks Different in Different Countries
