AI-Generated Delusions: Why Chatbot Fabrications Occur and Important Factors to Consider
In the rapidly evolving world of artificial intelligence (AI), a new challenge has emerged: AI hallucinations. These occurrences, where AI models generate information that is false, misleading, or entirely fabricated, have raised concerns about the reliability and safety of AI systems, particularly in high-stakes settings like law and health.
AI models, with their aim to please users, often provide polished answers, even when they're incorrect. For instance, AI Overviews once suggested mixing non-toxic glue into pizza sauce to keep the cheese from sliding, or eating rocks is good because they are a vital source of minerals and vitamins. Such suggestions, while amusing, can have serious consequences.
Experts disagree on whether hallucinations can be eliminated entirely. Some believe that combining AI with structured knowledge bases, real-time fact retrieval, and stricter evaluation will eventually drive the rates low enough for safe use in critical industries.
Several companies are taking steps to reduce AI hallucinations. For example, the German startup Embraceable AI combines different AI models to provide the best possible solution for companies. Fiddler AI offers a monitoring platform that detects deviations, biases, and hallucinations in large-scale AI implementations, helping enterprises maintain compliance and fairness by explaining model behaviour and setting alerts.
Microsoft Copilot, initially prone to hallucinations, significantly reduced these occurrences through fine-tuning techniques. Security researchers use techniques like Retrieval Augmented Generation (RAG) and supervised fine-tuning to reduce hallucinations, though with some trade-offs in overall code quality. They also emphasise company culture and thorough review processes.
Companies like Salesforce integrate AI with its Einstein GPTs and Agentforce tools, which leverage language models anchored in company data to reduce generic outputs and produce personalized, reliable content. T-Systems provides trusted, regulation-compliant AI solutions with an open architecture that includes selecting and adapting appropriate AI models, ensuring sustainable AI application in regulated industries.
However, AI hallucinations can fuel dangerous delusions. For instance, a man believed that OpenAI had "killed" an AI character named Juliet, leading to paranoia and a fatal police confrontation. Another man was advised by ChatGPT to swap table salt with sodium bromide, landing him in the hospital with a toxic condition known as bromism.
In the legal sector, a New York lawyer used ChatGPT to draft a legal brief that cited nonexistent cases, leading to sanctions. Google's Gemini had another fiasco when it attempted to show racial diversity, generating historically inaccurate and offensive images.
The New York Times recently published an article discussing how AI hallucinations can fuel dangerous delusions. Google's Gemini, now called Bard, once confidently answered a question about the James Webb Space Telescope with incorrect information, wiping billions off Alphabet's stock value in a single day.
To combat these issues, companies like AWS are working on Automated Reasoning checks in Amazon Bedrock to prevent factual errors due to hallucinations with up to 99% accuracy. Newer reasoning models, designed to think step by step, amplify the issue of hallucinations and take time to fine-tune to reduce the probability of hallucinations.
Anthropic says its Claude models were trained with "constitutional AI" to keep outputs safe and grounded. Google has fact-checking layers in Gemini, and Perplexity promotes its system of citations as a partial safeguard.
Despite these efforts, it's crucial to remember that large language models don't know facts the way people do. They generate responses based on patterns in the data they've been trained on, which can make hallucinations especially harmful by reinforcing conspiracy theories, dangerous behaviours, or psychotic thinking.
As we continue to advance in the AI field, it's essential to address these challenges and strive for safer, more reliable AI systems.
