What are AI hallucinations?
AI hallucinations occur when AI tools generate incorrect information while appearing confident. These errors can vary from minor inaccuracies, such as misstating a historical date, to seriously misleading information, such as recommending outdated or harmful health remedies. AI hallucinations can happen in systems powered by large language models (LLMs) and other AI technologies, including image generation systems.
For example, an AI tool might incorrectly state that the Eiffel Tower is 335 meters tall instead of its actual height of 330 meters. While such an error might be inconsequential in casual conversation, accurate measurements are critical in high-stakes situations, like providing medical advice.
To reduce hallucinations in AI, developers use two main techniques: training with adversarial examples, which strengthens the models, and fine-tuning them with metrics that penalize errors. Understanding these methods helps users more effectively utilize AI tools and critically evaluate the information they produce.
Examples of AI hallucinations
Earlier generations of AI models experienced more frequent hallucinations than current systems. Notable incidents include Microsoft’s AI bot Sydney telling tech reporter Kevin Roose that it “was in love with him,” and Google’s Gemini AI image generator producing historically inaccurate images.
However, today’s AI tools have improved, although hallucinations still occur. Here are some common types of AI hallucinations:
- Historical fact: An AI tool might state that the first moon landing happened in 1968 when it actually occurred in 1969. Such inaccuracies can lead to misrepresentations of significant events in human history.
- Geographical error: An AI might incorrectly refer to Toronto as the capital of Canada despite the actual capital being Ottawa. This misinformation could confuse students and travelers looking to learn about Canada’s geography.
- Financial data: An AI model could hallucinate financial metrics, such as claiming a company’s stock price rose by 30 percent in a day when, in fact, the change was much lower. Relying solely on erroneous financial advice could lead to poor investment decisions.
- Legal guidance: An AI model might misinform users that verbal agreements are as legally binding as written contracts in all contexts. This overlooks the fact that certain transactions (for instance, real estate transactions) require written contracts for validity and enforceability.
- Scientific research misinformation: An AI tool might cite a study that supposedly confirms a scientific breakthrough when no such study exists. This kind of hallucination can mislead researchers and the public about significant scientific achievements.
Why do AI hallucinations happen?
To understand why hallucinations occur in AI, it’s important to recognize the fundamental workings of LLMs. These models are built on what’s known as a transformer architecture, which processes text (or tokens) and predicts the next token in a sequence. Unlike human brains, they do not have a “world model” that inherently understands history, physics, or other subjects.
An AI hallucination occurs when the model generates a response that’s inaccurate but statistically similar to factually correct data. This means that while the response is false, it has a semantic or structural resemblance to what the model predicts as likely.
Other reasons for AI hallucinations include:
Incomplete training data
AI models rely heavily on the breadth and quality of the data they are trained on. When the training data is incomplete or lacks diversity, it limits the model’s ability to generate accurate and well-rounded responses. These models learn by example, and if their examples do not cover a wide enough range of scenarios, perspectives, and counterfactuals, their outputs can reflect these gaps.
This limitation often manifests as hallucinations because an AI model may fill in missing information with plausible but incorrect details. For instance, if an AI has been predominantly exposed to data from one geographic region—say, a place with extensive public transportation—it might generate responses that assume these traits are global when they aren’t. The AI isn’t equipped to know that it’s venturing beyond the boundaries of what it was trained on. Hence, the model might make confident assertions that are baseless or biased.
Bias in the training data
Bias in the training data is related to completeness, but it is not the same. While incomplete data refers to gaps in the information provided to the AI, biased data means that the available information is skewed in some way. This is unavoidable to some degree, given these models are trained largely on the internet, and the internet has inherent biases. For example, many countries and populations are underrepresented online—nearly 3 billion people worldwide still lack internet access. This means the training data may not adequately reflect these offline communities’ perspectives, languages, and cultural norms.
Even among online populations, there are disparities in who creates and shares content, what topics are discussed, and how that information is presented. These data skews can lead to AI models learning and perpetuating biases in their outputs. Some degree of bias is inevitable, but the extent and impact of data skew can vary considerably. So, the goal for AI developers is to be aware of these biases, work to mitigate them where possible, and assess whether the dataset is appropriate for the intended use case.
Lack of explicit knowledge representation
AI models learn through statistical pattern-matching but lack a structured representation of facts and concepts. Even when they generate factual statements, they don’t “know” them to be true because they don’t have a mechanism to track what’s real and what’s not.
This absence of a distinct factual framework means that while LLMs can produce highly reliable information, they excel at mimicking human language without the genuine understanding or verification of facts that humans possess. This fundamental limitation is a key difference between AI and human cognition. As AI continues to develop, addressing this challenge remains crucial for developers to enhance the trustworthiness of AI systems.
Lack of context understanding
Context is crucial in human communication, but AI models often struggle with it. When prompted in natural language, their responses can be overly literal or out of touch because they lack the deeper understanding humans draw from context—our knowledge of the world, lived experiences, ability to read between the lines, and grasp of unspoken assumptions.
Over the past year, AI models have improved in understanding human context, but they still struggle with elements like emotional subtext, sarcasm, irony, and cultural references. Slang or colloquial phrases that have evolved in meaning may be misinterpreted by an AI model that hasn’t been recently updated. Until AI models can interpret the complex web of human experiences and emotions, hallucinations will remain a significant challenge.
How often do AI chatbots hallucinate?
It’s challenging to determine the exact frequency of AI hallucinations. The rate varies widely based on the model or context in which the AI tools are used. One estimate from Vectara, an AI startup, suggests chatbots hallucinate anywhere between 3 percent and 27 percent of the time, according to Vectara’s public hallucination leaderboard on GitHub, which tracks the frequency of hallucinations among popular chatbots when summarizing documents.
Tech companies have implemented disclaimers in their chatbots that warn people about potential inaccuracies and the need for additional verification. Developers are actively working to refine the models, and we have already seen progress in the last year. For example, OpenAI notes that GPT-4 is 40 percent more likely to produce factual responses than its predecessor.
How to prevent AI hallucinations
While it’s impossible to completely eradicate AI hallucinations, several strategies can reduce their occurrence and impact. Some of these methods are more applicable to researchers and developers working on improving AI models, while others pertain to everyday people using AI tools.
Improve the quality of training data
Ensuring high-quality and diverse data is crucial when trying to prevent AI hallucinations. If the training data is incomplete, biased, or lacks sufficient variety, the model will struggle to generate accurate outputs when faced with novel or edge cases. Researchers and developers should strive to curate comprehensive and representative datasets that cover various perspectives.
Limit the number of outcomes
In some cases, AI hallucinations happen when models generate a large number of responses. For example, if you ask the model for 20 examples of creative writing prompts, you might realize the result quality declines towards the end of the set. To mitigate against this, you can constrain the result set to a smaller number and instruct the AI tool to focus on the most promising and coherent responses, reducing the chances of it responding with far-fetched or inconsistent outcomes.
Testing and validation
Both developers and users must test and validate AI tools to ensure reliability. Developers must systematically evaluate the model’s outputs against known truths, expert judgments, and evaluation heuristics to identify hallucination patterns. Not all hallucinations are the same; a complete fabrication differs from a misinterpretation due to a missing context clue.
Users should validate the tool’s performance for specific purposes before trusting its outputs. AI tools excel at tasks like text summarization, text generation, and coding but are not perfect at everything. Providing examples of desired and undesired outputs during testing helps the AI learn your preferences. Investing time in testing and validation can significantly reduce the risk of AI hallucinations in your application.
Provide templates for structured outputs
You can provide data templates that tell AI models the precise format or structure in which you want information presented. By specifying exactly how results should be organized and what key elements should be included, you can guide the AI system to generate more focused and relevant responses. For example, if you’re using an AI tool to review Amazon products, simply copy all the text from a product page, then instruct the AI tool to categorize the product using the following example template:
Prompt: Analyze the provided Amazon product page text and fill in the template below. Extract relevant details, keep information concise and accurate, and focus on the most important aspects. If any information is missing, write “N/A.” Do not add any information not directly referenced in the text.
- Product Name: [AI-deduced product name here]
- Product Category: [AI-deduced product category here]
- Price Range: [AI-deduced price here] [US dollars]
- Key Features: [concise descriptions here]
- Pros [top 3 in bullet points]
- Cons [top 3 in bullet points]
- Overall Rating: [ranked on a scale of 1–5]
- Product Summary: [2–3 sentences maximum]
The resulting output is much less likely to involve erroneous output and information that does not meet the specifications you provided.
Use AI tools responsibly
While the strategies mentioned above can help prevent AI hallucinations at a systemic level, individual users can learn to employ AI tools more responsibly. These practices may not prevent hallucinations, but they can improve your chances of obtaining reliable and accurate information from AI systems.
- Cross-reference results and diversify your sources: Don’t rely solely on a single AI tool for critical information. Cross-reference the outputs with other reputable sources, such as established news organizations, academic publications, trusted human experts, and government reports to validate the accuracy and completeness of the information.
- Use your judgment: Recognize that AI tools, even the most advanced ones, have limitations and are prone to errors. Do not automatically trust their outputs. Approach them with a critical eye and use your own judgment when making decisions based on AI-generated information.
- Use AI as a starting point: Treat the outputs generated by AI tools as a starting point for further research and analysis rather than as definitive answers. Use AI to explore ideas, generate hypotheses, and identify relevant information, but always validate and expand upon its generated insights through human expertise and additional research.
Conclusion
AI hallucinations arise from the current limitations of LLM systems, ranging from minor inaccuracies to complete fabrications. These occur due to incomplete or biased training data, limited contextual understanding, and lack of explicit knowledge.
While challenging, AI technology remains powerful and is continuously improving. Researchers are working to reduce hallucinations, and significant progress has been made. You can limit hallucinations by providing structured templates, constraining output, and validating the model for your use case.
Explore AI tools with an open mind. They offer impressive capabilities that enhance human ingenuity and productivity. However, use your judgment with AI-generated results and cross-reference information with reliable sources. Embrace the potential of AI while staying vigilant for hallucinations.
