OWASP LLM09:2025 – Misinformation: When AI Spreads Falsehoods with Confidence

As language models become more integrated into search engines, chatbots, and content creation tools, their credibility becomes a double-edged sword. While they can write fluently and convincingly, what happens when the content is confidently wrong?

This is the core of OWASP LLM09:2025 – Misinformation. When an LLM confidently outputs false information, the result can be damaging. It may misinform a user. It can also reinforce stereotypes or generate fake citations.

Let’s dive into what misinformation in LLMs really means, why it happens, and what can be done about it.

What Is Misinformation in AI?

LLMs are pattern prediction engines, not truth engines. They don’t “know” facts—they predict the most likely sequence of words based on their training data.

The training data may include outdated, biased, or fabricated content. The model might also hallucinate missing information. Both cases can generate misinformation that sounds trustworthy but is factually incorrect.

Real-World Impact of LLM Misinformation

• Medical Misadvice: An AI chatbot tells a user that mixing two medications is safe—when it isn’t.
• Fake Legal Precedents: LLMs generate non-existent court rulings with convincing names and citations.
• Election Disinformation: An LLM outputs manipulated narratives that affect voter perception.

Even if these outputs are not intentionally malicious, they can still lead to serious real-world harm.

Why LLMs Hallucinate or Mislead

1. Lack of Fact-Checking Mechanism: LLMs generate text, but they don’t verify the truth.
2. Training on Flawed Data: Garbage in, garbage out. Biased, spammy, or incorrect data leads to unreliable responses.
3. Overconfidence Bias: Language models often respond confidently even when unsure.
4. Prompt Sensitivity: Small changes in phrasing can flip the output from accurate to false.
5. No Awareness of Time: LLMs often don’t know current events or breaking news unless connected to external tools.

How to Reduce Misinformation Risks

1. Add a Retrieval Layer

Use RAG (Retrieval-Augmented Generation) to pull from trusted sources in real-time, like academic journals or verified databases.

2. Response Confidence Indicators

Display uncertainty or source citations instead of confident-sounding but unverified outputs.

3. Post-Processing Checks

Run AI outputs through a secondary filter or human reviewer—especially in legal, medical, or high-risk scenarios.

4. Training Data Hygiene

Use vetted, reputable data during training. Remove sources known for fake news or low-quality content.

5. Avoid Auto-Deployment

Don’t allow unchecked LLM content to go live (e.g., auto-publishing blogs or emails) without oversight.

Related OWASP Risks

Misinformation often overlaps with:

• LLM01: Prompt Injection – malicious users can cause an AI to mislead
• LLM04: Model Poisoning – bad training data creates falsehoods
• LLM08: Vector Weaknesses – attackers embed biased interpretations

Conclusion

As AI becomes a trusted source of knowledge, the risks of misinformation grow exponentially. OWASP LLM09 highlights a chilling truth: people may believe AI over humans, even when it’s wrong.

To build trustworthy AI, we must combine fluent generation with fact-checking, transparency, and accountability. Otherwise, the age of intelligent machines may also be the age of intelligent lies.