Skip to main content
Press slash or control plus K to focus the search. Use the arrow keys to navigate results and press enter to open a threat.
Reconnecting to live updates…

Researchers Find ChatGPT Vulnerabilities That Let Attackers Trick AI Into Leaking Data

0
Medium
Exploit
Published: Wed Nov 05 2025 (11/05/2025, 14:04:00 UTC)
Source: The Hacker News

Description

Researchers disclosed seven vulnerabilities in OpenAI's GPT-4o and GPT-5 models that enable attackers to perform indirect prompt injection attacks, tricking ChatGPT into leaking personal data from user memories and chat histories. These vulnerabilities exploit mechanisms such as malicious instructions embedded in web page comments, crafted URLs, and trusted domains to bypass safety filters and manipulate AI responses without user awareness. The attacks include zero-click and one-click prompt injections, conversation and memory poisoning, and hiding malicious prompts via markdown rendering bugs. While OpenAI has addressed some issues, prompt injection remains a systemic challenge in large language models (LLMs). The threat extends to AI tools integrated with external systems, increasing the attack surface and risk of data exfiltration. European organizations using ChatGPT or similar AI services face risks of confidential data leakage and manipulation of AI outputs. Mitigations require strict validation of external content, enhanced safety mechanisms, and user awareness. Countries with high AI adoption and digital transformation initiatives are most likely affected. Given the medium severity and complexity of exploitation, organizations must proactively manage AI-related risks.

AI-Powered Analysis

AILast updated: 11/06/2025, 02:28:54 UTC

Technical Analysis

The disclosed security threat involves seven distinct vulnerabilities in OpenAI's GPT-4o and GPT-5 large language models that enable attackers to conduct indirect prompt injection attacks. These attacks manipulate the AI's expected behavior by embedding malicious instructions in external content that the AI processes, such as web pages, search engine indexed sites, or specially crafted URLs. Key vulnerabilities include: (1) Indirect prompt injection via trusted browsing contexts where malicious comments in web pages cause the AI to execute unintended commands; (2) Zero-click injection in search contexts by querying indexed malicious sites; (3) One-click injection through crafted URLs that auto-execute prompts; (4) Safety mechanism bypass using allow-listed domains like bing.com to mask malicious URLs; (5) Conversation injection that poisons the AI's conversational context leading to unintended replies; (6) Malicious content hiding exploiting markdown rendering bugs to conceal harmful prompts; and (7) Memory injection that poisons user-specific AI memory by embedding hidden instructions in summarized content. These vulnerabilities allow attackers to exfiltrate personal information from users’ chat histories and AI memory without their knowledge. The threat is exacerbated by the integration of AI chatbots with external tools and systems, expanding the attack surface and enabling more sophisticated prompt injections. The research highlights the difficulty in systematically fixing prompt injection due to the fundamental architecture of LLMs. Additionally, related research shows that poisoning training data and backdooring AI models are feasible with relatively small amounts of malicious input, increasing the risk of long-term AI manipulation. The vulnerabilities do not require user interaction beyond querying or summarizing malicious content, and some attacks are zero-click, increasing ease of exploitation. OpenAI has mitigated some issues, but the systemic nature of prompt injection remains a concern. The threat landscape also includes similar vulnerabilities in other AI tools like Anthropic Claude and Microsoft 365 Copilot, indicating a broader industry challenge.

Potential Impact

For European organizations, these vulnerabilities pose significant risks to confidentiality and integrity of sensitive data processed by AI chatbots like ChatGPT. Attackers can stealthily extract personal information, proprietary data, or confidential chat histories, potentially leading to data breaches and compliance violations under GDPR. Manipulation of AI responses can result in misinformation, erroneous decision-making, or reputational damage. Organizations relying on AI for customer service, knowledge management, or automation may face operational disruptions or loss of trust. The indirect and zero-click nature of some attacks increases the likelihood of unnoticed exploitation. Furthermore, the integration of AI with enterprise systems expands the attack surface, potentially allowing attackers to pivot to other internal resources. The medium severity reflects the balance between the complexity of crafting attacks and the potential for impactful data leakage and AI misuse. European entities involved in AI development, deployment, or regulation must consider these risks in their security posture and governance frameworks.

Mitigation Recommendations

To mitigate these vulnerabilities, European organizations should implement multi-layered defenses beyond generic AI security advice: 1) Enforce strict validation and sanitization of all external content ingested by AI systems, including web pages, URLs, and search engine results, to detect and block malicious prompt injections. 2) Limit or disable AI browsing and summarization features that pull content from untrusted or user-generated sources unless content is pre-verified. 3) Enhance AI safety mechanisms by regularly updating allow-lists and URL filtering to prevent bypass via trusted domains used maliciously. 4) Monitor AI outputs for anomalous or unexpected responses that may indicate prompt injection or memory poisoning. 5) Implement user awareness training to recognize potential AI manipulation and encourage cautious use of AI chatbots for sensitive information. 6) Collaborate with AI vendors to ensure timely patching and deployment of security updates addressing prompt injection vulnerabilities. 7) Employ network-level controls to restrict AI system access to only necessary external resources and monitor for suspicious queries or traffic patterns. 8) For organizations developing or fine-tuning AI models, apply robust data curation and poisoning detection techniques to prevent training on malicious content. 9) Integrate AI security into broader cybersecurity governance and incident response plans, including AI-specific threat hunting and forensic capabilities. 10) Advocate for and participate in industry-wide efforts to develop standardized AI security best practices and prompt injection mitigation frameworks.

Need more detailed analysis?Get Pro

Technical Details

Article Source
{"url":"https://thehackernews.com/2025/11/researchers-find-chatgpt.html","fetched":true,"fetchedAt":"2025-11-06T02:28:27.429Z","wordCount":1701}

Threat ID: 690c07d3fd0d6d226481f700

Added to database: 11/6/2025, 2:28:35 AM

Last enriched: 11/6/2025, 2:28:54 AM

Last updated: 11/6/2025, 6:56:07 AM

Views: 3

Community Reviews

0 reviews

Crowdsource mitigation strategies, share intel context, and vote on the most helpful responses. Sign in to add your voice and help keep defenders ahead.

Sort by
Loading community insights…

Want to contribute mitigation steps or threat intel context? Sign in or create an account to join the community discussion.

Actions

PRO

Updates to AI analysis require Pro Console access. Upgrade inside Console → Billing.

Please log in to the Console to use AI analysis features.

Need enhanced features?

Contact root@offseq.com for Pro access with improved analysis and higher rate limits.

Latest Threats