Threats Tagged 'indirect prompt injection'

Threat actors are actively exploiting AI distribution platforms like Hugging Face and ClawHub to deliver malware by embedding malicious code within models, datasets, and agent extensions. Over 575 malicious skills across 13 developer accounts were identified in the OpenClaw ecosystem, targeting Windows and macOS with trojans, cryptominers, and AMOS stealer. Attackers abuse trust relationships between users and AI platforms through indirect prompt injection, where hidden instructions cause AI agents to execute malicious actions on behalf of users. Trojanized skills masquerade as legitimate tools while instructing users to execute encoded commands or install hidden malicious dependencies. On Hugging Face, repositories host payloads within multistep infection chains disguised as legitimate applications. These campaigns employ social engineering, obfuscation, encryption, in-memory execution, process injection, and persistence techniques to evade detection while establishing covert command-and-control communica...

X-Labs researchers discovered 10 verified Indirect Prompt Injection (IPI) payloads deployed across live web infrastructure. Unlike direct prompt injection where users send malicious input to AI models, IPI hides adversarial instructions inside ordinary web content. When AI agents crawl or summarize poisoned pages, they ingest and execute these instructions as legitimate commands. The discovered payloads span financial fraud, data destruction, API key exfiltration, and denial-of-service attacks. Attackers employ techniques including CSS invisibility, HTML comments, accessibility attribute abuse, meta namespace spoofing, and system prompt tag impersonation. The shared injection templates across multiple domains suggest organized tooling rather than isolated experimentation. Observed attack intents include unauthorized financial transactions, terminal command execution, content suppression, traffic hijacking, and sensitive information leakage, targeting AI systems that browse web pages, index content for RAG ...

This article analyzes real-world instances of indirect prompt injection (IDPI) attacks targeting AI agents and large language models integrated into web systems. The researchers identify 22 distinct techniques used by attackers to embed malicious prompts in webpages, including visual concealment, obfuscation, and dynamic execution methods. They categorize attacker intents ranging from low-severity disruptions to critical data destruction attempts. Notable findings include the first observed case of AI-based ad review evasion and attempts at search engine optimization manipulation. The article presents a taxonomy of web-based IDPI attacks and provides insights into attack trends based on telemetry data. The researchers emphasize the need for proactive, web-scale defenses to detect IDPI and distinguish between benign and malicious prompts.

MediumCampaignUnited StatesUnited KingdomGermany+8#prompt engineering#ai security#ai agents