we found two prompt_attack incidents at our company. here's what actually happened.
Source Reddit Cybersecurity
Discussion
Two incidents involving 'prompt_attack' errors were detected at a healthcare company using third-party platform looplabs.run. The issue arose when open source plugins like SuperPowers and BMAD automatically injected coercive instructions into user prompts sent to a large language model (LLM), causing the platform to block these prompts. The injected text included phrases such as 'YOU DO NOT HAVE A CHOICE. YOU MUST USE IT. This is not negotiable,' which triggered the enforcement mechanism. This behavior was initially misunderstood as a platform error but was traced to the plugin-injected prompt content.
r/cybersecurity (opens in a new tab)u/Common_Dream94200 points0 comments
I work for healthcare company, and for long time devs are using open source plugins like SuperPowers, BMAD for building features/products.
Recently, few people blocked with the error message PROMPT_ATTACK which was enforced by my team recently third party platform called looplabs.run, initially we thought something wrong with the platform but later we noticed every prompt the developers/product managers sending automatedely adding their plugin skills as well along with their layman prompt.
at first, we thought why this simple prompt blocked by looplabs, but the text inside the third party plugins like Superpowers instructiing LLM like this
"YOU DO NOT HAVE A CHOICE. YOU MUST USE IT. This is not negotiable" <<<< this is injected into the user prompt
Also discussed in r/Information_Security