AI jailbreaking via poetry: bypassing chatbot defenses with rhyme | Kaspersky official blog
Source Kaspersky Security Blog
Description
A new study shows that verse-based prompts can slash the effectiveness of AI safety constraints. We’re breaking down an experiment involving 25 language models and its key takeaways.
Impact
For European organizations, the impact of this threat is multifaceted. AI chatbots are increasingly integrated into customer service, healthcare, finance, and public administration, where they handle sensitive data and provide critical information. Successful jailbreaking via poetic prompts could lead to the generation of misleading or harmful content, damaging organizational reputation and eroding user trust. In regulated industries, such as finance and healthcare, this could also result in compliance violations under GDPR and other data protection laws if inappropriate or confidential information is inadvertently disclosed. Additionally, attackers might exploit this vulnerability to bypass content moderation, facilitating the spread of disinformation or malicious instructions. The broad applicability across multiple AI platforms means that organizations relying on third-party AI services are also at risk, complicating mitigation efforts. The medium severity reflects the balance between the ease of exploitation and the current lack of known active attacks, but the potential for significant operational and reputational harm remains high.
Defensive guidance
To mitigate this threat, European organizations should implement several specific measures beyond generic AI security advice. First, AI safety frameworks must be enhanced to detect and analyze stylistic and structural features of input, including rhyme and meter, to identify potential evasion attempts. This may involve integrating advanced natural language understanding modules capable of semantic and poetic analysis. Second, organizations should employ layered defense strategies combining AI safety filters with human-in-the-loop review for sensitive or high-risk interactions. Third, continuous monitoring and logging of AI interactions should be established to detect anomalous patterns indicative of jailbreaking attempts. Fourth, collaboration with AI vendors is critical to ensure timely updates and patches that address linguistic evasion techniques. Fifth, organizations should conduct regular red-teaming exercises using creative prompt engineering to test and improve AI safety robustness. Finally, raising awareness among AI developers and users about the risks of poetic jailbreaking can help in early detection and response.
Summary
The threat involves a novel method of bypassing AI chatbot safety constraints by using poetic or verse-based prompts, a technique termed 'AI jailbreaking via poetry.' Traditional AI safety filters and content moderation systems rely heavily on pattern recognition and keyword detection to prevent the generation of harmful or inappropriate content. However, the study conducted by Kaspersky tested 25 different language models and found that when prompts are crafted in poetic form—leveraging rhyme, meter, and stylistic nuances—these safety mechanisms become significantly less effective. This is because the linguistic creativity inherent in poetry can obscure the intent or content of the prompt, making it difficult for AI safety layers to detect and block malicious or policy-violating requests. The vulnerability does not depend on specific software versions or require authentication, indicating a broad attack surface across many AI-powered chatbot deployments. While no active exploitation has been reported, the research highlights a fundamental challenge in AI safety: the need to handle complex, creative language inputs that can subvert straightforward filtering techniques. This jailbreaking method could be used to coerce AI systems into generating disallowed content, leaking sensitive information, or performing unauthorized actions, thereby undermining trust and compliance. The lack of patches or direct fixes necessitates a focus on adaptive AI safety designs and continuous threat intelligence to anticipate and mitigate such linguistic evasion tactics.