Using AI to help with GRC tasks · do prompt structures matter?
Source Reddit Cybersecurity
Discussion
Using AI to help with GRC tasks - do prompt structures matter? Source: https://www.riskquilt.com/blog/grc-prompting
r/cybersecurity (opens in a new tab)u/FreeRadical19980 points0 comments
I'm both a CISO and a SaaS GRC tool founder, and from both angles I'm interested in how AI can help with some of the repetitive work. Like it or not, there's a strong push to find ways to use it, so I'd like to know how to do that effectively.
NIST's draft SP 1353 (using AI for CSF 2.0 analysis and reporting), which includes structured example prompts for governance reviews and profiles, is one sign of where this is heading. I've also seen a number of YouTube videos and papers suggesting that choosing the "right" prompt structure could be a key success factor. I couldn't find any testing on GRC work specifically, so I pulled together the general research:
- Changing only the format (plain text, Markdown, JSON) can swing results a long way, but the best format doesn't carry over between models.
- Named frameworks like CO-STAR and RISEN barely differ. The one direct comparison found them within 0.05 points.
- Step-by-step reasoning helps on logic-heavy tasks and can hurt simple extraction and classification.
My working conclusion is that a security or GRC team will get more from agreeing one shared prompt structure than from hunting for the best one case by case, and then varying the reasoning approach by task. Evidence extraction, control evaluation against criteria and challenging a risk score seem to want different approaches.
Write-up with a hypothesis map and task table:
Disclosure: it's on my company's blog, but the article has no product pitch and should be relevant whether or not you use a GRC tool.
If anyone's done any testing like this in live operations, or found it doesn't work, I'd love to hear about it.