An AI code scanner matched humans on every critical/high bug. Would you trust it to replace a human code review?
Source Reddit Cybersecurity
Discussion
A recent benchmark compared an AI code scanner against human code reviewers across over 1,000 codebases. The AI matched humans on every critical and high severity bug found and even identified some issues missed by humans. The AI also optimizes resource usage by stopping scans when no new bugs are detected and shows improved findings with increased compute resources. This raises questions about the potential for AI to supplement or replace human code reviews in security audits.
r/cybersecurity (opens in a new tab)u/Personal_Limit_56400 points0 comments
Just found an interesting benchmark that’s making me rethink how i approach security research. They ran a bug hunting agent across 1k+ codebase and compared its results to 8 human code reviews. What’s interesting is the AI matched what humans discovered, on all critical and high findings, and even found some they missed.
Something that I found wild. it auto-detects when a scan stops finding new bugs and stops instead of burning budget. and they identify a clean scaling relationship: more compute = more findings (about 40% more findings w a doibled budget).
Question for people who do security research: does matching crit+high on different codebases mean anything to you? Where would you actually use something like this?
https://www.cantina.security/blog/ai-vs-human-security-audit-benchmark