SentinelOne created the first long-horizon reverse-engineering benchmark for frontier AI models based on the Fast16 malware, a 2005 Windows malware associated with nuclear sabotage activities. The benchmark assesses whether AI models can sustain a trustworthy investigation through eight stages where new evidence may contradict earlier conclusions, requiring the model to retract and correct prior errors comprehensively. Tested models included OpenAI's GPT-5.5 and GPT-5.6 Sol, Z.ai's GLM-5.2, and Anthropic's Opus 4.x. Only GPT-5.6 Sol successfully completed all stages, while others failed to progress beyond initial or intermediate stages. The key differentiator was 'project-scale recovery'—the ability to withdraw disproven conclusions and propagate corrections throughout the investigation. SentinelLabs concluded that human analysts remain essential due to persistent semantic errors and premature readiness claims by AI models.

The benchmark reveals that most frontier AI models currently lack the capability to independently conduct thorough and reliable malware investigations, especially when faced with evolving contradictory evidence. This limitation could affect the effectiveness of AI-assisted malware analysis and reverse engineering in cybersecurity operations. However, no direct exploitation or malware campaign is associated with this benchmark itself. The findings emphasize the continued need for human expertise to oversee AI-driven investigations to avoid technical mistakes and incomplete analyses.

This is not a vulnerability or exploit but an evaluation of AI models' investigative capabilities. No patch or fix is applicable. Organizations should continue to rely on human experts to supervise AI-assisted malware investigations, using AI as a tool rather than a replacement for skilled analysts. Monitoring advancements in AI model capabilities and integrating them cautiously with human oversight is recommended.