Trained to betray: How post-training supply chain attacks hijack coding agents
Source Microsoft Security Blog
Description
Two studies by Microsoft reveal post-training supply chain attacks on AI coding agents through dataset poisoning and model poisoning. Dataset poisoning involves inserting crafted examples into fine-tuning data that teach the model to misuse available command-execution tools when triggered by benign-looking user requests. Model poisoning assumes attacker control over training and distribution of model checkpoints, embedding reliable backdoors that activate unsafe behaviors while maintaining general performance. These backdoors can cause agents to execute commands, modify files, or access data without obvious malicious triggers, posing risks to sensitive development environments. The attacks exploit trust in training data and model artifacts, emphasizing the need for controls over training pipelines, model provenance, and tool permissions.