CVE-2024-47744: Vulnerability in Linux Linux

Severity: mediumType: vulnerabilityCVE-2024-47744

In the Linux kernel, the following vulnerability has been resolved: KVM: Use dedicated mutex to protect kvm_usage_count to avoid deadlock Use a dedicated mutex to guard kvm_usage_count to fix a potential deadlock on x86 due to a chain of locks and SRCU synchronizations. Translating the below lockdep splat, CPU1 #6 will wait on CPU0 #1, CPU0 #8 will wait on CPU2 #3, and CPU2 #7 will wait on CPU1 #4 (if there's a writer, due to the fairness of r/w semaphores). CPU0 CPU1 CPU2 1 lock(&kvm->slots_lock); 2 lock(&vcpu->mutex); 3 lock(&kvm->srcu); 4 lock(cpu_hotplug_lock); 5 lock(kvm_lock); 6 lock(&kvm->slots_lock); 7 lock(cpu_hotplug_lock); 8 sync(&kvm->srcu); Note, there are likely more potential deadlocks in KVM x86, e.g. the same pattern of taking cpu_hotplug_lock outside of kvm_lock likely exists with __kvmclock_cpufreq_notifier(): cpuhp_cpufreq_online() | -> cpufreq_online() | -> cpufreq_gov_performance_limits() | -> __cpufreq_driver_target() | -> __target_index() | -> cpufreq_freq_transition_begin() | -> cpufreq_notify_transition() | -> ... __kvmclock_cpufreq_notifier() But, actually triggering such deadlocks is beyond rare due to the combination of dependencies and timings involved. E.g. the cpufreq notifier is only used on older CPUs without a constant TSC, mucking with the NX hugepage mitigation while VMs are running is very uncommon, and doing so while also onlining/offlining a CPU (necessary to generate contention on cpu_hotplug_lock) would be even more unusual. The most robust solution to the general cpu_hotplug_lock issue is likely to switch vm_list to be an RCU-protected list, e.g. so that x86's cpufreq notifier doesn't to take kvm_lock. For now, settle for fixing the most blatant deadlock, as switching to an RCU-protected list is a much more involved change, but add a comment in locking.rst to call out that care needs to be taken when walking holding kvm_lock and walking vm_list. ====================================================== WARNING: possible circular locking dependency detected 6.10.0-smp--c257535a0c9d-pip #330 Tainted: G S O ------------------------------------------------------ tee/35048 is trying to acquire lock: ff6a80eced71e0a8 (&kvm->slots_lock){+.+.}-{3:3}, at: set_nx_huge_pages+0x179/0x1e0 [kvm] but task is already holding lock: ffffffffc07abb08 (kvm_lock){+.+.}-{3:3}, at: set_nx_huge_pages+0x14a/0x1e0 [kvm] which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #3 (kvm_lock){+.+.}-{3:3}: __mutex_lock+0x6a/0xb40 mutex_lock_nested+0x1f/0x30 kvm_dev_ioctl+0x4fb/0xe50 [kvm] __se_sys_ioctl+0x7b/0xd0 __x64_sys_ioctl+0x21/0x30 x64_sys_call+0x15d0/0x2e60 do_syscall_64+0x83/0x160 entry_SYSCALL_64_after_hwframe+0x76/0x7e -> #2 (cpu_hotplug_lock){++++}-{0:0}: cpus_read_lock+0x2e/0xb0 static_key_slow_inc+0x16/0x30 kvm_lapic_set_base+0x6a/0x1c0 [kvm] kvm_set_apic_base+0x8f/0xe0 [kvm] kvm_set_msr_common+0x9ae/0xf80 [kvm] vmx_set_msr+0xa54/0xbe0 [kvm_intel] __kvm_set_msr+0xb6/0x1a0 [kvm] kvm_arch_vcpu_ioctl+0xeca/0x10c0 [kvm] kvm_vcpu_ioctl+0x485/0x5b0 [kvm] __se_sys_ioctl+0x7b/0xd0 __x64_sys_ioctl+0x21/0x30 x64_sys_call+0x15d0/0x2e60 do_syscall_64+0x83/0x160 entry_SYSCALL_64_after_hwframe+0x76/0x7e -> #1 (&kvm->srcu){.+.+}-{0:0}: __synchronize_srcu+0x44/0x1a0 ---truncated---

AI Analysis

Technical Summary

CVE-2024-47744 addresses a concurrency vulnerability in the Linux kernel's KVM (Kernel-based Virtual Machine) subsystem on x86 architectures. The issue arises from a circular locking dependency involving multiple mutexes and synchronization primitives, specifically between kvm_usage_count, kvm->slots_lock, kvm_lock, cpu_hotplug_lock, and kvm->srcu. This circular dependency can lead to a deadlock scenario where multiple CPUs wait indefinitely for locks held by each other, effectively halting progress in the affected kernel threads. The vulnerability is rooted in the way KVM manages virtual CPU (vCPU) state and CPU hotplug events, particularly when operations such as setting NX huge pages or CPU frequency transitions occur concurrently with CPU online/offline events. Although the triggering conditions are rare due to the specific timing and system state dependencies (e.g., older CPUs without constant TSC, simultaneous CPU hotplug and VM memory operations), the potential for deadlock exists and can cause system hangs or degraded performance in virtualized environments. The fix implemented involves introducing a dedicated mutex to protect kvm_usage_count, breaking the circular lock dependency and preventing deadlocks. The report also notes that other similar deadlock patterns may exist in KVM's x86 code, suggesting that further architectural changes (like converting vm_list to an RCU-protected list) might be necessary for a comprehensive resolution. However, the immediate patch focuses on the most critical deadlock scenario to improve stability.

Potential Impact

For European organizations relying on Linux-based virtualization infrastructure, especially those using KVM on x86 servers, this vulnerability poses a risk of system deadlocks that can disrupt critical services. Deadlocks in the kernel can cause virtual machines to become unresponsive, leading to downtime for hosted applications, including cloud services, enterprise workloads, and containerized environments. This can impact data center operations, cloud providers, and enterprises with private clouds or virtualized infrastructure. The rarity of triggering conditions somewhat limits the likelihood of widespread exploitation, but environments with frequent CPU hotplug events, dynamic CPU frequency scaling on older hardware, or heavy VM memory management operations could experience stability issues. Such disruptions could affect service availability, operational continuity, and potentially lead to cascading failures in dependent systems. Additionally, troubleshooting deadlocks at the kernel level can be complex and time-consuming, increasing operational costs and incident response times. Given the widespread use of Linux and KVM in European data centers, cloud providers, and enterprises, the impact could be significant in sectors like finance, telecommunications, government, and critical infrastructure where high availability is essential.

Mitigation Recommendations

1. Apply the official Linux kernel patches that introduce the dedicated mutex protecting kvm_usage_count as soon as they become available and are tested for compatibility with existing systems. 2. Monitor kernel updates from trusted Linux distributions and prioritize updates for systems running KVM on x86 architectures. 3. Minimize CPU hotplug operations and dynamic CPU frequency scaling on older hardware where possible, especially during critical VM operations, to reduce the likelihood of triggering the deadlock. 4. Implement robust monitoring and alerting for kernel deadlocks and VM responsiveness issues, enabling rapid detection and remediation. 5. Consider architectural changes such as migrating to newer hardware with constant TSC support and updated CPU frequency management to avoid legacy code paths involved in the deadlock. 6. For environments with high availability requirements, deploy redundancy and failover mechanisms to mitigate the impact of potential deadlocks. 7. Engage with Linux vendor support channels for backported patches and guidance tailored to specific enterprise distributions. 8. Review and audit custom kernel modules or third-party virtualization extensions that might interact with KVM locking mechanisms to ensure compatibility with the fix.