CVE-2023-52587: Vulnerability in Linux Linux

Severity: highType: vulnerabilityCVE-2023-52587

In the Linux kernel, the following vulnerability has been resolved: IB/ipoib: Fix mcast list locking Releasing the `priv->lock` while iterating the `priv->multicast_list` in `ipoib_mcast_join_task()` opens a window for `ipoib_mcast_dev_flush()` to remove the items while in the middle of iteration. If the mcast is removed while the lock was dropped, the for loop spins forever resulting in a hard lockup (as was reported on RHEL 4.18.0-372.75.1.el8_6 kernel): Task A (kworker/u72:2 below) | Task B (kworker/u72:0 below) -----------------------------------+----------------------------------- ipoib_mcast_join_task(work) | ipoib_ib_dev_flush_light(work) spin_lock_irq(&priv->lock) | __ipoib_ib_dev_flush(priv, ...) list_for_each_entry(mcast, | ipoib_mcast_dev_flush(dev = priv->dev) &priv->multicast_list, list) | ipoib_mcast_join(dev, mcast) | spin_unlock_irq(&priv->lock) | | spin_lock_irqsave(&priv->lock, flags) | list_for_each_entry_safe(mcast, tmcast, | &priv->multicast_list, list) | list_del(&mcast->list); | list_add_tail(&mcast->list, &remove_list) | spin_unlock_irqrestore(&priv->lock, flags) spin_lock_irq(&priv->lock) | | ipoib_mcast_remove_list(&remove_list) (Here, `mcast` is no longer on the | list_for_each_entry_safe(mcast, tmcast, `priv->multicast_list` and we keep | remove_list, list) spinning on the `remove_list` of | >>> wait_for_completion(&mcast->done) the other thread which is blocked | and the list is still valid on | it's stack.) Fix this by keeping the lock held and changing to GFP_ATOMIC to prevent eventual sleeps. Unfortunately we could not reproduce the lockup and confirm this fix but based on the code review I think this fix should address such lockups. crash> bc 31 PID: 747 TASK: ff1c6a1a007e8000 CPU: 31 COMMAND: "kworker/u72:2" -- [exception RIP: ipoib_mcast_join_task+0x1b1] RIP: ffffffffc0944ac1 RSP: ff646f199a8c7e00 RFLAGS: 00000002 RAX: 0000000000000000 RBX: ff1c6a1a04dc82f8 RCX: 0000000000000000 work (&priv->mcast_task{,.work}) RDX: ff1c6a192d60ac68 RSI: 0000000000000286 RDI: ff1c6a1a04dc8000 &mcast->list RBP: ff646f199a8c7e90 R8: ff1c699980019420 R9: ff1c6a1920c9a000 R10: ff646f199a8c7e00 R11: ff1c6a191a7d9800 R12: ff1c6a192d60ac00 mcast R13: ff1c6a1d82200000 R14: ff1c6a1a04dc8000 R15: ff1c6a1a04dc82d8 dev priv (&priv->lock) &priv->multicast_list (aka head) ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 --- <NMI exception stack> --- #5 [ff646f199a8c7e00] ipoib_mcast_join_task+0x1b1 at ffffffffc0944ac1 [ib_ipoib] #6 [ff646f199a8c7e98] process_one_work+0x1a7 at ffffffff9bf10967 crash> rx ff646f199a8c7e68 ff646f199a8c7e68: ff1c6a1a04dc82f8 <<< work = &priv->mcast_task.work crash> list -hO ipoib_dev_priv.multicast_list ff1c6a1a04dc8000 (empty) crash> ipoib_dev_priv.mcast_task.work.func,mcast_mutex.owner.counter ff1c6a1a04dc8000 mcast_task.work.func = 0xffffffffc0944910 <ipoib_mcast_join_task>, mcast_mutex.owner.counter = 0xff1c69998efec000 crash> b 8 PID: 8 TASK: ff1c69998efec000 CPU: 33 COMMAND: "kworker/u72:0" -- #3 [ff646f1980153d50] wait_for_completion+0x96 at ffffffff9c7d7646 #4 [ff646f1980153d90] ipoib_mcast_remove_list+0x56 at ffffffffc0944dc6 [ib_ipoib] #5 [ff646f1980153de8] ipoib_mcast_dev_flush+0x1a7 at ffffffffc09455a7 [ib_ipoib] #6 [ff646f1980153e58] __ipoib_ib_dev_flush+0x1a4 at ffffffffc09431a4 [ib_ipoib] #7 [ff ---truncated---

AI Analysis

Technical Summary

CVE-2023-52587 is a vulnerability in the Linux kernel's InfiniBand (IB) IP over IB (ipoib) driver, specifically related to multicast list locking in the ipoib_mcast_join_task() function. The flaw arises from improper lock management when iterating over the multicast list. The ipoib_mcast_join_task() function releases the priv->lock spinlock while iterating through priv->multicast_list, which creates a race condition. Concurrently, ipoib_mcast_dev_flush() can remove multicast entries from the list while the lock is dropped, causing the iteration in ipoib_mcast_join_task() to spin indefinitely on a list entry that has been removed. This results in a hard lockup of the kernel thread, effectively causing a denial of service (DoS) condition. The root cause is the unsafe unlocking during list iteration combined with concurrent list modifications by another kernel worker thread. The fix involves holding the lock throughout the iteration and switching to GFP_ATOMIC memory allocation to avoid sleeping while holding the lock. Although the lockup could not be reproduced in testing, code review strongly indicates this fix addresses the issue. The vulnerability affects Linux kernels that include the ipoib driver, commonly used in high-performance computing and data center environments that utilize InfiniBand networking for low-latency, high-throughput communication. The issue is a kernel-level concurrency bug that can cause system instability and denial of service due to kernel worker thread lockup.

Potential Impact

For European organizations, especially those operating data centers, HPC clusters, or enterprise environments using InfiniBand networking on Linux systems, this vulnerability poses a risk of kernel-level denial of service. A hard lockup in kernel worker threads can cause system hangs or crashes, potentially disrupting critical services and workloads. This is particularly impactful for industries relying on high-performance computing, scientific research, financial services, and telecommunications where InfiniBand is prevalent. The inability to process multicast group joins correctly can degrade network communication reliability and availability. Although no known exploits are reported in the wild, the vulnerability could be triggered by local or privileged users or processes that interact with the ipoib multicast functionality, potentially leading to service outages. The impact on confidentiality and integrity is minimal since the flaw causes availability issues rather than data leakage or corruption. However, availability degradation in critical infrastructure can have significant operational and financial consequences.

Mitigation Recommendations

European organizations should apply the Linux kernel patches that address CVE-2023-52587 as soon as they become available from their Linux distribution vendors. Specifically, ensure that the kernel version includes the fix that holds the priv->lock throughout the multicast list iteration and uses GFP_ATOMIC allocation to prevent sleeping while locked. For environments where immediate patching is not feasible, consider temporarily disabling the ipoib driver if InfiniBand multicast is not required, to mitigate the risk of triggering the lockup. Additionally, monitor kernel logs and system behavior for signs of worker thread lockups or hangs related to ipoib multicast operations. Implement robust kernel crash and hang detection mechanisms to enable rapid recovery. For critical systems, test patches in staging environments to validate stability before production deployment. Finally, maintain strict access controls to limit which users or processes can interact with InfiniBand multicast configurations, reducing the attack surface.