CVE-2024-27014: Vulnerability in Linux Linux

Severity: highType: vulnerabilityCVE-2024-27014

In the Linux kernel, the following vulnerability has been resolved: net/mlx5e: Prevent deadlock while disabling aRFS When disabling aRFS under the `priv->state_lock`, any scheduled aRFS works are canceled using the `cancel_work_sync` function, which waits for the work to end if it has already started. However, while waiting for the work handler, the handler will try to acquire the `state_lock` which is already acquired. The worker acquires the lock to delete the rules if the state is down, which is not the worker's responsibility since disabling aRFS deletes the rules. Add an aRFS state variable, which indicates whether the aRFS is enabled and prevent adding rules when the aRFS is disabled. Kernel log: ====================================================== WARNING: possible circular locking dependency detected 6.7.0-rc4_net_next_mlx5_5483eb2 #1 Tainted: G I ------------------------------------------------------ ethtool/386089 is trying to acquire lock: ffff88810f21ce68 ((work_completion)(&rule->arfs_work)){+.+.}-{0:0}, at: __flush_work+0x74/0x4e0 but task is already holding lock: ffff8884a1808cc0 (&priv->state_lock){+.+.}-{3:3}, at: mlx5e_ethtool_set_channels+0x53/0x200 [mlx5_core] which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #1 (&priv->state_lock){+.+.}-{3:3}: __mutex_lock+0x80/0xc90 arfs_handle_work+0x4b/0x3b0 [mlx5_core] process_one_work+0x1dc/0x4a0 worker_thread+0x1bf/0x3c0 kthread+0xd7/0x100 ret_from_fork+0x2d/0x50 ret_from_fork_asm+0x11/0x20 -> #0 ((work_completion)(&rule->arfs_work)){+.+.}-{0:0}: __lock_acquire+0x17b4/0x2c80 lock_acquire+0xd0/0x2b0 __flush_work+0x7a/0x4e0 __cancel_work_timer+0x131/0x1c0 arfs_del_rules+0x143/0x1e0 [mlx5_core] mlx5e_arfs_disable+0x1b/0x30 [mlx5_core] mlx5e_ethtool_set_channels+0xcb/0x200 [mlx5_core] ethnl_set_channels+0x28f/0x3b0 ethnl_default_set_doit+0xec/0x240 genl_family_rcv_msg_doit+0xd0/0x120 genl_rcv_msg+0x188/0x2c0 netlink_rcv_skb+0x54/0x100 genl_rcv+0x24/0x40 netlink_unicast+0x1a1/0x270 netlink_sendmsg+0x214/0x460 __sock_sendmsg+0x38/0x60 __sys_sendto+0x113/0x170 __x64_sys_sendto+0x20/0x30 do_syscall_64+0x40/0xe0 entry_SYSCALL_64_after_hwframe+0x46/0x4e other info that might help us debug this: Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(&priv->state_lock); lock((work_completion)(&rule->arfs_work)); lock(&priv->state_lock); lock((work_completion)(&rule->arfs_work)); *** DEADLOCK *** 3 locks held by ethtool/386089: #0: ffffffff82ea7210 (cb_lock){++++}-{3:3}, at: genl_rcv+0x15/0x40 #1: ffffffff82e94c88 (rtnl_mutex){+.+.}-{3:3}, at: ethnl_default_set_doit+0xd3/0x240 #2: ffff8884a1808cc0 (&priv->state_lock){+.+.}-{3:3}, at: mlx5e_ethtool_set_channels+0x53/0x200 [mlx5_core] stack backtrace: CPU: 15 PID: 386089 Comm: ethtool Tainted: G I 6.7.0-rc4_net_next_mlx5_5483eb2 #1 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x60/0xa0 check_noncircular+0x144/0x160 __lock_acquire+0x17b4/0x2c80 lock_acquire+0xd0/0x2b0 ? __flush_work+0x74/0x4e0 ? save_trace+0x3e/0x360 ? __flush_work+0x74/0x4e0 __flush_work+0x7a/0x4e0 ? __flush_work+0x74/0x4e0 ? __lock_acquire+0xa78/0x2c80 ? lock_acquire+0xd0/0x2b0 ? mark_held_locks+0x49/0x70 __cancel_work_timer+0x131/0x1c0 ? mark_held_locks+0x49/0x70 arfs_del_rules+0x143/0x1e0 [mlx5_core] mlx5e_arfs_disable+0x1b/0x30 [mlx5_core] mlx5e_ethtool_set_channels+0xcb/0x200 [mlx5_core] ethnl_set_channels+0x28f/0x3b0 ethnl_default_set_doit+0xec/0x240 genl_family_rcv_msg_doit+0xd0/0x120 genl_rcv_msg+0x188/0x2c0 ? ethn ---truncated---

AI Analysis

Technical Summary

CVE-2024-27014 is a concurrency vulnerability in the Linux kernel's Mellanox mlx5 Ethernet driver, specifically within the mlx5e module that handles adaptive Receive Flow Steering (aRFS). The vulnerability arises from a deadlock condition when disabling aRFS under the priv->state_lock mutex. The disabling process involves canceling scheduled aRFS work using cancel_work_sync, which waits for any ongoing work to complete. However, the work handler itself attempts to acquire the same priv->state_lock that is already held by the disabling thread, creating a circular locking dependency and resulting in a deadlock. This deadlock occurs because the worker thread tries to delete rules if the network state is down, which conflicts with the disabling logic that also deletes these rules. The fix involves introducing an aRFS state variable to track whether aRFS is enabled, preventing rule additions when aRFS is disabled and thus breaking the circular lock dependency. The vulnerability manifests as kernel warnings about circular locking dependencies and can cause the affected system to hang or become unresponsive due to the deadlock. The issue affects Linux kernel versions containing the mlx5e driver code prior to the fix and is relevant for systems using Mellanox network adapters with aRFS enabled. There are no known exploits in the wild at this time, and no CVSS score has been assigned yet. The vulnerability is primarily a denial-of-service risk caused by kernel deadlock rather than direct code execution or privilege escalation.

Potential Impact

For European organizations, this vulnerability poses a risk of system hangs or kernel-level deadlocks on servers and infrastructure using Mellanox network adapters with the affected Linux kernel versions. Such deadlocks can lead to denial of service conditions, impacting availability of critical network services, especially in data centers, cloud environments, and high-performance computing clusters where Mellanox hardware is prevalent. The deadlock can cause kernel panics or require system reboots, leading to downtime and potential disruption of business operations. While the vulnerability does not directly expose confidentiality or integrity risks, the availability impact can be significant for organizations relying on continuous network connectivity and low-latency communications. This is particularly relevant for European financial institutions, telecommunications providers, research institutions, and cloud service providers that deploy Linux servers with Mellanox adapters. The lack of known exploits reduces immediate risk, but the presence of this bug in kernel versions used in production environments means organizations should prioritize patching to avoid unexpected outages.

Mitigation Recommendations

1. Apply the official Linux kernel patches that address CVE-2024-27014 as soon as they become available from trusted sources or Linux distributions. 2. If patching is delayed, consider disabling aRFS on affected systems as a temporary workaround to prevent triggering the deadlock condition. 3. Monitor kernel logs for warnings about circular locking dependencies or deadlocks related to mlx5e and aRFS to detect potential issues early. 4. Conduct thorough testing of network driver updates in staging environments before production deployment to ensure stability. 5. For critical systems, implement high-availability and failover mechanisms to minimize downtime in case of kernel deadlocks. 6. Maintain an inventory of systems using Mellanox adapters and verify kernel versions to identify vulnerable hosts. 7. Coordinate with hardware vendors and Linux distribution maintainers for timely updates and advisories. These steps go beyond generic advice by focusing on specific driver features (aRFS), kernel log monitoring, and operational mitigations tailored to the nature of this deadlock vulnerability.