In the Linux kernel, the following vulnerability has been resolved:
tracing: Restructure trace_clock_global() to never block
It was reported that a fix to the ring buffer recursion detection would cause a hung machine when performing suspend / resume testing. The following backtrace was extracted from debugging that case:
Call Trace: trace_clock_global+0x91/0xa0 __rb_reserve_next+0x237/0x460 ring_buffer_lock_reserve+0x12a/0x3f0 trace_buffer_lock_reserve+0x10/0x50 __trace_graph_return+0x1f/0x80 trace_graph_return+0xb7/0xf0 ? trace_clock_global+0x91/0xa0 ftrace_return_to_handler+0x8b/0xf0 ? pv_hash+0xa0/0xa0 return_to_handler+0x15/0x30 ? ftrace_graph_caller+0xa0/0xa0 ? trace_clock_global+0x91/0xa0 ? __rb_reserve_next+0x237/0x460 ? ring_buffer_lock_reserve+0x12a/0x3f0 ? trace_event_buffer_lock_reserve+0x3c/0x120 ? trace_event_buffer_reserve+0x6b/0xc0 ? trace_event_raw_event_device_pm_callback_start+0x125/0x2d0 ? dpm_run_callback+0x3b/0xc0 ? pm_ops_is_empty+0x50/0x50 ? platform_get_irq_byname_optional+0x90/0x90 ? trace_device_pm_callback_start+0x82/0xd0 ? dpm_run_callback+0x49/0xc0
With the following RIP:
RIP: 0010:native_queued_spin_lock_slowpath+0x69/0x200
Since the fix to the recursion detection would allow a single recursion to happen while tracing, this lead to the trace_clock_global() taking a spin lock and then trying to take it again:
ring_buffer_lock_reserve() { trace_clock_global() { arch_spin_lock() { queued_spin_lock_slowpath() { /* lock taken */ (something else gets traced by function graph tracer) ring_buffer_lock_reserve() { trace_clock_global() { arch_spin_lock() { queued_spin_lock_slowpath() { /* DEAD LOCK! */
Tracing should *never* block, as it can lead to strange lockups like the above.
Restructure the trace_clock_global() code to instead of simply taking a lock to update the recorded "prev_time" simply use it, as two events happening on two different CPUs that calls this at the same time, really doesn't matter which one goes first. Use a trylock to grab the lock for updating the prev_time, and if it fails, simply try again the next time. If it failed to be taken, that means something else is already updating it.
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=212761
| Product | Vendor Advisory | Solution File | Added | Published |
|---|---|---|---|---|
| Alma_linux | — | Upgrade kernel-tools-libs-develUpgrade kernel-rt-debug-kvmUpgrade kernel-rt-kvmUpgrade kernel-rt-modules-extraUpgrade kernel-rt-develUpgrade kernelUpgrade kernel-debugUpgrade kernel-zfcpdump-develUpgrade kernel-headersUpgrade kernel-cross-headersUpgrade kernel-rt-debugUpgrade python3-perfUpgrade kernel-develUpgrade kernel-rt-coreUpgrade bpftoolUpgrade kernel-debug-develUpgrade kernel-rt-debug-coreUpgrade kernel-zfcpdump-modulesUpgrade kernel-toolsUpgrade kernel-rt-debug-modulesUpgrade kernel-rt-modulesUpgrade kernel-zfcpdump-modules-extraUpgrade kernel-zfcpdumpUpgrade kernel-tools-libsUpgrade kernel-modules-extraUpgrade kernel-debug-modulesUpgrade kernel-rt-debug-modules-extraUpgrade kernel-coreUpgrade kernel-rtUpgrade kernel-zfcpdump-coreUpgrade kernel-docUpgrade kernel-debug-coreUpgrade kernel-debug-modules-extraUpgrade kernel-abi-stablelistsUpgrade perfUpgrade kernel-rt-debug-develUpgrade kernel-modules | Aug 12, 2024 | Feb 27, 2024 |
| Amazon Linux Ami 2 | — | Upgrade bpftoolUpgrade kernel-debuginfo-common-x86_64Upgrade bpftool-debuginfoUpgrade kernel-toolsUpgrade kernel-develUpgrade kernel-tools-debuginfoUpgrade kernelUpgrade kernel-debuginfo-common-aarch64Upgrade kernel-tools-develUpgrade python-perfUpgrade kernel-headersUpgrade kernel-debuginfoUpgrade python-perf-debuginfoUpgrade perf-debuginfoUpgrade perfUpgrade kernel-livepatch-4.14.238-182.421 | Apr 25, 2024 | Feb 27, 2024 |
| Debian | — | Upgrade linux | Jul 30, 2024 | Feb 27, 2024 |
| Huawei Euleros 2_0_sp8 | — | Upgrade kernel-toolsUpgrade python-perfUpgrade kernel-tools-libsUpgrade kernel-develUpgrade kernelUpgrade python3-perfUpgrade kernel-headersUpgrade bpftoolUpgrade perf | Jul 23, 2024 | Feb 27, 2024 |
| Huawei Euleros 2_0_sp9 | — | Upgrade kernelUpgrade python3-perfUpgrade kernel-tools-libsUpgrade kernel-tools | Jul 17, 2024 | Feb 27, 2024 |
| Oracle_linux | — | Upgrade kernelUpgrade kernel-uek | Aug 16, 2024 | Feb 27, 2024 |
| Redhat_linux | — | No solution existsUpgrade kernel-rtUpgrade kernel | Aug 12, 2024 | Feb 27, 2024 |
| Rocky_linux | — | Upgrade kernel-debugUpgrade kernel-rt-debug-modulesUpgrade kernel-tools-debuginfoUpgrade kernel-modulesUpgrade kernel-rt-debug-debuginfoUpgrade kernel-coreUpgrade kernel-develUpgrade kernelUpgrade kernel-modules-extraUpgrade kernel-debuginfo-common-x86_64Upgrade kernel-cross-headersUpgrade kernel-debug-develUpgrade kernel-headersUpgrade kernel-debug-modulesUpgrade kernel-rt-debugUpgrade kernel-rt-coreUpgrade kernel-tools-libsUpgrade python3-perfUpgrade kernel-debug-coreUpgrade kernel-rt-debug-coreUpgrade kernel-rt-modules-extraUpgrade kernel-debuginfoUpgrade kernel-rt-debug-kvmUpgrade kernel-rt-debug-develUpgrade python3-perf-debuginfoUpgrade kernel-debug-debuginfoUpgrade perf-debuginfoUpgrade kernel-rt-modulesUpgrade kernel-rtUpgrade kernel-rt-debuginfo-common-x86_64Upgrade kernel-rt-debuginfoUpgrade kernel-debug-modules-extraUpgrade perfUpgrade kernel-rt-debug-modules-extraUpgrade kernel-rt-kvmUpgrade kernel-rt-develUpgrade kernel-tools-libs-develUpgrade kernel-toolsUpgrade bpftool-debuginfoUpgrade bpftool | Aug 22, 2024 | Feb 27, 2024 |
| Suse | — | Upgrade kernel-default-manUpgrade kernel-defaultUpgrade kernel-devel-azureUpgrade kernel-azure-develUpgrade kernel-default-extraUpgrade kernel-zfcpdumpUpgrade kernel-syms-azureUpgrade kernel-macrosUpgrade kernel-azure-baseUpgrade kernel-symsUpgrade kernel-docsUpgrade kernel-develUpgrade kernel-default-baseUpgrade kernel-64kbUpgrade reiserfs-kmp-defaultUpgrade kernel-source-azureUpgrade kernel-sourceUpgrade kernel-64kb-develUpgrade kernel-azureUpgrade kernel-default-develUpgrade kernel-preempt-develUpgrade kernel-preemptUpgrade kernel-obs-build | Aug 9, 2024 | Feb 27, 2024 |
| Ubuntu | — | Upgrade linux-image-4.4.0-254-lowlatencyUpgrade linux-image-virtualUpgrade linux-image-lowlatency-lts-xenialUpgrade linux-image-4.4.0-1132-kvmUpgrade linux-image-awsUpgrade linux-image-virtual-lts-xenialUpgrade linux-image-kvmUpgrade linux-image-4.4.0-1131-awsUpgrade linux-image-4.4.0-254-genericUpgrade linux-image-generic-lts-xenialUpgrade linux-image-lowlatencyUpgrade linux-image-4.4.0-1169-awsUpgrade linux-image-generic | May 20, 2024 | Feb 27, 2024 |
With curated Threat Intelligence, you can see which vulnerabilities truly put you at risk, prioritize what matters most, and act before attackers do.
Explore Intelligence Hub