Nicholas Piggin 21641bd9a7 lazy tlb: fix hotplug exit race with MMU_LAZY_TLB_SHOOTDOWN ... CPU unplug first calls __cpu_disable(), and that's where powerpc calls cleanup_cpu_mmu_context(), which clears this CPU from mm_cpumask() of all mms in the system. However this CPU may still be using a lazy tlb mm, and its mm_cpumask bit will be cleared from it. The CPU does not switch away from the lazy tlb mm until arch_cpu_idle_dead() calls idle_task_exit(). If that user mm exits in this window, it will not be subject to the lazy tlb mm shootdown and may be freed while in use as a lazy mm by the CPU that is being unplugged. cleanup_cpu_mmu_context() could be moved later, but it looks better to move the lazy tlb mm switching earlier. The problem with doing the lazy mm switching in idle_task_exit() is explained in commit bf2c59fce4 ("sched/core: Fix illegal RCU from offline CPUs"), which added a wart to switch away from the mm but leave it set in active_mm to be cleaned up later. So instead, switch away from the lazy tlb mm at sched_cpu_wait_empty(), which is the last hotplug state before teardown (CPUHP_AP_SCHED_WAIT_EMPTY). This CPU will never switch to a user thread from this point, so it has no chance to pick up a new lazy tlb mm. This removes the lazy tlb mm handling wart in CPU unplug. With this, idle_task_exit() is not needed anymore and can be cleaned up. This leaves the prototype alone, to be cleaned after this change. herton: took the suggestions from https://lore.kernel.org/all/87jzvyprsw.ffs@tglx/ and made adjustments on the initial patch proposed by Nicholas. Link: https://lkml.kernel.org/r/20230524060455.147699-1-npiggin@gmail.com Link: https://lore.kernel.org/all/20230525205253.E2FAEC433EF@smtp.kernel.org/ Link: https://lkml.kernel.org/r/20241104142318.3295663-1-herton@redhat.com Fixes: 2655421ae6 ("lazy tlb: shoot lazies, non-refcounting lazy tlb mm reference handling scheme") Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Herton R. Krzesinski <herton@redhat.com> Suggested-by: Thomas Gleixner <tglx@linutronix.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Michael Ellerman <mpe@ellerman.id.au> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>		2025-01-13 22:40:42 -08:00
..
affinity.h
autogroup.h
clock.h
cond_resched.h
coredump.h	mm: move mm flags to mm_types.h	2024-11-05 16:56:26 -08:00
cpufreq.h
cputime.h
deadline.h
debug.h
ext.h	sched_ext: Change for v6.13	2024-11-20 10:08:00 -08:00
hotplug.h	lazy tlb: fix hotplug exit race with MMU_LAZY_TLB_SHOOTDOWN	2025-01-13 22:40:42 -08:00
idle.h
init.h
isolation.h
jobctl.h
loadavg.h
mm.h	Revert "mm: introduce PF_MEMALLOC_NORECLAIM, PF_MEMALLOC_NOWARN"	2024-10-09 12:47:19 -07:00
nohz.h
numa_balancing.h
posix-timers.h
prio.h
rseq_api.h
rt.h
sd_flags.h
signal.h	signal: Provide ignored_posix_timers list	2024-11-07 02:14:45 +01:00
smt.h
stat.h
sysctl.h
task_flags.h
task_stack.h	Scheduler changes for v6.13:	2024-11-19 14:16:06 -08:00
task.h
thread_info_api.h
topology.h
types.h
user.h
vhost_task.h
wake_q.h
xacct.h