mirror_ubuntu-kernels/arch/powerpc/kernel
Rohan McLure 1df45d78b8 powerpc/64s: Zeroise gprs on interrupt routine entry on Book3S
Zeroise user state in gprs (assign to zero) to reduce the influence of user
registers on speculation within kernel syscall handlers. Clears occur
at the very beginning of the sc and scv 0 interrupt handlers, with
restores occurring following the execution of the syscall handler.

Zeroise GPRS r0, r2-r11, r14-r31, on entry into the kernel for all
other interrupt sources. The remaining gprs are overwritten by
entry macros to interrupt handlers, irrespective of whether or not a
given handler consumes these register values. If an interrupt does not
select the IMSR_R12 IOption, zeroise r12.

Prior to this commit, r14-r31 are restored on a per-interrupt basis at
exit, but now they are always restored on 64bit Book3S. Remove explicit
REST_NVGPRS invocations on 64-bit Book3S. 32-bit systems do not clear
user registers on interrupt, and continue to depend on the return value
of interrupt_exit_user_prepare to determine whether or not to restore
non-volatiles.

The mmap_bench benchmark in selftests should rapidly invoke pagefaults.
See ~0.8% performance regression with this mitigation, but this
indicates the worst-case performance due to heavier-weight interrupt
handlers. This mitigation is able to be enabled/disabled through
CONFIG_INTERRUPT_SANITIZE_REGISTERS.

Reviewed-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Rohan McLure <rmclure@linux.ibm.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Link: https://lore.kernel.org/r/20221201071019.1953023-5-rmclure@linux.ibm.com
2022-12-02 20:46:05 +11:00
..
ptrace powerpc: Fix all occurences of duplicate words 2022-07-25 12:05:15 +10:00
syscalls powerpc/32: fix syscall wrappers with 64-bit arguments 2022-11-01 10:24:09 +11:00
trace powerpc: add definition for pt_regs offset within an interrupt frame 2022-12-02 17:54:08 +11:00
vdso powerpc: Drops STABS_DEBUG from linker scripts 2022-09-30 18:35:52 +10:00
.gitignore
85xx_entry_mapping.S powerpc: Remove CONFIG_FSL_BOOKE 2022-09-26 22:47:37 +10:00
align.c powerpc/inst: Define ppc_inst_t 2021-12-09 22:41:21 +11:00
asm-offsets.c powerpc: add a define for the switch frame size and regs offset 2022-12-02 17:54:08 +11:00
audit.c Revert "powerpc/audit: Convert powerpc to AUDIT_ARCH_COMPAT_GENERIC" 2021-10-27 22:30:32 +11:00
btext.c powerpc: Fix all occurences of duplicate words 2022-07-25 12:05:15 +10:00
cacheinfo.c powerpc: Remove asm/prom.h from all files that don't need it 2022-05-08 22:15:04 +10:00
cacheinfo.h
compat_audit.c Revert "powerpc/audit: Convert powerpc to AUDIT_ARCH_COMPAT_GENERIC" 2021-10-27 22:30:32 +11:00
cpu_setup_6xx.S
cpu_setup_44x.S
cpu_setup_e500.S powerpc: Remove CONFIG_PPC_BOOK3E_MMU 2022-09-26 23:00:14 +10:00
cpu_setup_pa6t.S
cpu_setup_power.c powerpc/cputable: Move __cpu_setup() prototypes out of cputable.h 2022-09-26 22:26:49 +10:00
cpu_setup_ppc970.S
cpu_specs_8xx.h powerpc/cputable: Split cpu_specs[] out of cputable.h 2022-09-26 22:47:13 +10:00
cpu_specs_40x.h powerpc/cputable: Split cpu_specs[] out of cputable.h 2022-09-26 22:47:13 +10:00
cpu_specs_44x.h powerpc/cputable: Split cpu_specs[] out of cputable.h 2022-09-26 22:47:13 +10:00
cpu_specs_47x.h powerpc/cputable: Split cpu_specs[] out of cputable.h 2022-09-26 22:47:13 +10:00
cpu_specs_85xx.h powerpc/cputable: Split cpu_specs[] for mpc85xx and e500mc 2022-09-26 23:00:05 +10:00
cpu_specs_book3s_32.h powerpc/cputable: Split cpu_specs[] out of cputable.h 2022-09-26 22:47:13 +10:00
cpu_specs_book3s_64.h powerpc/cputable: Split cpu_specs[] out of cputable.h 2022-09-26 22:47:13 +10:00
cpu_specs_e500mc.h powerpc/cputable: Split cpu_specs[] for mpc85xx and e500mc 2022-09-26 23:00:05 +10:00
cpu_specs.h powerpc/cputable: Split cpu_specs[] for mpc85xx and e500mc 2022-09-26 23:00:05 +10:00
cputable.c powerpc/cputable: Split cpu_specs[] out of cputable.h 2022-09-26 22:47:13 +10:00
crash_dump.c powerpc updates for 5.19 2022-05-28 11:27:17 -07:00
dawr.c powerpc: Include asm/firmware.h in all users of firmware_has_feature() 2022-06-29 16:45:05 +10:00
dbell.c powerpc/64s/interrupt: Don't enable MSR[EE] in irq handlers unless perf is in use 2021-12-16 21:31:45 +11:00
dma-iommu.c powerpc/iommu: Report the correct most efficient DMA mask for PCI devices 2021-09-30 17:10:17 +10:00
dma-mask.c
dma-swiotlb.c swiotlb: add a SWIOTLB_ANY flag to lift the low memory restriction 2022-04-18 07:21:12 +02:00
dt_cpu_ftrs.c powerpc: move from strlcpy with unused retval to strscpy 2022-08-26 11:02:20 +10:00
early_32.c powerpc: Move C prototypes out of asm-prototypes.h 2022-03-08 22:06:25 +11:00
eeh_cache.c powerpc/kernel: Add __init attribute to eligible functions 2021-12-23 22:33:10 +11:00
eeh_driver.c powerpc: Fix all occurences of duplicate words 2022-07-25 12:05:15 +10:00
eeh_event.c powerpc: fix typos in comments 2022-05-05 22:12:44 +10:00
eeh_pe.c powerpc: Fix all occurences of "the the" 2022-05-22 15:59:43 +10:00
eeh_sysfs.c powerpc: Add missing headers 2022-05-08 22:15:40 +10:00
eeh.c powerpc: fix typos in comments 2022-05-05 22:12:44 +10:00
entry_32.S powerpc: add a define for the switch frame size and regs offset 2022-12-02 17:54:08 +11:00
entry_64.S powerpc: Remove CONFIG_PPC_BOOK3E 2022-09-26 23:00:13 +10:00
epapr_hcalls.S
epapr_paravirt.c powerpc/inst: Define ppc_inst_t 2021-12-09 22:41:21 +11:00
exceptions-64e.S powerpc: add a definition for the marker offset within the interrupt frame 2022-12-02 17:54:08 +11:00
exceptions-64s.S powerpc/64s: Zeroise gprs on interrupt routine entry on Book3S 2022-12-02 20:46:05 +11:00
fadump.c powerpc updates for 5.19 2022-05-28 11:27:17 -07:00
firmware.c powerpc: fix unbalanced node refcount in check_kvm_guest() 2021-10-09 00:15:59 +11:00
fpu.S powerpc/64s/interrupt: handle MSR EE and RI in interrupt entry wrapper 2021-12-16 21:31:45 +11:00
head_8xx.S powerpc: allow minimum sized kernel stack frames 2022-12-02 17:54:09 +11:00
head_32.h powerpc: add a definition for the marker offset within the interrupt frame 2022-12-02 17:54:08 +11:00
head_40x.S powerpc: allow minimum sized kernel stack frames 2022-12-02 17:54:09 +11:00
head_44x.S powerpc: allow minimum sized kernel stack frames 2022-12-02 17:54:09 +11:00
head_64.S powerpc: allow minimum sized kernel stack frames 2022-12-02 17:54:09 +11:00
head_85xx.S powerpc: allow minimum sized kernel stack frames 2022-12-02 17:54:09 +11:00
head_book3s_32.S powerpc: allow minimum sized kernel stack frames 2022-12-02 17:54:09 +11:00
head_booke.h powerpc: add a definition for the marker offset within the interrupt frame 2022-12-02 17:54:08 +11:00
hw_breakpoint_constraints.c powerpc/inst: Define ppc_inst_t 2021-12-09 22:41:21 +11:00
hw_breakpoint.c powerpc/8xx: Fix warning in hw_breakpoint_handler() 2022-11-24 23:31:49 +11:00
idle_6xx.S powerpc/mm: Switch obsolete dssall to .long 2021-12-23 22:35:13 +11:00
idle_64e.S powerpc: add ISA v3.0 / v3.1 wait opcode macro 2022-09-28 19:22:10 +10:00
idle_85xx.S powerpc: Cleanup idle for e500 2022-09-26 23:00:14 +10:00
idle_book3s.S powerpc/idle: Don't corrupt back chain when going idle 2021-10-20 21:37:58 +11:00
idle.c powerpc/idle: Fix return value of __setup() handler 2022-05-04 19:37:46 +10:00
ima_arch.c
interrupt_64.S powerpc/64s: Zeroise gprs on interrupt routine entry on Book3S 2022-12-02 20:46:05 +11:00
interrupt.c powerpc/64s/interrupt: Perf NMI should not take normal exit path 2022-10-18 22:46:19 +11:00
io-workarounds.c powerpc: Fix reverse map real-mode address lookup with huge vmalloc 2021-05-28 22:54:27 +10:00
io.c
iomap.c powerpc: inline iomap accessors 2020-12-04 01:01:09 +11:00
iommu.c powerpc/iommu: Fix iommu_table_in_use for a small default DMA window case 2022-07-28 16:22:13 +10:00
irq_64.c powerpc/64/irq: tidy soft-masked irq replay and improve documentation 2022-09-28 19:22:11 +10:00
irq.c powerpc: allow minimum sized kernel stack frames 2022-12-02 17:54:09 +11:00
isa-bridge.c powerpc: Add missing headers 2022-05-08 22:15:40 +10:00
jump_label.c powerpc: Don't use 'struct ppc_inst' to reference instruction location 2021-06-17 00:09:00 +10:00
kdebugfs.c powerpc: rename powerpc_debugfs_root to arch_debugfs_dir 2021-08-13 22:04:26 +10:00
kgdb.c powerpc: add definition for pt_regs offset within an interrupt frame 2022-12-02 17:54:08 +11:00
kprobes-ftrace.c ftrace: disable preemption when recursion locked 2021-10-27 11:21:49 -04:00
kprobes.c powerpc/kprobes: Use preempt_enable() rather than the no_resched variant 2022-11-24 23:31:49 +11:00
kvm_emul.S
kvm.c powerpc: Remove CONFIG_PPC_BOOK3E_MMU 2022-09-26 23:00:14 +10:00
l2cr_6xx.S powerpc/mm: Switch obsolete dssall to .long 2021-12-23 22:35:13 +11:00
legacy_serial.c powerpc/kernel: Add missing of_node_put() in legacy_serial.c 2022-09-05 17:28:26 +10:00
Makefile powerpc/32: fix syscall wrappers with 64-bit arguments of unaligned register-pairs 2022-10-13 00:49:58 +11:00
mce_power.c powerpc/inst: Define ppc_inst_t 2021-12-09 22:41:21 +11:00
mce.c powerpc/mce: use early_cpu_to_node() in mce_init() 2022-07-27 21:36:04 +10:00
misc_32.S powerpc: allow minimum sized kernel stack frames 2022-12-02 17:54:09 +11:00
misc_64.S powerpc: allow minimum sized kernel stack frames 2022-12-02 17:54:09 +11:00
misc.S powerpc: Avoid link stack corruption in misc asm functions 2021-08-25 13:35:47 +10:00
module_32.c powerpc/modules: Use PPC_LI macros instead of opencoding 2022-05-22 15:58:27 +10:00
module_64.c powerpc/64: Add module check for ELF ABI version 2022-12-02 17:54:07 +11:00
module.c powerpc: Replace PPC64_ELF_ABI_v{1/2} by CONFIG_PPC64_ELF_ABI_V{1/2} 2022-05-19 23:11:29 +10:00
msi.c
note.S
nvram_64.c powerpc: Add missing headers 2022-05-08 22:15:40 +10:00
of_platform.c
optprobes_head.S powerpc: add definition for pt_regs offset within an interrupt frame 2022-12-02 17:54:08 +11:00
optprobes.c powerpc/kprobes: Have optimized_callback() use preempt_enable() 2022-11-24 23:31:49 +11:00
paca.c powerpc: Remove CONFIG_PPC_BOOK3E 2022-09-26 23:00:13 +10:00
pci_32.c powerpc/pci: Enable PCI domains in /proc when PCI bus numbers are not unique 2022-08-25 17:47:08 +10:00
pci_64.c powerpc/pci: Hide pci_device_from_OF_node() for non-powermac code 2022-07-27 21:36:03 +10:00
pci_dn.c powerpc/pci_dn: Add missing of_node_put() 2022-09-05 17:30:25 +10:00
pci_of_scan.c powerpc: Add missing headers 2022-05-08 22:15:40 +10:00
pci-common.c powerpc/pci-common: Fix refcount bug for 'phb->dn' 2022-09-05 17:30:29 +10:00
pci-hotplug.c powerpc: Add missing headers 2022-05-08 22:15:40 +10:00
pmc.c
ppc_save_regs.S powerpc: add definition for pt_regs offset within an interrupt frame 2022-12-02 17:54:08 +11:00
proc_powerpc.c powerpc: Add missing headers 2022-05-08 22:15:40 +10:00
process.c powerpc: allow minimum sized kernel stack frames 2022-12-02 17:54:09 +11:00
prom_init_check.sh powerpc/64: Remove unused prom_init_toc symbols 2022-09-26 20:58:16 +10:00
prom_init.c powerpc/prom_init: drop PROM_BUG() 2022-09-26 20:58:18 +10:00
prom_parse.c
prom.c powerpc: Add device-tree model to the hardware description 2022-09-30 18:35:53 +10:00
reloc_32.S powerpc: Avoid link stack corruption in misc asm functions 2021-08-25 13:35:47 +10:00
reloc_64.S powerpc/64: avoid using r13 in relocate 2022-09-28 19:22:13 +10:00
rtas_entry.S powerpc/rtas: Fix RTAS MSR[HV] handling for Cell 2022-08-26 08:41:54 +10:00
rtas_flash.c powerpc: fix typos in comments 2022-05-05 22:12:44 +10:00
rtas_pci.c powerpc: Add missing headers 2022-05-08 22:15:40 +10:00
rtas-proc.c powerpc: Add missing headers 2022-05-08 22:15:40 +10:00
rtas-rtc.c powerpc: Remove asm/prom.h from all files that don't need it 2022-05-08 22:15:04 +10:00
rtas.c powerpc/rtas: block error injection when locked down 2022-09-28 19:22:14 +10:00
rtasd.c powerpc: Remove asm/prom.h from all files that don't need it 2022-05-08 22:15:04 +10:00
secure_boot.c
security.c powerpc: Remove CONFIG_PPC_FSL_BOOK3E 2022-09-26 23:00:13 +10:00
secvar-ops.c
secvar-sysfs.c powerpc/secvar: fix refcount leak in format_show() 2022-03-08 22:07:41 +11:00
setup_32.c powerpc: Change CONFIG_E500 to CONFIG_PPC_E500 2022-09-26 23:00:13 +10:00
setup_64.c powerpc/64s/interrupt: halt early boot interrupts if paca is not set up 2022-09-28 19:22:13 +10:00
setup-common.c powerpc: Add hardware description string 2022-09-30 18:35:52 +10:00
setup.h powerpc: Remove CONFIG_PPC_BOOK3E 2022-09-26 23:00:13 +10:00
signal_32.c powerpc: Include all arch-specific syscall prototypes 2022-09-28 19:22:08 +10:00
signal_64.c powerpc/signal: Update comment for clarity 2022-07-28 16:22:14 +10:00
signal.c powerpc/signal: Report minimum signal frame size to userspace via AT_MINSIGSTKSZ 2022-05-19 23:11:26 +10:00
signal.h powerpc: Don't add __powerpc_ prefix to syscall entry points 2022-10-07 00:59:54 +11:00
smp-tbsync.c
smp.c powerpc: allow minimum sized kernel stack frames 2022-12-02 17:54:09 +11:00
stacktrace.c powerpc: allow minimum sized kernel stack frames 2022-12-02 17:54:09 +11:00
static_call.c powerpc/32: Add support for out-of-line static calls 2021-10-22 15:22:05 +11:00
suspend.c
swsusp_32.S powerpc/mm: Switch obsolete dssall to .long 2021-12-23 22:35:13 +11:00
swsusp_64.c powerpc/machdep: Remove stale functions from ppc_md structure 2021-10-22 15:22:05 +11:00
swsusp_85xx.S powerpc: Remove CONFIG_FSL_BOOKE 2022-09-26 22:47:37 +10:00
swsusp_asm64.S powerpc/64: asm use consistent global variable declaration and access 2022-09-28 19:22:12 +10:00
swsusp.c
sys_ppc32.c powerpc/32: fix syscall wrappers with 64-bit arguments 2022-11-01 10:24:09 +11:00
syscall.c powerpc: Provide syscall wrapper 2022-09-28 19:22:09 +10:00
syscalls.c powerpc: Adopt SYSCALL_DEFINE for arch-specific syscall handlers 2022-09-28 19:22:08 +10:00
sysfs.c powerpc: Remove CONFIG_PPC_FSL_BOOK3E 2022-09-26 23:00:13 +10:00
systbl.c powerpc: Don't add __powerpc_ prefix to syscall entry points 2022-10-07 00:59:54 +11:00
tau_6xx.c powerpc: Move C prototypes out of asm-prototypes.h 2022-03-08 22:06:25 +11:00
time.c powerpc: remove the last remnants of cputime_t 2022-11-24 23:31:48 +11:00
tm.S powerpc: add definition for pt_regs offset within an interrupt frame 2022-12-02 17:54:08 +11:00
traps.c powerpc/64s: early boot machine check handler 2022-09-28 19:22:13 +10:00
ucall.S
udbg_16550.c powerpc: Add support for early debugging via Serial 16550 console 2022-09-28 19:22:09 +10:00
udbg.c powerpc: Add support for early debugging via Serial 16550 console 2022-09-28 19:22:09 +10:00
uprobes.c powerpc: Reject probes on instructions that can't be single stepped 2022-05-06 00:00:20 +10:00
vdso32_wrapper.S powerpc/vdso: Merge vdso64 and vdso32 into a single directory 2022-02-12 22:47:43 +11:00
vdso64_wrapper.S powerpc/vdso: Merge vdso64 and vdso32 into a single directory 2022-02-12 22:47:43 +11:00
vdso.c - Yu Zhao's Multi-Gen LRU patches are here. They've been under test in 2022-10-10 17:53:04 -07:00
vecemu.c powerpc/inst: Define ppc_inst_t 2021-12-09 22:41:21 +11:00
vector.S powerpc/64: asm use consistent global variable declaration and access 2022-09-28 19:22:12 +10:00
vmlinux.lds.S powerpc: Fix writable sections being moved into the rodata region 2022-11-16 21:37:14 +11:00
watchdog.c powerpc/watchdog: introduce a NMI watchdog's factor 2022-07-27 21:36:02 +10:00