Maciej S. Szmigiero 8f5c44f953 KVM: x86/mmu: Make HVA handler retpoline-friendly ... When retpolines are enabled they have high overhead in the inner loop inside kvm_handle_hva_range() that iterates over the provided memory area. Let's mark this function and its TDP MMU equivalent __always_inline so compiler will be able to change the call to the actual handler function inside each of them into a direct one. This significantly improves performance on the unmap test on the existing kernel memslot code (tested on a Xeon 8167M machine): 30 slots in use: Test Before After Improvement Unmap 0.0353s 0.0334s 5% Unmap 2M 0.00104s 0.000407s 61% 509 slots in use: Test Before After Improvement Unmap 0.0742s 0.0740s None Unmap 2M 0.00221s 0.00159s 28% Looks like having an indirect call in these functions (and, so, a retpoline) might have interfered with unrolling of the whole loop in the CPU. Signed-off-by: Maciej S. Szmigiero <maciej.szmigiero@oracle.com> Message-Id: <732d3fe9eb68aa08402a638ab0309199fa89ae56.1612810129.git.maciej.szmigiero@oracle.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>		2021-02-09 08:42:09 -05:00
..
mmu_audit.c	KVM: Stop using deprecated jump label APIs	2021-02-04 05:27:29 -05:00
mmu_internal.h	KVM: x86: compile out TDP MMU on 32-bit systems	2021-02-08 14:49:01 -05:00
mmu.c	KVM: x86/mmu: Make HVA handler retpoline-friendly	2021-02-09 08:42:09 -05:00
mmutrace.h	kvm: x86/mmu: Add TDP MMU SPTE changed trace point	2020-11-19 10:57:16 -05:00
page_track.c	KVM: x86/mmu: Use an rwlock for the x86 MMU	2021-02-04 05:27:43 -05:00
paging_tmpl.h	KVM: x86/mmu: Use an rwlock for the x86 MMU	2021-02-04 05:27:43 -05:00
spte.c	KVM: x86: use static calls to reduce kvm_x86_ops overhead	2021-02-04 05:27:30 -05:00
spte.h	KVM: x86/mmu: Flush TLBs after zap in TDP MMU PF handler	2021-02-04 05:27:44 -05:00
tdp_iter.c	KVM: x86/mmu: Protect TDP MMU page table memory with RCU	2021-02-04 05:27:42 -05:00
tdp_iter.h	KVM: x86/mmu: Protect TDP MMU page table memory with RCU	2021-02-04 05:27:42 -05:00
tdp_mmu.c	KVM: x86/mmu: Make HVA handler retpoline-friendly	2021-02-09 08:42:09 -05:00
tdp_mmu.h	KVM: x86: compile out TDP MMU on 32-bit systems	2021-02-08 14:49:01 -05:00