This commit further avoids conflation of rcuperf with the kernel's perf
feature by renaming kernel/rcu/rcuperf.c to kernel/rcu/rcuscale.c, and
also by similarly renaming the functions and variables inside this file.
This has the side effect of changing the names of the kernel boot
parameters, so kernel-parameters.txt and ver_functions.sh are also
updated. The rcutorture --torture type was also updated from rcuperf
to rcuscale.
[ paulmck: Fix bugs located by Stephen Rothwell. ]
Reported-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, parse-torture.sh looks at the fifth field of torture-test
console output for the version number. This works fine for rcutorture,
but not for scftorture, which lacks the pointer field. This commit
therefore adjusts matching lines so that the parse-console.sh awk script
always sees the version number as the first field in the lines passed
to it.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
In the dim distant past, qemu commands needed to be run from the
rcutorture directory, but this is no longer the case. This commit
therefore removes the now-useless "cd $KVM" from the kvm-test-1-run.sh
script.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, the qemu command is constructed twice, once to dump it
to the qemu-cmd file and again to execute it. This is of course an
accident waiting to happen, but is done to ensure that the remainder
of the script has an accurate idea of the running qemu command's PID.
This commit therefore places both the qemu command and the PID capture
into a new temporary file and sources that temporary file. Thus the
single construction of the qemu command into the qemu-cmd file suffices
for both purposes.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds a script that transforms qemu-cmd files to allow them
and the corresponding kernels to be run in contexts other than the one
that they were created for, including on systems other than the one that
they were built on. For example, this allows the build products from a
--buildonly run to be transformed to allow distributed rcutorture testing.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Using --kcsan when the compiler does not support KCSAN results in this:
:CONFIG_KCSAN=y: improperly set
:CONFIG_KCSAN_REPORT_ONCE_IN_MS=100000: improperly set
:CONFIG_KCSAN_VERBOSE=y: improperly set
:CONFIG_KCSAN_INTERRUPT_WATCHER=y: improperly set
Clean KCSAN run in /home/git/linux-rcu/tools/testing/selftests/rcutorture/res/2020.06.16-09.53.16
This is a bit obtuse, so this commit adds checks resulting in this:
:CONFIG_KCSAN=y: improperly set
:CONFIG_KCSAN_REPORT_ONCE_IN_MS=100000: improperly set
:CONFIG_KCSAN_VERBOSE=y: improperly set
:CONFIG_KCSAN_INTERRUPT_WATCHER=y: improperly set
Compiler or architecture does not support KCSAN!
Did you forget to switch your compiler with --kmake-arg CC=<cc-that-supports-kcsan>?
Suggested-by: Marco Elver <elver@google.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Acked-by: Marco Elver <elver@google.com>
Currently, kvm-recheck.sh complains that qemu failed for --buildonly
runs, which is sort of true given that qemu can hardly succeed if not
invoked in the first place. Nevertheless, this commit swaps the order
of checks in kvm-recheck.sh so that --buildonly runs will be summarized
more straightforwardly.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
We need to pass the arguments provided to --kmake-arg to all make
invocations. In particular, the make invocations generating the configs
need to see the final make arguments, e.g. if config variables depend on
particular variables that are passed to make.
For example, when using '--kcsan --kmake-arg CC=clang-11', we would lose
CONFIG_KCSAN=y due to 'make oldconfig' not seeing that we want to use a
compiler that supports KCSAN.
Signed-off-by: Marco Elver <elver@google.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit pulls the simple pattern-based error detection from the
console log into a new console-badness.sh file. This will enable future
commits to end a run on the first error.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
When bisecting RCU issues, it is often the case that the first error in
an unsuccessful run will happen quickly, but that a successful run must
go on for some time in order to obtain a sufficiently low false-negative
error rate. In many cases, a bisection requires multiple concurrent
runs, in which case the first failure in any run indicates failure,
pure and simple. In such cases, it would speed things up greatly if
the first failure terminated all runs.
This commit therefore adds scripting that checks for a file named "STOP"
in the top-level results directory, terminating the run when it appears.
Note that in-progress builds will continue until completion, but future
builds and all runs will be cut short.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
One reason to do a --buildonly run is to use the build products elsewhere,
for example, to do the actual test on some other system. Part of doing
the test is the actual qemu command, which is not currently produced
by --buildonly runs. This commit therefore causes --buildonly runs to
create this file.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Leaving off the kvm.sh script's --cpus argument results in the script
testing the scenarios sequentially, which can be quite slow. However,
having to specify the actual number of CPUs can be error-prone.
This commit therefore adds a --allcpus argument that causes kvm.sh to
use all available CPUs.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The identify_qemu_vcpus bash function can return numbers including
whitespace characters, which can be a bit annoying in some bash
dollar-sign substitutions. This commit therefore strips all spaces and
tabs from the value that identify_qemu_vcpus outputs.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The current console parsing assumes that console lines containing "!!!"
are statistics lines from which it can parse the number of rcutorture
too-short grace-period failures. This prints confusing output for
other problems, including memory exhaustion. This commit therefore
differentiates between these cases and prints an appropriate error string.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The torture-test recheck logic fails to set the configfile variable to
the current scenario, so this commit properly initializes this variable.
This change isn't critical given that all errors for a given scenario
follow that scenario's heading, but it is easier on the eyes to repeat it.
And this repetition also prevents confusion as to whether a given message
goes with the previous heading or the next one.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds a kvm-check-branches.sh script that takes a list
of commits and commit ranges and runs a short rcutorture test on all
scenarios on each specified commit. A summary is printed at the end, and
the script returns success if all rcutorture runs completed without error.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
On some (probably misconfigured) systems, the torture-test scripting
will cause qemu to complain about missing EFI firmware, often because
qemu is trying to traverse broken symbolic links to find that firmware.
Which is a bit silly given that the default torture-test guest OS has
but a single binary for its userspace, and thus is unlikely to do much
in the way of networking in any case.
This commit therefore avoids such problems by specifying "-net none"
to qemu unless the TORTURE_QEMU_INTERACTIVE environment variable is set
(for example, by having specified "--interactive" to kvm.sh), in which
case "-net nic -net user" is specified to qemu instead. Either choice
may be overridden by specifying the "-net" argument of your choice to
the kvm.sh "--qemu-args" parameter.
Link: https://lore.kernel.org/lkml/20190701141403.GA246562@google.com
Reported-by: Joel Fernandes <joel@joelfernandes.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
This commit renames the rcutorture config/refperf to config/refscale to
further avoid conflation with the Linux kernel's perf feature.
Reported-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit further avoids conflation of refperf with the kernel's perf
feature by renaming kernel/rcu/refperf.c to kernel/rcu/refscale.c,
and also by similarly renaming the functions and variables inside
this file. This has the side effect of changing the names of the
kernel boot parameters, so kernel-parameters.txt and ver_functions.sh
are also updated.
The rcutorture --torture type remains refperf, and this will be
addressed in a separate commit.
Reported-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The old Kconfig option name is all too easy to conflate with the
unrelated "perf" feature, so this commit renames RCU_REF_PERF_TEST to
RCU_REF_SCALE_TEST.
Reported-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, it is necessary to manually edit the console output to see
anything more than statistics, and sometimes the statistics can indicate
outliers that need more investigation. This commit therefore dumps out
the per-experiment measurements, sorted in ascending order, just before
dumping out the statistics.
Cc: Joel Fernandes (Google) <joel@joelfernandes.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The experiment-number column is currently labeled "Threads", which is
misleading at best. This commit therefore relabels it as "Runs", and
adjusts the scripts accordingly.
Cc: Joel Fernandes (Google) <joel@joelfernandes.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit updates the rcutorture scripting to include the new refperf
torture-test module.
Cc: Joel Fernandes (Google) <joel@joelfernandes.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
fixes.2020.04.27a: Miscellaneous fixes.
kfree_rcu.2020.04.27a: Changes related to kfree_rcu().
rcu-tasks.2020.04.27a: Addition of new RCU-tasks flavors.
stall.2020.04.27a: RCU CPU stall-warning updates.
torture.2020.05.07a: Torture-test updates.
Make it a bit easier to apply KASAN to rcutorture runs with a new --kasan
argument, again leveraging the config_override_param() bash function.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit saves a few lines of code by also using the bash
config_override_param() to set the initial list of Kconfig options from
the CFcommon file. While in the area, it makes this function capable of
update-in-place on the file containing the cumulative Kconfig options,
thus avoiding annoying changes when adding another source of options.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit applies config_override_param() to allow scenario-specific
Kconfig options to override those in CFcommon. This in turn will allow
additional Kconfig options to be placed in CFcommon, for example, an
option common to all but a few scenario can be placed in CFcommon and
then overridden in those few scenarios. Plus this change saves one
whole line of code.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, attempting to override a --kcsan default with a --kconfig
option might or might not work. However, it would be good to allow the
user to adjust the --kcsan defaults, for example, to specify a different
time for CONFIG_KCSAN_REPORT_ONCE_IN_MS. This commit therefore uses the
new config_override_param() bash function to apply the --kcsan defaults
and then apply the --kconfig options, which allows this overriding
to occur.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit introduces a config_override_param() bash function that
folds in an additional set of Kconfig options. This is initially applied
to fold in the --kconfig kvm.sh parameter, but later commits will also
apply it to the Kconfig options added by the --kcsan kvm.sh parameter.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The #CHECK# directives that can be present in CFcommon and in the
rcutorture scenario Kconfig files are both copied to ConfigFragment
and grepped out of the two directive files and added to ConfigFragment.
This commit therefore removes the redundant "grep" commands and takes
advantage of the consequent opportunity to simplify redirection.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The KCSAN tool emits a great many warnings for current kernels, for
example, a one-hour run of the full set of rcutorture scenarios results
in no fewer than 3252 such warnings, many of which are duplicates
or are otherwise closely related. This commit therefore introduces
a kcsan-collapse.sh script that maps these warnings down to a set of
function pairs (22 of them given the 3252 individual warnings), placing
the resulting list in decreasing order of frequency of occurrence into
a kcsan.sum file. If any KCSAN warnings were produced, the pathname of
this file is emitted at the end of the summary of the rcutorture runs.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Although the existing --kconfig argument can be used to run KCSAN for
an rcutorture test, it is not as straightforward as one might like:
--kconfig "CONFIG_DEBUG_INFO=y CONFIG_KCSAN=y \
CONFIG_KCSAN_ASSUME_PLAIN_WRITES_ATOMIC=n \
CONFIG_KCSAN_REPORT_VALUE_CHANGE_ONLY=n \
CONFIG_KCSAN_REPORT_ONCE_IN_MS=100000 \
CONFIG_KCSAN_VERBOSE=y CONFIG_KCSAN_INTERRUPT_WATCHER=y"
This commit therefore adds a "--kcsan" argument that emulates the above
--kconfig command. Note that if you specify a Kconfig option using
-kconfig that conflicts with one that --kcsan adds, you get whatever
the script and the build system decide to give you.
Cc: Marco Elver <elver@google.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The number of CPUs is tuned to allow "4*CFLIST TREE10" on a large system,
up from "3*CFLIST TREE10" previously.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
System hangs or killed rcutorture guest OSes can result in truncated
"Reader Pipe:" lines, which can in turn result in false-positive
reader-batch near-miss warnings. This commit therefore adjusts the
reader-batch checks to account for possible line truncation.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds a TRACE02 scenario which enables preemption and RCU
Tasks Trace IPIs, more specifically, disabling heavyweight readers.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Here are 3 SPDX patches for 5.7-rc1.
One fixes up the SPDX tag for a single driver, while the other two go
through the tree and add SPDX tags for all of the .gitignore files as
needed.
Nothing too complex, but you will get a merge conflict with your current
tree, that should be trivial to handle (one file modified by two things,
one file deleted.)
All 3 of these have been in linux-next for a while, with no reported
issues other than the merge conflict.
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
-----BEGIN PGP SIGNATURE-----
iG0EABECAC0WIQT0tgzFv3jCIUoxPcsxR9QN2y37KQUCXodg5A8cZ3JlZ0Brcm9h
aC5jb20ACgkQMUfUDdst+ykySQCgy9YDrkz7nWq6v3Gohl6+lW/L+rMAnRM4uTZm
m5AuCzO3Azt9KBi7NL+L
=2Lm5
-----END PGP SIGNATURE-----
Merge tag 'spdx-5.7-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/spdx
Pull SPDX updates from Greg KH:
"Here are three SPDX patches for 5.7-rc1.
One fixes up the SPDX tag for a single driver, while the other two go
through the tree and add SPDX tags for all of the .gitignore files as
needed.
Nothing too complex, but you will get a merge conflict with your
current tree, that should be trivial to handle (one file modified by
two things, one file deleted.)
All three of these have been in linux-next for a while, with no
reported issues other than the merge conflict"
* tag 'spdx-5.7-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/spdx:
ASoC: MT6660: make spdxcheck.py happy
.gitignore: add SPDX License Identifier
.gitignore: remove too obvious comments
This commit enables the KCSAN Kconfig options that (1) detect data
races between reads and writes even when the writes do not change the
variable's value and (2) detect data races involving plain C-language
writes. These changes only affect scripted rcutorture runs and can be
overridden using the kvm.sh --kconfig argument.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, kvm-find-errors.sh gives a usage prompt when given a bad
directory, but then soldiers on, giving a series of confusing error
messages. This commit therefore prints an error message and exits when
given a bad directory, hopefully reducing confusion.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
When running the default list of tests, the run summary of a successful
(that is, failed to find any errors) run fits easily on a 24-line screen.
But a run with something like "--configs '5*CFLIST'" will be 80 lines long,
and it is all too easy to miss a failure message when scrolling back.
This commit therefore prints out the number of runs with failing builds
or runtime failures, but only if there are any such failures.
For example, a run with a single build error and a single runtime error
would print two lines like this:
1 runs with build errors.
1 runs with runtime errors.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The small-system rcutorture configurations have served us well for a great
many years, but it is now time to add a larger one. This commit does
just that, but does not add it to the defaults in CFLIST. This allows
the kvm.sh argument '--configs "4*CFLIST TREE10" to run four instances
of each of the default configurations concurrently with one instance of
the large configuration.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The names of the per-test results directories are of the form
2019.11.29-20:42:19. This works, but the ":" characters make
tab-based shell name completion a bit onerous because the user must
remember to include a quote character somewhere before the first ":".
This commit therefore changes the ":" characters to periods, as in
2019.12.01-20.48.01", which allows tab-based completion to work more
naturally.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The bootparam_hotplug_cpu() bash function was checking for CPU-hotplug
kernel-boot parameters from --bootargs, but that check was specific to
rcutorture ("rcutorture\.onoff_"). This commit therefore makes this
check also work for locktorture ("torture\.onoff_").
Note that rcuperf does not do CPU-hotplug operations, so it is not
necessary to make a similar change for rcuperf.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
On some kernels, concurrent calls to the lscpu command result in severe
slowdowns. For example, on v4.16, a single lscpu invocation takes about
two milliseconds, four concurrent invocations more than two seconds,
and 16 concurrent invocations more than 20 seconds. Given that the only
goal is to learn the number of CPUs, invoking lscpu but once suffices.
This commit therefore invokes lscpu early in kvm.sh execution, setting
the initial value of the TORTURE_ALLOTED_CPUS environment variable.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
On a large system, it can be convenient to tell rcutorture to run
several instances of the default scenarios. Currently, this requires
explicitly listing them, for example, "--configs '2*SRCU-N 2*SRCU-P...'".
Although this works, it is rather inconvenient.
This commit therefore allows "CFLIST" to be specified to indicate the
default list of scenarios called out in the relevant CFLIST file, for
example, for RCU, tools/testing/selftests/rcutorture/configs/rcu/CFLIST.
In addition, multipliers may be used to run multiple instances of all
the scenarios. For example, on a 256-CPU system, "--configs '3*CFLIST'"
would run three instances of each scenario concurrently with one CPU
left over. Thus "--configs '3*CFLIST TINY01'" would exactly consume all
256 CPUs, which makes rcutorture's jitter feature more effective.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds the worst-case results from any call_rcu()
forward-progress tests to the rcutorture test-summary output.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The rcutorture scripting uses the mpstat command to determine how much
the system is being used, and adjusts make's -j argument accordingly.
However, mpstat isn't installed by default, so it would be good if the
scripting does something useful when mpstat isn't present.
This commit therefore makes the scripts assumes that if mpstat is not
present, they are free to use all the CPUs.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, jitter.sh assumes that the underlying hypervisor will be
configured with all CPUs hotpluggable, with the possible exception
of CPU 0. However, there are installations where the hypervisor
prohibits offlining, which breaks jitter.sh. This commit therefore
lists the CPUs that cannot be offlined up front, and checks for the
case where no CPU can be offlined in the loop.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The dracut scripting does not work on all platforms, and there are no
known failures from the init binary based on the statically linked C
program. This commit therefore removes the dracut scripting so that the
statically linked C program is always used to create the init "script".
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
In many environments, gawk provides systime(), but awk doesn't.
This commit therefore changes awk scripts using systime() to instead be
gawk scripts.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
A number of mainstream CPU families are no longer capable of building
kernels having CONFIG_SMP=y and CONFIG_HOTPLUG_CPU=n, so this commit
removes this combination from the rcutorture scenarios having it.
People wishing to try out this combination may still do so using the
"--kconfig CONFIG_HOTPLUG_CPU=n CONFIG_SUSPEND=n CONFIG_HIBERNATION=n"
argument to the tools/testing/selftests/rcutorture/bin/kvm.sh script
that is used to run rcutorture.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Since commit 05f415715c ("rcu: Speed up expedited GPs when interrupting
RCU reader") in v5.0 and through v5.1, booting with the threadirqs kernel
boot parameter caused self-deadlocks, which can be reproduced using the
following command on an 8-CPU system:
tools/testing/selftests/rcutorture/bin/kvm.sh --duration 5 --configs "TREE03" --bootargs "threadirqs"
This commit therefore adds the threadirqs kernel boot parameter to
the TREE03 rcutorture scenario in order to more quickly detect future
similar bugs.
Link: http://lkml.kernel.org/r/20190626135447.y24mvfuid5fifwjc@linutronix.de
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Joel Fernandes <joel@joelfernandes.org>
The kvm-test-1-run.sh script says 'test -z "last_ts"' which always
evaluates to true (AKA zero) regardless of the value of the last_ts shell
variable. This commit therefore inserts the needed dollar sign ("$").
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
When trace_printk() is used, a message including "BUG" is printed to
the console, which fools the rcutorture scripting into believing that
the corresponding test scenario failed. This commit therefore filters
out this particular instance of "BUG", thus avoiding the false-positive
test-failure report.
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
The current rcutorture scripts unconditionally do "make clean", which is
a good way of getting the needed testing done despite any imperfections in
Makefile dependency tracking. However, this can be a bit irritating when
repeatedly running a single scenario after small changes, for example,
when debugging a problem that affects only a single scenario. This commit
therefore adds a --trust-make argument that suppresses the "make clean".
Even when using ccache, this speeds up kernel builds by up to almost an
order of magnitude on my laptop.
Reported-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
Currently, rcutorture will use relatively few CPUs to build the kernel
on a busy system, which is often as it should be. However, if the user
has used the --cpus argument to dedicate a specified number of CPUs to
this torture test, it would be good if the kernel build also made use
of them. This commit therefore changes the cpus2use.sh script to use
--cpus when specified and to do the idleness calculations otherwise.
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
For historical reasons, rcutorture places its build products in a
tools/testing/selftests/rcutorture/b1 directory using the O= kbuild
command-line argument. However, doing this requires that the source
directory be pristine: Not just "make clean" pristine, but instead "make
mrproper" (or, equivalently, "make distclean") pristine. Therefore,
rcutorture executes a "make mrproper" before each build. Unfortunately,
"make mrproper" has the side effect of removing pretty much everything,
including tags files and cscope databases, which can be inconvenient
to people whose workflow centers around a single source tree.
This commit therefore makes rcutorture do the build directly in the
source directory, removing the need for "make mrproper". This works
because all needed build products are moved to their proper place in the
"res" directory immediately after the build completes, so that multiple
rcutorture kernels can still run concurrently.
Reported-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
Currently qemu output appears on standard output, but is inaccessible
later on. This commit therefore captures this output and causes
kvm-recheck.sh to output this output if QEMU gave a non-zero non-137
exit code. (And exit code of 137 indicates that QEMU was killed, in
which case we want to know about the hang rather than the fact that
QEMU was killed.)
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
In one of my rcutorture tests the TSC clocksource got marked unstable
due to a large difference in the TSC value. I'm not sure if the guest
run for a long time with disabled interrupts or if the host was very
busy and didn't schedule the guest for some time.
I took a look on the qemu/KVM options and decided to update the options:
- Use kvm{32|64} as CPU. We could probably use `host' (like ARM does)
for maximum available features but since we don't run any userland I'm
not sure if it makes any difference.
- Drop the "noapic" option. There is no history why the APIC was disabled,
I see no reason for it. Once old qemu versions fade away, we can add
"x2apic=on,tsc-deadline=on,hypervisor=on,tsc_adjust=on".
- Additional config options. It ensures that the kernel knowns that it
runs as a kvm guest and can use virt devices like the kvm-clock as
clocksource. The kvm-clock was the main motivation here.
- I didn't add a random HW device. It would make the random device ready
earlier (not it doesn't complete the initialisation at all) but I
doubt that there is any need for this.
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
[ paulmck: The world is not quite ready for CONFIG_PARAVIRT_SPINLOCKS=y
and x2apic, so they are omitted for the time being. ]
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
I have been showing off a trivial RCU implementation for non-preemptive
environments for some time now:
#define rcu_read_lock()
#define rcu_read_unlock()
#define rcu_dereference(p) READ_ONCE(p)
#define rcu_assign_pointer(p, v) smp_store_release(&(p), (v))
void synchronize_rcu(void)
{
int cpu;
for_each_online_cpu(cpu)
sched_setaffinity(current->pid, cpumask_of(cpu));
}
Trivial or not, as the old saying goes, "if it ain't tested, it don't
work!". This commit therefore adds a "trivial" flavor to rcutorture
and a corresponding TRIVIAL test scenario. This variant does not handle
CPU hotplug, which is unconditionally enabled on x86 for post-v5.1-rc3
kernels, which is why the TRIVIAL.boot says "rcutorture.onoff_interval=0".
This commit actually does handle CONFIG_PREEMPT=y kernels, but only
because it turns back the Linux-kernel clock in order to provide these
alternative definitions (or the moral equivalent thereof):
#define rcu_read_lock() preempt_disable()
#define rcu_read_unlock() preempt_enable()
In CONFIG_PREEMPT=n kernels without debugging, these are equivalent to
empty macros give or take a compiler barrier. However, the have been
successfully tested with actual empty macros as well.
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
[ paulmck: Fix symbol issue reported by kbuild test robot <lkp@intel.com>. ]
[ paulmck: Work around sched_setaffinity() issue noted by Andrea Parri. ]
[ paulmck: Add rcutorture.shuffle_interval=0 to TRIVIAL.boot to fix
interaction with shuffler task noted by Peter Zijlstra. ]
Tested-by: Andrea Parri <andrea.parri@amarulasolutions.com>
Because TREE01 can end up running more vCPUs that physical CPUs,
hammering these shortchanged CPUs with tight loops containing call_rcu()
invocations seems a bit like overkill. This commit therefore exempts
TREE01 from rcutorture's forward-progress testing.
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
This commit provides a rudimentary Makefile that runs a 10-minute
rcutorture test on scenario TREE01. This must be run on a system capable
of spawning virtual machines and with everything installed to permit
building Linux kernels.
Reported-by: Shuah Khan <shuah@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
This commit causes both kvm-find-errors.sh and kvm-recheck.sh to provide
an exit status based on whether or not errors were located. In the
case of kvm-recheck.sh, this will be the error status of the last run.
This change allows these commands to be used in scripting and Makefiles
to automatically report failed rcutorture runs.
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
jitter.sh currently does not add CPU0 to the list of CPUs for adding of
jitter. Let us add it to this list even when it is not hot-pluggable.
Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
The rcutorture jitter.sh script selects a random CPU but does not check
if it is offline or online. This leads to taskset errors many times. On
my machine, hyper threading is disabled so half the cores are offline
causing taskset errors a lot of times. Let us fix this by checking from
only the online CPUs on the system.
Cc: rcu@vger.kernel.org
Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
-----BEGIN PGP SIGNATURE-----
iQJEBAABCAAuFiEEwPw5LcreJtl1+l5K99NY+ylx4KYFAlzR0AAQHGF4Ym9lQGtl
cm5lbC5kawAKCRD301j7KXHgpo0MD/47D1kBK9rGzkAwIz1Jkh1Qy/ITVaDJzmHJ
UP5uncQsgKFLKMR1LbRcrWtmk2MwFDNULGbteHFeCYE1ypCrTgpWSp5+SJluKd1Q
hma9krLSAXO9QiSaZ4jafshXFIZxz6IjakOW8c9LrT80Ze47yh7AxiLwDafcp/Jj
x6NW790qB7ENDtfarDkZk14NCS8HGLRHO5B21LB+hT0Kfbh0XZaLzJdj7Mck1wPA
VT8hL9mPuA++AjF7Ra4kUjwSakgmajTa3nS2fpkwTYdztQfas7x5Jiv7FWxrrelb
qbabkNkWKepcHAPEiZR7o53TyfCucGeSK/jG+dsJ9KhNp26kl1ci3frl5T6PfVMP
SPPDjsKIHs+dqFrU9y5rSGhLJqewTs96hHthnLGxyF67+5sRb5+YIy+dcqgiyc/b
TUVyjCD6r0cO2q4v9VhwnhOyeBUA9Rwbu8nl7JV5Q45uG7qI4BC39l1jfubMNDPO
GLNGUUzb6ER7z6lYINjRSF2Jhejsx8SR9P7jhpb1Q7k/VvDDxO1T4FpwvqWFz9+s
Gn+s6//+cA6LL+42eZkQjvwF2CUNE7TaVT8zdb+s5HP1RQkZToqUnsQCGeRTrFni
RqWXfW9o9+awYRp431417oMdX/LvLGq9+ZtifRk9DqDcowXevTaf0W2RpplWSuiX
RcCuPeLAVg==
=Ot0g
-----END PGP SIGNATURE-----
Merge tag 'for-5.2/block-20190507' of git://git.kernel.dk/linux-block
Pull block updates from Jens Axboe:
"Nothing major in this series, just fixes and improvements all over the
map. This contains:
- Series of fixes for sed-opal (David, Jonas)
- Fixes and performance tweaks for BFQ (via Paolo)
- Set of fixes for bcache (via Coly)
- Set of fixes for md (via Song)
- Enabling multi-page for passthrough requests (Ming)
- Queue release fix series (Ming)
- Device notification improvements (Martin)
- Propagate underlying device rotational status in loop (Holger)
- Removal of mtip32xx trim support, which has been disabled for years
(Christoph)
- Improvement and cleanup of nvme command handling (Christoph)
- Add block SPDX tags (Christoph)
- Cleanup/hardening of bio/bvec iteration (Christoph)
- A few NVMe pull requests (Christoph)
- Removal of CONFIG_LBDAF (Christoph)
- Various little fixes here and there"
* tag 'for-5.2/block-20190507' of git://git.kernel.dk/linux-block: (164 commits)
block: fix mismerge in bvec_advance
block: don't drain in-progress dispatch in blk_cleanup_queue()
blk-mq: move cancel of hctx->run_work into blk_mq_hw_sysfs_release
blk-mq: always free hctx after request queue is freed
blk-mq: split blk_mq_alloc_and_init_hctx into two parts
blk-mq: free hw queue's resource in hctx's release handler
blk-mq: move cancel of requeue_work into blk_mq_release
blk-mq: grab .q_usage_counter when queuing request from plug code path
block: fix function name in comment
nvmet: protect discovery change log event list iteration
nvme: mark nvme_core_init and nvme_core_exit static
nvme: move command size checks to the core
nvme-fabrics: check more command sizes
nvme-pci: check more command sizes
nvme-pci: remove an unneeded variable initialization
nvme-pci: unquiesce admin queue on shutdown
nvme-pci: shutdown on timeout during deletion
nvme-pci: fix psdt field for single segment sgls
nvme-multipath: don't print ANA group state by default
nvme-multipath: split bios with the ns_head bio_set before submitting
...
Currently support for 64-bit sector_t and blkcnt_t is optional on 32-bit
architectures. These types are required to support block device and/or
file sizes larger than 2 TiB, and have generally defaulted to on for
a long time. Enabling the option only increases the i386 tinyconfig
size by 145 bytes, and many data structures already always use
64-bit values for their in-core and on-disk data structures anyway,
so there should not be a large change in dynamic memory usage either.
Dropping this option removes a somewhat weird non-default config that
has cause various bugs or compiler warnings when actually used.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
The scripting must supply the CONFIG_INITRAMFS_SOURCE Kconfig option
so that kbuild can find the desired initrd, but the configcheck.sh
script gets confused by this option because it takes a string instead
of the expected y/n/m. This causes checkconfig.sh to complain about
CONFIG_INITRAMFS_SOURCE in the torture-test output (though not in the
summary). As more people use rcutorture, the resulting confusion is
an increasing concern.
This commit therefore suppresses this false-positive warning by filtering
CONFIG_INITRAMFS_SOURCE from within the checkconfig.sh script.
Reported-by: Joel Fernandes <joel@joelfernandes.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
Replace the license boiler plate with a SPDX license identifier.
While in the area, update an email address and add copyright notices.
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
As suggested by Ingo, this header file might benefit other tools than
just rcutorture. For now it's quite limited, but is easy to extend, so
exposing it into tools/include/nolibc/ will make it much easier to
adopt by other tools.
The mkinitrd.sh script in rcutorture was updated to use this new location.
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Signed-off-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
Ingo rightfully asked for a bit more documentation in the nolibc header,
so this patch adds some explanation about its purpose, how it's made, and
how to use it.
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
Reviewed-by: Joey Pabalinas <joeypabalinas@gmail.com>
Reviewed-by: Randy Dunlap <rdunlap@infradead.org>
A few macros had their rightmost backslash misaligned, and the pollfd
struct definition resisted the previous code reindent. Nothing else
changed.
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Signed-off-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
Reviewed-by: Joey Pabalinas <joeypabalinas@gmail.com>
A last-minute checkpatch cleanup caused most of list of clobbered
registers to be lost in the MIPS syscall definition. Although this code
is not yet used on MIPS, it is nevertheless better to fix it before it
does get used.
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Signed-off-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
Why a Bourne-shell "for" loop? And why 192 instances of "a"? This commit
adds a shell comment to present the answer to these mysteries. It also
uses a series of factor-of-four Bourne-shell assignments to make it
easy to see how many instances there are, replacing the earlier wall of
'a' characters.
Reported-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
[ paulmck: Fix wrong-variable bugs noted by Andrea Parri. ]
bug.2018.11.12a: Get rid of BUG_ON() and friends
consolidate.2018.12.01a: Continued RCU flavor-consolidation cleanup
doc.2018.11.12a: Documentation updates
fixes.2018.11.12a: Miscellaneous fixes
initrd.2018.11.08b: Automate creation of rcutorture initrd
sil.2018.11.12a: Remove more spin_unlock_wait() calls
Now that synchronize_rcu() waits for preempt-disable regions of code
as well as RCU read-side critical sections, synchronize_sched() can be
replaced by synchronize_rcu(). This commit therefore makes this change,
even though it is but a comment.
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
This reduces the size of the init executable from ~800 kB to ~800 bytes
on x86_64. This is only implemented for x86_64, i386, arm and arm64.
Others not tested.
Signed-off-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This is a definition of the most common syscalls needed in minimalist
init executables, allowing to statically build them with no external
dependencies. It is sufficient in its current form to build rcutorture's
init on x86_64, i386, arm, and arm64. Others have not been ported or
tested. Updates may be found here :
http://git.formilux.org/?p=people/willy/nolibc.git
Signed-off-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
If the build fails, we can end up with an empty initrd directory which
prevents the build script from operating again. Better rely on the
resulting init executable instead.
Signed-off-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Strip using -s on the compiler command line instead of calling the "strip"
utility as the latter isn't necessarily compatible with the target arch.
Signed-off-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This adds the CROSS_COMPILE environment to the initrd.sh script's
gcc command to enable cross compilation.
Reported-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, the initrd/init script and executable remain blocked almost
all the time. However, it is necessary to test nohz_full userspace
execution, which both variants of initrd/init fail to do. This commit
therefore causes initrd/init to spend about a millisecond per second
executing in userspace.
Reported-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The support for creating initrd directories using dracut is a great
improvement over having to always hand-create them, it is a bit annoying
to have to install some otherwise irrelevant package just to be able to
run rcutorture. This commit therefore adds support for creating initrd
directories on systems innocent of dracut. You do need gcc, but then
again you need that to build the kernel (or to build llvm) in any case.
The idea is to create an initrd directory containing nothing but a
statically linked binary having a for-loop over a long-term sleep().
The result is a Linux kernel with almost no userspace: even the
time-honored /dev, /lib, /tmp, and /usr directories are gone. In fact,
the only directory present is "/", but only because I don't know how to
get rid of it, at least short of not having an initrd in the first place.
Although statically linked binaries are much maligned, and rightly so,
their disadvantages seem to be irrelevant for this particular use case.
From https://www.akkadia.org/drepper/no_static_linking.html:
1. Fixes are difficult to apply to hordes of widely scattered
statically linked binaries. But in this case, there is only one
binary, but there would otherwise be no fewer than four libraries.
2. Security measures like local address randomization cannot be used.
Prudence prevents me from asserting that it is impossible to
base a remote attack on a networking-free rcutorture instance.
Nevertheless, bonus points to the first person who comes up with
such an attack!
3. More efficient use of physical memory. Not in this case, given
that libc is 1.8MB and the statically linked binary "only" 800K.
4. Features such as locales, name service switch (NSS),
internationalized domain names (IDN) tool, and so on require
dynamic linking. Bonus points to the first person coming up
with a valid rcutorture use case requiring these features in
its initrd.
5. Accidental violations of (L)GPL. Actually, this change actually
helps -avoid- such violations by reducing the temptation to
pass around tarballs of rcutorture-ready initrd directories.
After all, the rcutorture scripts automatically create an initrd
directory for you, so why bother with the tarballs?
6. Tools and hacks like ltrace, LD_PRELOAD, LD_PROFILE, and LD_AUDIT
don't work. Again, bonus points to the first person coming up
with a valid rcutorture use case requiring these features in
its initrd.
Nevertheless, the script will use dracut if available, and will create the
statically linked binary only when dracut are missing. Those preferring
the smaller initrd directory resulting from the statically linked binary
(like me) are free to hand-edit mkinitrd.sh to remove the code using
dracut. ;-)
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The rcutorture scripts currently expect the user to create the
tools/testing/selftests/rcutorture/initrd directory. Should the user
fail to do this, the kernel build will fail with obscure and confusing
error messages. This commit therefore adds explicit checks for the
tools/testing/selftests/rcutorture/initrd directory, and if not present,
creates one on systems on which dracut is installed. If this directory
could not be created, a less obscure error message is emitted and the
test is aborted.
Suggested-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Connor Shu <Connor.Shu@ibm.com>
[ paulmck: Adapt the script to fit into the rcutorture framework and
severely abbreviate the initrd/init script. ]
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Now that SRCU permits call_srcu() to be invoked at early boot, this
commit ensures that the rcutorture scripting tests early boot call_srcu().
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Now that the RCU-bh and RCU-sched update-side functions are simple
wrappers around their RCU counterparts, there isn't a whole lot of
point in testing them. This commit therefore removes the self-test
capability and removes the corresponding kernel-boot parameters.
It also updates the various rcutorture .boot files to remove the
kernel boot parameters that call for testing RCU-bh and RCU-sched.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Now that there is only one RCU flavor to rule them all, the TREE06
and TREE08 test scenarios are redundant. This commit therefore removes
them. Later changes will rebalance and renumber the tests.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The old approach placed all the build products into the b* directories,
which meant that some of these build products needed to be copied to
the proper directory in the res hierarchy. The new approach leaves
things like .config and the .o files in the b1 directory, but directs
build output and diagnostics directly to the proper directory in the
res hierarchy. Unfortunately, one of the copies was still carried out,
which could (and sometimes did) overwrite the build output and diagnostics
with obsolete output remaining in the b1 directory.
This commit therefore removes the offending "cp" command.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Some RCU bugs have been sensitive to the frequency of CPU-hotplug
operations, which have been gradually increased over time. But this
frequency is now at the one-second lower limit that can be specified using
the rcutorture.onoff_interval kernel parameter. This commit therefore
changes the units of rcutorture.onoff_interval from seconds to jiffies,
and also sets the value specified for this kernel parameter in the TREE03
rcutorture scenario to 200, which is 200 milliseconds for HZ=1000.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The main race with the early part of grace-period initialization appears
to be with CPU hotplug. To more fully open this race window, this commit
moves the rcu_gp_slow() from the beginning of the early initialization
loop to follow that loop, thus widening the race window, especially for
the rcu_node structures that are initialized last. This commit also
expands rcutree.gp_preinit_delay from 3 to 12, giving the same overall
delay in the grace period, but concentrated in the spot where it will
do the most good.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Although warnings about close calls are printed by kvm-recheck.sh,
kvm-find-errors.sh currently ignores them. This could easily result
in someone failing to investigate close calls, so this commit makes
them visible to kvm-find-errors.sh.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The TREE08-T.boot file was used to provide alternative options for
debugging, but things have changed, it has not kept up, and it has not
been used or missed. This commit therefore removes it.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The torture scripting currently builds each kernel from a set of parallel
runs in its own build directory. This can waste quite a bit of space when
running large numbers of concurrent scenarios, and pointlessly given that
the builds are run sequentially (albeit with a largish -j number passed to
"make"). This commit therefore places all build-command output in the
results directory, then does all builds in a single "b1" build directory.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, kvm-find-errors.sh looks only for build errors ("error:"),
so this commit makes it also locate build warnings ("warning:").
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Nicholas Piggin <npiggin@gmail.com>
With the addition of the end-of-test state, it is not uncommon for the
kvm.sh summary lines to overflow 80 characters. This commit therefore
applies abbreviations in order to make the line fit into 80 characters
with high probability.
And yes, I did make heavy use of punched cards back in the day, so 80
columns it is for my xterms! ;-)
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Nicholas Piggin <npiggin@gmail.com>
This commit adds the end-of-test test, if present in the console output,
to the kvm.sh test summary that is printed by kvm-recheck.sh. Note that
this only applies to rcutorture console output.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Nicholas Piggin <npiggin@gmail.com>
The rcutorture scripting scans the console output twice, once to look
for various sorts of hangs and again to find warnings and panics.
Unfortunately, only the output of the second scan gets written to the
console.log.diags file, which can cause hangs to be overlooked.
This commit therefore folds the parse-torture.sh script (which looks
for hangs) into the parse-console.sh script (which looks for warnings
and panics). This allows both types of failure information to be
added to console.log.diags, while still reliably removing this file
when it proves to be empty.
This also fixes a long-standing bug where rcuperf log files would
unconditionally complain about a hang.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Nicholas Piggin <npiggin@gmail.com>
This commit adds a script that allows viewing the build and/or
console output from failed rcutorture, locktorture, or rcuperf runs.
This replaces a time-honored but inefficient manual procedure that uses
cut and paste.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Nicholas Piggin <npiggin@gmail.com>
The default values for nreader and nwriter are apparently not all that
user-friendly, resulting in people doing scalability tests that ran all
runs at large scale. This commit therefore makes both the nreaders and
nwriters module default to the number of CPUs, and adds a comment to
rcuperf.c stating that the number of CPUs should be specified using the
nr_cpus kernel boot parameter. This commit also eliminates the redundant
rcuperf scripting specification of default values for these parameters.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The rcuperf trace-event processing counted every "done" trace event
as a piggyback, which is incorrect because the task that started the
grace period didn't piggyback at all. This commit fixes this problem
by recording the task that started a given grace period and ignoring
that task's "done" record for that grace period.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The rcuperf event-trace processing assumes that expedited grace periods
start and end on the same task, an assumption that was violated by moving
expedited grace-period processing to workqueues. This commit removes
this now-fallacious assumption from rcuperf's event-trace processing.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The purpose of jitter is to expose concurrency bugs due to invalid
assumptions about forward progress. There is usually little point
in jitter when measuring performance. This commit therefore defaults
jitter off when running rcuperf. You can override this by specifying
the kvm.sh "--jitter" argument -after- the "--torture rcuperf"
argument. No idea why you would want this, but if you do, that is
how you do it.
One example of a conccurrency bug that this jitter might expose is one
in which the developer assumed that a given short region of code would be
guaranteed to execute within some short time limit. Such assumptions are
invalid in virtualized environments because the hupervisor can preempt
the guest OS at any point, even when the guest OS thinks that it has
disabled interrupts.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The 512 megabyte memory size has served quite well, but more memory
is required when using large trace buffers on large systems. This
commit therefore adds a --memory argument to the kvm.sh script, which
allows the memory size to be specified on the command line, for example,
"--memory 768", --memory 800M", or "--memory 2G".
Reported-by: Sebastian Siewior <bigeasy@linutronix.de>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds support of the qemu command qemu-system-aarch64
to rcutorture.
Signed-off-by: Lihao Liang <lianglihao@huawei.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The kvm.sh header comment is a bit of a relic, so this commit brings
it up to date.
Reported-by: Lihao Liang <lianglihao@huawei.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Commit 6f1982fedd ("sched/isolation: Handle the nohz_full= parameter")
broke CONFIG_NO_HZ_FULL_ALL=y kernels. This breakage is due to the code
under CONFIG_NO_HZ_FULL_ALL failing to invoke the shiny new housekeeping
functions. This means that rcutorture scenario TREE04 now emits RCU CPU
stall warnings due to the RCU grace-period kthreads not being awakened
at a time of their choosing, or perhaps even not at all:
[ 27.731422] rcu_bh kthread starved for 21001 jiffies! g18446744073709551369 c18446744073709551368 f0x0 RCU_GP_WAIT_FQS(3) ->state=0x402 ->cpu=3
[ 27.731423] rcu_bh I14936 9 2 0x80080000
[ 27.731435] Call Trace:
[ 27.731440] __schedule+0x31a/0x6d0
[ 27.731442] schedule+0x31/0x80
[ 27.731446] schedule_timeout+0x15a/0x320
[ 27.731453] ? call_timer_fn+0x130/0x130
[ 27.731457] rcu_gp_kthread+0x66c/0xea0
[ 27.731458] ? rcu_gp_kthread+0x66c/0xea0
Because no one has complained about CONFIG_NO_HZ_FULL_ALL=y being broken,
I hypothesize that no one is in fact using it, other than rcutorture.
This commit therefore eliminates CONFIG_NO_HZ_FULL_ALL and updates
rcutorture's config files to instead use the nohz_full= kernel parameter
to put the desired CPUs into nohz_full mode.
Fixes: 6f1982fedd ("sched/isolation: Handle the nohz_full= parameter")
Reported-by: kernel test robot <xiaolong.ye@intel.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Frederic Weisbecker <frederic@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Wanpeng Li <kernellwp@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: John Stultz <john.stultz@linaro.org>
Cc: Jonathan Corbet <corbet@lwn.net>
The purpose of torture_runnable is to allow rcutorture and locktorture
to be started and stopped via sysfs when they are built into the kernel
(as in not compiled as loadable modules). However, the 0444 permissions
for both instances of torture_runnable prevent this use case from ever
being put into practice. Given that there have been no complaints
about this deficiency, it is reasonable to conclude that no one actually
makes use of this sysfs capability. The perf_runnable module parameter
for rcuperf is in the same situation.
This commit therefore removes both torture_runnable instances as well
as perf_runnable.
Reported-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Check for build-directory existence and write permissions are provided in
both 'kvm-test-1-run.sh' an 'kvm-build.sh'. Because the 'kvm-build.sh'
is dependent on 'kvm-test-1-run.sh' ('kvm-build.sh' uses variables that
defined from its caller.), these checks are unnecessarily duplicated.
This commit therefore removes the check in from the 'kvm-build.sh' script.
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Inclusions of 'functions.sh' from 'kvm-test-1-run.sh' and
'kvm-recheck*.sh' use its absolute path. Because the directory containing
'functions.sh' is already in PATH, the full path is unnecessary. This
commit therefore simplifies the inclusions to use the short relative path.
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Both the 'kvm.sh' and 'kvm-test-1-run.sh' scripts log messages by printing
the message to 'stdout' and then also printing it into the log file.
Generation of the message thus occurs twice, once for 'stdout' and once
for the log file. Moreover, many of the messages contain 'date' output,
which results in date being invoked twice (once for stdout print, once
for log file write). As a result, the date information in stdout and
log file can differ, which could cause confusion.
This commit therefore simplifies the logging procedure by using 'tee'.
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The kvm-recheck-(lock|rcu|rcuperf).sh scripts check whether the
user-specified results directory exists. If not, it prints out error
message that says the specified directory is unreadable. To make the
message more precise, this commit adds a readability check.
Fixes: 2193e1604e ("rcutorture: Abstract kvm-recheck.sh")
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The 'kvm.sh' rcutorture script requires that it be invoked from the top
of Linux-kernel source tree. It is just a subtle restriction, but users
using it for the first time could forget the restriction and be confused.
Moreover, it makes commands a little longer, which can be frustrating.
This commit therefore lets users invoke the script from any location.
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The '--qemu-args' option's help text is wrongly copied from '--qemu-cmd'
option and its argument type description message format is inconsistent
with other arguments. This commit fixes the usage and type messages to
be consistent with others.
Fixes: e9ce640001 ("rcutorture: Add --qemu-args argument to kvm.sh")
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The variable `alldone` is defined but not used within an awk script.
This commit therefore removes it.
Fixes:53954671033d ("rcutorture: Do better bin packing")
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The 'config2frag.sh' script is not used, so this commit removes it.
Fixes: c87b9c601a ("rcutorture: Add KVM-based test framework")
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The 'configinit.sh' script checks the format of optional argument for the
build directory, printing an error message if the format is not valid.
However, the error message uses the wrong variable, indicating an empty
string even though the user entered a non-empty (but erroneous) string.
This commit fixes the script to use the correct variable.
Fixes: c87b9c601a ("rcutorture: Add KVM-based test framework")
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Because %p prints "(null)" and %pK prints "0000000000000000" or (on
32-bit systems) "00000000", this commit adjusts torture-test scripting
accordingly.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Pull core locking updates from Ingo Molnar:
"The main changes in this cycle are:
- Another attempt at enabling cross-release lockdep dependency
tracking (automatically part of CONFIG_PROVE_LOCKING=y), this time
with better performance and fewer false positives. (Byungchul Park)
- Introduce lockdep_assert_irqs_enabled()/disabled() and convert
open-coded equivalents to lockdep variants. (Frederic Weisbecker)
- Add down_read_killable() and use it in the VFS's iterate_dir()
method. (Kirill Tkhai)
- Convert remaining uses of ACCESS_ONCE() to
READ_ONCE()/WRITE_ONCE(). Most of the conversion was Coccinelle
driven. (Mark Rutland, Paul E. McKenney)
- Get rid of lockless_dereference(), by strengthening Alpha atomics,
strengthening READ_ONCE() with smp_read_barrier_depends() and thus
being able to convert users of lockless_dereference() to
READ_ONCE(). (Will Deacon)
- Various micro-optimizations:
- better PV qspinlocks (Waiman Long),
- better x86 barriers (Michael S. Tsirkin)
- better x86 refcounts (Kees Cook)
- ... plus other fixes and enhancements. (Borislav Petkov, Juergen
Gross, Miguel Bernal Marin)"
* 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (70 commits)
locking/x86: Use LOCK ADD for smp_mb() instead of MFENCE
rcu: Use lockdep to assert IRQs are disabled/enabled
netpoll: Use lockdep to assert IRQs are disabled/enabled
timers/posix-cpu-timers: Use lockdep to assert IRQs are disabled/enabled
sched/clock, sched/cputime: Use lockdep to assert IRQs are disabled/enabled
irq_work: Use lockdep to assert IRQs are disabled/enabled
irq/timings: Use lockdep to assert IRQs are disabled/enabled
perf/core: Use lockdep to assert IRQs are disabled/enabled
x86: Use lockdep to assert IRQs are disabled/enabled
smp/core: Use lockdep to assert IRQs are disabled/enabled
timers/hrtimer: Use lockdep to assert IRQs are disabled/enabled
timers/nohz: Use lockdep to assert IRQs are disabled/enabled
workqueue: Use lockdep to assert IRQs are disabled/enabled
irq/softirqs: Use lockdep to assert IRQs are disabled/enabled
locking/lockdep: Add IRQs disabled/enabled assertion APIs: lockdep_assert_irqs_enabled()/disabled()
locking/pvqspinlock: Implement hybrid PV queued/unfair locks
locking/rwlocks: Fix comments
x86/paravirt: Set up the virt_spin_lock_key after static keys get initialized
block, locking/lockdep: Assign a lock_class per gendisk used for wait_for_completion()
workqueue: Remove now redundant lock acquisitions wrt. workqueue flushes
...
Pull RCU updates from Ingo Molnar:
"The main changes in this cycle are:
- Documentation updates
- RCU CPU stall-warning updates
- Torture-test updates
- Miscellaneous fixes
Size wise the biggest updates are to documentation. Excluding
documentation most of the code increase comes from a single commit
which expands debugging"
* 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (24 commits)
srcu: Add parameters to SRCU docbook comments
doc: Rewrite confusing statement about memory barriers
memory-barriers.txt: Fix typo in pairing example
rcu/segcblist: Include rcupdate.h
rcu: Add extended-quiescent-state testing advice
rcu: Suppress lockdep false-positive ->boost_mtx complaints
rcu: Do not include rtmutex_common.h unconditionally
torture: Provide TMPDIR environment variable to specify tmpdir
rcutorture: Dump writer stack if stalled
rcutorture: Add interrupt-disable capability to stall-warning tests
rcu: Suppress RCU CPU stall warnings while dumping trace
rcu: Turn off tracing before dumping trace
rcu: Make RCU CPU stall warnings check for irq-disabled CPUs
sched,rcu: Make cond_resched() provide RCU quiescent state
sched: Make resched_cpu() unconditional
irq_work: Map irq_work_on_queue() to irq_work_on() in !SMP
rcu: Create call_rcu_tasks() kthread at boot time
rcu: Fix up pending cbs check in rcu_prepare_for_idle
memory-barriers: Rework multicopy-atomicity section
memory-barriers: Replace uses of "transitive"
...
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.
This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that became
the concluded license(s).
- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
preference to ACCESS_ONCE(), and new code is expected to use one of the
former. So far, there's been no reason to change most existing uses of
ACCESS_ONCE(), as these aren't currently harmful.
However, for some features it is necessary to instrument reads and
writes separately, which is not possible with ACCESS_ONCE(). This
distinction is critical to correct operation.
The bulk of the kernel code can be transformed via Coccinelle to use
{READ,WRITE}_ONCE(), though this only modifies users of ACCESS_ONCE(),
and not the implementation itself. As such, it has the potential to
break homebrew ACCESS_ONCE() macros seen in some user code in the kernel
tree (e.g. the virtio code, as fixed in commit ea9156fb3b).
To avoid fragility if/when that transformation occurs, this patch
reworks the definitions of {READ,WRITE}_ONCE() in the rcutorture formal
tests, and removes the unused ACCESS_ONCE() helper. There should be no
functional change as a result of this patch.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: davem@davemloft.net
Cc: linux-arch@vger.kernel.org
Cc: mpe@ellerman.id.au
Cc: shuah@kernel.org
Cc: snitzer@redhat.com
Cc: thor.thayer@linux.intel.com
Cc: tj@kernel.org
Cc: viro@zeniv.linux.org.uk
Cc: will.deacon@arm.com
Link: http://lkml.kernel.org/r/1508792849-3115-13-git-send-email-paulmck@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Both rcutorture and locktorture currently place temporary files in /tmp,
in keeping with decades-long tradition. However, sometimes it is useful
to specify an alternative temporary directory, for example, for space
or performance reasons. This commit therefore causes the torture-test
scripting to use the path specified in the TMPDIR environment variable,
or to fall back to traditional /tmp if this variable is not set.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The actual use of TASKS_RCU is only when PREEMPT, otherwise RCU-sched
is used instead. This commit therefore makes synchronize_rcu_tasks()
and call_rcu_tasks() available always, but mapped to synchronize_sched()
and call_rcu_sched(), respectively, when !PREEMPT. This approach also
allows some #ifdefs to be removed from rcutorture.
Reported-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org>
Acked-by: Ingo Molnar <mingo@kernel.org>
The maxcpus= kernel boot parameter limits the number of CPUs brought
online at boot time, but it does nothing to prevent additional CPUs
from being brought up later. Placing a hard cap on the total number
of CPUs is instead the job of the nr_cpus= boot parameter. This commit
therefore switches the configfrag_boot_cpus() shell function from maxcpus=
to nr_cpus=. This commit also adds a nr_cpus=43 kernel parameter to RCU's
TREE01 test scenario, but retains the maxcpus=8 kernel parameter in order
to test the ability of RCU expedited grace periods to handle new CPUs
coming online for the first time during grace-period initialization.
Finally, this commit makes the torture scheduling allow maxcpus= to
override other means of specifying the number of CPUs to allow for.
This last works because the torture kernel modules size their workloads
based on the number of CPUs present at the start of the test, not the
ultimate number of CPUs.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Strings used in event tracing need to be specially handled, for example,
being copied to the trace buffer instead of being pointed to by the trace
buffer. Although the TPS() macro can be used to "launder" pointed-to
strings, this might not be all that effective within a loadable module.
This commit therefore copies rcutorture's strings to the trace buffer.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Currently, rcutorture groups runs in batches, building each scenario in
a given batch, then invoking qemu to run all the kernels in the batch.
Of course, if a given scenario's kernel fails to build, there is no qemu
run for that scenario. And if all of the kernels in a given batch fail
to build, there are no runs, and rcutorture immediately starts on the
next batch.
But not if --jitter has been specified, which it is by default. In this
case, the jitter scripts are started unconditionally, and rcutorture
waits for them to complete, even though there are no kernels to run.
This commit therefore checks for this situation, and refuses to start
jitter unless at least one of the kernels in the batch built successfully.
This saves substantial time when all scenarios' kernels fail to build,
particularly if a long --duration was specified.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, testing a variant of an existing scenario requires editing
that scenario's file or creating a new scenario file. This is messy
and error prone with respect to changes to scenarios.
This commit therefore adds a --kconfig argument to kvm.sh, so that
'--kconfig "CONFIG_RCU_TRACE=y CONFIG_RCU_EQS_DEBUG=n" will override those
two Kconfig options. In addition, there is now clear precedence:
the config fragment overrides CFcommon, and the --kconfig argument
overrides both.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit selects CONFIG_PROVE_LOCKING for the SRCU-u scenario
to get better test coverage.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Classic SRCU is no more, so this commit removes the corresponding
rcutorture boot-parameters file.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tiny RCU's job is to be tiny, so this commit removes its RCU CPU
stall warning code. After this, there is no longer any need for
rcu_sched_ctrlblk and rcu_bh_ctrlblk to be in tiny_plugin.h, so this
commit also moves them to tiny.c.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The CONFIG_RCU_NOCB_CPU_ALL, CONFIG_RCU_NOCB_CPU_NONE, and
CONFIG_RCU_NOCB_CPU_ZERO Kconfig options are used only in testing and
are redundant with the rcu_nocbs= boot parameter. This commit therefore
removes these three Kconfig options and adjusts the rcutorture scripts
to use the boot parameter instead.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
RCU's debugfs tracing used to be the only reasonable low-level debug
information available, but ftrace and event tracing has since surpassed
the RCU debugfs level of usefulness. This commit therefore removes
RCU's debugfs tracing.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Classic SRCU was only ever intended to be a fallback in case of issues
with Tree/Tiny SRCU, and the latter two are doing quite well in testing.
This commit therefore removes Classic SRCU.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The sparse-based checking for non-RCU accesses to RCU-protected pointers
has been around for a very long time, and it is now the only type of
sparse-based checking that is optional. This commit therefore makes
it unconditional.
Reported-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Fengguang Wu <fengguang.wu@intel.com>
The PROVE_RCU_REPEATEDLY Kconfig option was initially added due to
the volume of messages from PROVE_RCU: Doing just one per boot would
have required excessive numbers of boots to locate them all. However,
PROVE_RCU messages are now relatively rare, so there is no longer any
reason to need more than one such message per boot. This commit therefore
removes the PROVE_RCU_REPEATEDLY Kconfig option.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Ingo Molnar <mingo@kernel.org>
The NO_HZ_FULL_SYSIDLE full-system-idle capability was added in 2013
by commit 0edd1b1784 ("nohz_full: Add full-system-idle state machine"),
but has not been used. This commit therefore removes it.
If it turns out to be needed later, this commit can always be reverted.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Ingo Molnar <mingo@kernel.org>
Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
Anything that can be done with the RCU_KTHREAD_PRIO Kconfig option can
also be done with the rcutree.kthread_prio kernel boot parameter.
This commit therefore removes this Kconfig option.
Reported-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Rik van Riel <riel@redhat.com>
The RCU_TORTURE_TEST_SLOW_PREINIT, RCU_TORTURE_TEST_SLOW_PREINIT_DELAY,
RCU_TORTURE_TEST_SLOW_PREINIT_DELAY, RCU_TORTURE_TEST_SLOW_INIT,
RCU_TORTURE_TEST_SLOW_INIT_DELAY, RCU_TORTURE_TEST_SLOW_CLEANUP,
and RCU_TORTURE_TEST_SLOW_CLEANUP_DELAY Kconfig options are only
useful for torture testing, and there are the rcutree.gp_cleanup_delay,
rcutree.gp_init_delay, and rcutree.gp_preinit_delay kernel boot parameters
that rcutorture can use instead. The effect of these parameters is to
artificially slow down grace period initialization and cleanup in order
to make some types of race conditions happen more often.
This commit therefore simplifies Tree RCU a bit by removing the Kconfig
options and adding the corresponding kernel parameters to rcutorture's
.boot files instead. However, this commit also leaves out the kernel
parameters for TREE02, TREE04, and TREE07 in order to have about the
same number of tests slowed as not slowed. TREE01, TREE03, TREE05,
and TREE06 are slowed, and the rest are not slowed.
Reported-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, when running from a git archive, the testid.txt file contains
only the branch name, the output of "git status", and the SHA-1 of
the current HEAD. This is useful, but does not uniquely identify the
source code that was built. This commit therefore adds the output of
"git diff HEAD", which means that if two testid.txt files compare equal,
they correspond to exactly the same source code. Give or take the
possibility of SHA-1 collisions, that is. ;-)
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Most OS distribution have awk in /usr/bin not in /bin
Without this patch, kernel-devsrc fails to build as
runtime dependency for srcu-cbmc script /bin/awk is
not found.
Signed-off-by: Kushwaha, Priyalee <priyalee.kushwaha@intel.com>
Acked-by: Lance Roy <ldr709@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Given that the plan is to retire Classic SRCU in the near future, this
commit reduces the number of CPUs dedicated to testing Classic SRCU.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds a TINY rcuperf test scenario, which allows performance
testing of Tiny RCU and Tiny SRCU.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds a Kconfig-fragment file for Classic SRCU to ease
performance comparisons with Tree SRCU.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The TREE and TREE54 rcuperf scenarios' Kconfig fragment files specified
conflicting values for CONFIG_RCU_TRACE. This commit therefore removes
the =n line in favor of the =y line.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
A number of the rcutorture test scenarios were not using the desired
Kconfig options because dependencies were preventing the selections in the
Kconfig-fragment files from being honored. This commit therefore updates
the Kconfig-fragment files to account for these changes in dependencies.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The rcutorture scripting handles the CONFIG_*_TORTURE_TEST Kconfig
options specially, and therefore greps them out of the Kconfig-fragment
files. Unfortunately, a poor choice of grep pattern means that the
CONFIG_RCU_TORTURE_TEST_SLOW_CLEANUP, CONFIG_RCU_TORTURE_TEST_SLOW_INIT,
and CONFIG_RCU_TORTURE_TEST_SLOW_PREINIT Kconfig options are also grepped
out, preventing rcutorture from using them. This commit therefore fixes
the offending grep pattern to focus only on the CONFIG_*_TORTURE_TEST
Kconfig options.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
A robust combination of paranoia and cowardice has resulted in retaining
Classic SRCU (CONFIG_CLASSIC_SRCU) as a backup for the shiny new Tiny
and Tree SRCU implementations. If it is to be a viable backup, it of
course needs to be tested. This commit therefore adds an rcutorture
scenario named SRCU-C for Classic SRCU. This commit also adds this
scenario to the set that are run by default.
Once sufficient good experience has accumulated for Tiny and Tree SRCU,
this test will be removed, along with the Classic SRCU implementation
itself.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds an SRCU-t rcutorture scenario for the new Tiny SRCU
implementation, removing the need to pass the --bootargs parameter to
kvm.sh to run Tiny SRCU tests. This commit also adds SRCU-t to the set
of scenarios that are run by default.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Kconfig "select" clauses can defeat Kconfig-fragment file attempts to
clear a given Kconfig variable, and dependencies can defeat attempts to
set a given Kconfig variable. Because "select" clauses and dependencies
can be added at any time, there needs to be a way to verify that the
Kconfig-fragment file's requests were honored. And there is, except
that it is buggy. This commit therefore provides the needed fix.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds a test for a three-level srcu_node tree for Tree SRCU
in the existing SRCU-P scenario. This requires enabling CONFIG_RCU_EXPERT,
so the CONFIG_RCU_EXPERT=n scenario is now SRCU-N. The reason for using
SRCU-P for the tall tree is that preemption raises the possibility of
locating more bugs than does the non-preemptive SRCU-N.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Back when SRCU was simpler, there wasn't much need for lockdep.
However, with Tree SRCU, it is needed. This commit therefore adds
CONFIG_PROVE_LOCKING to the SRCU-P scenario.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, the rcutorture scripting will give an error message if
running a duplicate scenario that happens also to have a non-existent
build directory (b1, b2, ... in the rcutorture directory). Worse yet, if
the build directory has already been created and used for a real build,
the script will silently grab the wrong Kconfig fragment, which could
cause confusion to the poor sap (me) analyzing old test results. At
least the actual test runs correctly...
This commit therefore accesses the Kconfig fragment from the results
directory corresponding to the first of the duplicate scenarios, for
which a build was actually carried out. This prevents both the messages
and at least one form of later confusion.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Pull locking updates from Ingo Molnar:
"The main changes in this cycle were:
- Implement wraparound-safe refcount_t and kref_t types based on
generic atomic primitives (Peter Zijlstra)
- Improve and fix the ww_mutex code (Nicolai Hähnle)
- Add self-tests to the ww_mutex code (Chris Wilson)
- Optimize percpu-rwsems with the 'rcuwait' mechanism (Davidlohr
Bueso)
- Micro-optimize the current-task logic all around the core kernel
(Davidlohr Bueso)
- Tidy up after recent optimizations: remove stale code and APIs,
clean up the code (Waiman Long)
- ... plus misc fixes, updates and cleanups"
* 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (50 commits)
fork: Fix task_struct alignment
locking/spinlock/debug: Remove spinlock lockup detection code
lockdep: Fix incorrect condition to print bug msgs for MAX_LOCKDEP_CHAIN_HLOCKS
lkdtm: Convert to refcount_t testing
kref: Implement 'struct kref' using refcount_t
refcount_t: Introduce a special purpose refcount type
sched/wake_q: Clarify queue reinit comment
sched/wait, rcuwait: Fix typo in comment
locking/mutex: Fix lockdep_assert_held() fail
locking/rtmutex: Flip unlikely() branch to likely() in __rt_mutex_slowlock()
locking/rwsem: Reinit wake_q after use
locking/rwsem: Remove unnecessary atomic_long_t casts
jump_labels: Move header guard #endif down where it belongs
locking/atomic, kref: Implement kref_put_lock()
locking/ww_mutex: Turn off __must_check for now
locking/atomic, kref: Avoid more abuse
locking/atomic, kref: Use kref_get_unless_zero() more
locking/atomic, kref: Kill kref_sub()
locking/atomic, kref: Add kref_read()
locking/atomic, kref: Add KREF_INIT()
...
This commit creates a formal/srcu-cbmc directory containing scripts that
pull SRCU in from the source code, filter it to remove things that CBMC
cannot handle, and run a series of verifications on it. This has a number
of shortcomings:
1. It does not yet hook into the upper-level self-test Makefiles.
2. It tests only a single scenario, store buffering.
3. There is no gcc-based syntax-error prefiltering.
Nevertheless, it does fully verify a piece of SRCU under a moderately
weak memory model (PSO).
Signed-off-by: Lance Roy <ldr709@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The RCU torture tests currently do not run any Tiny RCU scenarios for
CONFIG_DEBUG_OBJECTS_RCU_HEAD=y. This is a hole in the test, given
that someone might need this in real life and given that Tiny RCU uses
different callback-handling code than does Tree RCU. This commit
therefore enables this Kconfig option for scenario TINY02.
Reported-by: "Ahmed, Iftekhar" <ahmedi@oregonstate.edu>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit runs TREE04 and TREE08 with CONFIG_RCU_EQS_DEBUG=y,
enabling dyntick-counter checking on those two tests.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit sets CONFIG_DEBUG_LOCK_ALLOC but not CONFIG_PROVE_LOCKING
for TREE08 in order to have at least one test with this configuration.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit enables the CONFIG_DEBUG_OBJECTS_RCU_HEAD Kconfig option
in TREE02 in order to do at least some testing with this enabled.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit moves CONFIG_RCU_TORTURE_TEST_SLOW_CLEANUP,
CONFIG_RCU_TORTURE_TEST_SLOW_INIT, and CONFIG_RCU_TORTURE_TEST_SLOW_PREINIT
from CFcommon to all of the TREE scenarios other than TREE08 and TREE09
in order to do at least some testing without these Kconfig options set.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit adds CONFIG_PROVE_RCU_REPEATEDLY=y, which has been untested
for quite some time.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit verifies coverage of testing with CONFIG_RCU_STALL_COMMON=n.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Although ww_mutexes degenerate into mutexes, it would be useful to
torture the deadlock handling between multiple ww_mutexes in addition to
torturing the regular mutexes.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Maarten Lankhorst <dev@mblankhorst.nl>
Cc: Nicolai Hähnle <nhaehnle@gmail.com>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/20161201114711.28697-3-chris@chris-wilson.co.uk
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Currently, if the --jitter flag specifies jitter for a --build-only
run, the system will obediently build a kernel, refuse to launch it,
launch the requested number of jitter processes, and wait for the
specified kernel run time, which defaults to 30 minutes. This is
of course quite pointless.
This commit therefore disables jitter on build-only runs.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit enables jitter by default. It may be manually disabled
by passing "--jitter 0" to kvm.sh.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Because recent testing shows that "-soundhw pcspkr" is no longer required
in the kernel boot arguments, this commit drops this qemu argument.
Reported-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Do not restrict the cpu type to POWER7 for QEMU as we have POWER8 now.
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The option "-soundhw pcspk" gives me a error on PPC as follow:
qemu-system-ppc64: ISA bus not available for pcspk
This means this option doesn't work on ppc by default. So simply make
this an x86-specific option via identify_qemu_args().
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The vmlinux image is available for all the architectures, and suitable
for running a KVM guest by QEMU, besides, we used to copy the vmlinux
to $resdir anyway. Therefore it makes sense to use it as the fallback
kernel image for rcutorture KVM tests.
This patch makes identify_boot_image() return vmlinux if
${TORTURE_BOOT_IMAGE} is not set on non-x86 architectures, also fixes
several places that hard-code "bzImage" as $KERNEL.
This also fixes a problem that PPC doesn't have a bzImage file as build
results.
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Using dracut is another way to get an initramfs for KVM-based RCU
torture tests, which is more flexible than using the host's initramfs
image, because modules and binaries may be added or removed via dracut
command options. So add an example in the document, in case that there
are some situations where host's initramfs couldn't be used.
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds a string of the form "Starves: 10" to the summary
line for error conditions found in the console output.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit avoids killing qemu if a trace dump is making progress
or if console log output is continuing and the console log timestamp
does not exceed the total plus grace period.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, the rcutorture scripts do not test boot-time adjustment of
leaf fanout (via the rcutree.rcu_fanout_leaf boot parameter), as was noted
during testing carried out by Iftekhar Ahmed of Oregon State University.
This commit therefore adjusts TREE04's CONFIG_RCU_FANOUT_LEAF from 4 to 3,
and also adds rcutree.rcu_fanout_leaf=4 to its boot parameters. This
change forces RCU's boot-time geometry-change code to be exercised.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, if the user specifies multiple runs of a given test
configuration, the scripting does multiple kernel builds. This wastes
both time and disk space, so this commit makes the scripting use the
first build for all runs of a given test configuration.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds a --jitter OS-jitter capability to expose bugs based
on no-delay assumptions.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit converts test duration from minutes to seconds early on
in order to prepare for upcoming OS-jitter-injection changes.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The current hang-check machinery in the rcutorture scripts uses "$!" of
a parenthesized bash statement to capture the pid. Unfortunately, this
captures not qemu's pid, but rather that of its parent that implements
the parenthesized statement. This commit therefore adjusts things so as
to capture qemu's actual pid, which then allows the script to actually
kill qemu in event of a kernel hang.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds an rcuperf scenario named TREE54 that uses 54 CPUs and
provides a four-level rcu_node combining tree.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The rcuperf event-trace data is more accurate than are the rcuperf
printk()s because locking keeps things ordered. This commit therefore
parses and analyzes this event-trace data if present, and falls back on
the printk()s otherwise.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit enables ftrace in the rcuperf TREE kernel build and adds
an ftrace_dump() at the end of rcuperf processing. This data will be
used to measure the actual durations of the expedited grace periods
without the added delays inherent in the kernel-module measurements.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds a line giving the number of grace periods, the number
of batches, and the ratio. The larger the ratio, the greater the
batching efficiency.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds the scripting changes to add support for the shiny
new rcuperf kernel module.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The current scripts parse console output only for cases where one CPU
detect a stall on some other CPU or task. This commit therefore adds
checks for self-detected stalls.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, an error-free run produces an empty console.log.diags file.
This can be annoying when using "vi */console.log.diags" to see a full
summary of the errors. This commit therefore removes any empty files
during the analysis process.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds checks for rcutorture writer starvation, so that
instances will be added to the test summary.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, the console output files ("console.log") are placed in the
build directory initially, then copied to the results directory.
One problem with this is if a qemu refuses to die in a timely fashion
after a kernel hang, it will continue to write after the next qemu
starts up, resulting in confusing output from the old instance of
qemu. This commit prevents such confusion by placing the console.log
files into the results directory to begin with, so that a given instance
of qemu is always writing only to its own console.log file.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Currently, the scripts print a list of warning/bug indicators from the
console.log file. This works well if there are only a few warnings or
bugs, but can be quite annoying if there is a large number. This commit
therefore prints a summary listing the number of each type of warning/bug
indicator, but only if there is at least one such indicator. The full
list is stored in the results directory at console.log.diags, which
makes it easier to find the warning/bugs in the full console.log.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Commit d1ec4c34c7 ("rcu: Drop RCU_USER_QS in favor of NO_HZ_FULL") has
removed RCU_USER_QS from Kconfig file, so remove it from some documents
to avoid any confusion.
Signed-off-by: Yang Shi <yang.shi@linaro.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: Frederic Weisbecker <fweisbec@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
The default test grace period of two minutes is insufficient in some
cases and excessive in others. This commit therefore increases the
default to three minutes, but also adds a --shutdown-grace parameter
to allow the default to be overridden.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Currently, the scripts print "----Start batch" at the beginning of each
batch, which does serve as a good visual delimiter between batches.
Unfortunately, if there are a lot of batches, it is hard to quickly
estimate test runtime from the output of "--dryrun sched". This commit
therefore adds a batch number, so that the beginning-of-batch output
looks like this "----Start batch 10" for the tenth batch.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit allows --bootarg instead of --bootargs, --config instead of
--configs, and --qemu-arg instead of --qemu-args. For those cases where
a native English speaker might auto-correct the argument to be incorrect.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit adds percpu_rwsem tests based on the earlier rwsem tests.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Real time mutexes is one of the few general primitives
that we do not have in locktorture. Address this -- a few
considerations:
o To spice things up, enable competing thread(s) to become
rt, such that we can stress different prio boosting paths
in the rtmutex code. Introduce a ->task_boost callback,
only used by rtmutex-torturer. Tasks will boost/deboost
around every 50k (arbitrarily) lock/unlock operations.
o Hold times are similar to what we have for other locks:
only occasionally having longer hold times (per ~200k ops).
So we roughly do two full rt boost+deboosting ops with
short hold times.
Signed-off-by: Davidlohr Bueso <dbueso@suse.de>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
The CONFIG_RCU_CPU_STALL_INFO has been default-y for a couple of
releases with no complaints, so it is time to eliminate this Kconfig
option entirely, so that the long-form RCU CPU stall warnings cannot
be disabled. This commit does just that.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently none of the RCU-tasks scenarios enables lockdep-RCU, which
causes bugs to be missed. This commit therefore enables lockdep-RCU
on TASKS01.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Although it is currently possible to run the same test in parallel,
'--config "TINY01 TINY01 TINY01"' can get a bit verbose, especially
if you want to run 48 instances of TINY01 in parallel. This commit
therefore allows prefixing the Kconfig fragment with a repeat count,
for example, '--config "48*TINY01"' to run 48 instances in parallel.
At least assuming that you have 48 CPUs and also gave '--cpus 48'.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
The current rcutorture scripting fails to dump out errors from
"make oldconfig", so this commit addresses this issue.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit updates TREE_RCU-kconfig.txt to reflect changes in RCU's
Kconfig setup. This commit also updates rcutorture's Kconfig fragments
to account for Kconfig parameters that are now driven directly off of
other Kconfig parameters.
The #CHECK# prefix tells the rcutorture scripts to take no action to try
to set the Kconfig parameter, but to check that it does in fact get set.
This is useful for verifying that Kconfig parameters that are supposed
to be automatically set do in fact get set to the required values.
Reported-by: Pranith Kumar <bobby.prani@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit causes the rcutorture scripts to force RCU_EXPERT so that
these scripts can cause rcutorture to torture RCU in the various required
configurations. However, SRCU-P, TASKS03, and TREE09 retain !RCU_EXPERT
in order to ensure testing of the vanilla configuration.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Pranith Kumar <bobby.prani@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit updates rcutortures configuration-fragment files to account
for the move from the CONFIG_RCU_FANOUT_EXACT Kconfig parameter to the
new rcutree.rcu_fanout_exact= boot parameter.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Pranith Kumar <bobby.prani@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
The TASKS01, TASKS02, and TASKS03 rcutorture config fragments currently
set CONFIG_TASKS_RCU. However, now that the value of this Kconfig
parameter is set via "select" statements, it is no longer necessary to
set it explicitly. This commit therefore removes it from the Kconfig
fragments.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Pranith Kumar <bobby.prani@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
The current rcutorture testing does not do any cleanup operations.
This works because the srcu_struct is statically allocated, but it
does represent a memory leak of the associated dynamically allocated
->per_cpu_ref per-CPU variables. However, rcutorture currently uses
a statically allocated srcu_struct, which cannot legally be passed to
cleanup_srcu_struct(). Therefore, this commit adds a second form
of srcu (called srcud) that dynamically allocates and frees the
associated per-CPU variables. This commit also adds a ->cleanup()
member to rcu_torture_ops that is invoked at the end of the test,
after ->cb_barriers(). This ->cleanup() pointer is NULL for all
existing tests, and thus only used for scrud. Finally, the SRCU-P
torture-test configuration selects scrud instead of srcu, with SRCU-N
continuing to use srcu, thereby testing both static and dynamic
srcu_struct structures.
Reported-by: "Ahmed, Iftekhar" <ahmedi@onid.oregonstate.edu>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
TREE03 has been especially effective at finding bugs lately. This commit
makes it even more effective by speeding up its CPU hotplug testing and
increasing its NR_CPUs from 8 to 16. TREE08's NR_CPUS is decreased from
16 to 8 in order to maintain the same test duration.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Given that the combination of PREEMPT_RCU and HOTPLUG_CPU is producing the
most bugs lately, this commit swaps the TREE03 and TREE04 rcu_node-tree
geometries so that the test exercising PREEMPT_RCU and HOTPLUG_CPU has
three-level rather than two-level rcu_node trees.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Tiny RCU supports both RCU-sched and RCU-bh, but only RCU-sched is
currently tested by the rcutorture scripts. This commit therefore
changes the TINY02 configuration to test RCU-bh, with TINY01 continuing
to test RCU-sched.
This shortcoming of the current rcutorture tests was located by mutation
testing by Iftekhar. The idea behind mutation testing is to automatically
mutate the code under test. If a given mutant is not caught by testing,
this is a hint that the testing might need to be improved, as was the
case here. Note that this is only a hint because it is possible to mutate
the code into something else that still works. For example, a mutation
that removes (say) a WARN_ON() will not normally result in a test failure.
This change resulted in the test failure caused by list mishandling,
which is fixed by the next commit.
Reported-by: "Ahmed, Iftekhar" <ahmedi@onid.oregonstate.edu>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Grace-period scans of the rcu_node combining tree normally
proceed quite quickly, so that it is very difficult to reproduce
races against them. This commit therefore allows grace-period
pre-initialization and cleanup to be artificially slowed down,
increasing race-reproduction probability. A pair of pairs of new
Kconfig parameters are provided, RCU_TORTURE_TEST_SLOW_PREINIT to
enable the slowing down of propagating CPU-hotplug changes up the
combining tree along with RCU_TORTURE_TEST_SLOW_PREINIT_DELAY to
specify the delay in jiffies, and RCU_TORTURE_TEST_SLOW_CLEANUP
to enable the slowing down of the end-of-grace-period cleanup scan
along with RCU_TORTURE_TEST_SLOW_CLEANUP_DELAY to specify the delay
in jiffies. Boot-time parameters named rcutree.gp_preinit_delay and
rcutree.gp_cleanup_delay allow these delays to be specified at boot time.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit sets CONFIG_RCU_TORTURE_TEST_SLOW_INIT=y, but leaves the
default time zero. This can be overridden by passing the
"--bootargs rcutree.gp_init_delay=1" argument to kvm.sh.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
A system misconfiguration that prevents qemu from running at all (for
example, a missing dynamically linked library) will keep the console.log
file from the previous run. This can fool the developer into thinking
that this failed run actually completed correctly. This commit therefore
overwrites the console.log file just before launching qemu.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
More recent qemu implementations really want "-enable-kvm", and the
"-soundhw pcspk" makes the script a bit less dependent on odd audio
libraries being installed. This commit therefore adds both to the
default qemu command line.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The mpstat command recently added the %gnice column, which messes up
the cpu2use.sh script's idle-CPU calculations. This commit therefore
uses $NF instead of $12 to select the last (%idle) column.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Normal rcutorture checking overestimates grace periods somewhat due to
the fact that there is a delay from a grace-period request until the
start of the corresponding grace period and another delay from the end
of that grace period to notification of the requestor. This means that
rcutorture's detection of RCU bugs is less sensitive than it might be.
It turns out that rcutorture also checks the underlying grace-period
"completed" counter (displayed in Reader Batch output), which in theory
allows rcutorture to do exact checks. In practice, memory misordering
(by both compiler and CPU) can result in false positives. However,
experience on x86 shows that these false positives are quite rare,
occurring less than one time per 1,000 hours of testing. This commit
therefore does the exact checking, giving a warning if any Reader Batch
blows happen, and flagging an error if they happen more often than
once every three hours in long tests.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The current rcutorture scripting checks for actual stalls (via the "Call
Trace:" check), but fails to spot the case where a stall ends just as it
is being detected. This commit therefore adds a check for this case.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Now that we have removed configs based on kernel version, we can also remove the
kversion parameter in kvm.sh.
Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Remove rcutorture configuration files which are no longer necessary.
Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Add config and boot parameters to enable the self tests in rcutorture testing.
Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The rcutorture scripts create a qemu-cmd script containing the actual
qemu command. However, this command references the build directory,
which will be overwritten by later builds. This commit therefore runs
the kernel out of the results directory so that less hand-editing is
required to re-run a previous test.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Pranith Kumar <bobby.prani@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
PREEMPT_RCU and TREE_PREEMPT_RCU serve the same function after
TINY_PREEMPT_RCU has been removed. This patch removes TREE_PREEMPT_RCU
and uses PREEMPT_RCU config option in its place.
Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Rename CONFIG_RCU_BOOST_PRIO to CONFIG_RCU_KTHREAD_PRIO and use this
value for both the per-CPU kthreads (rcuc/N) and the rcu boosting
threads (rcub/n).
Also, create the module_parameter rcutree.kthread_prio to be used on
the kernel command line at boot to set a new value (rcutree.kthread_prio=N).
Signed-off-by: Clark Williams <clark.williams@gmail.com>
[ paulmck: Ported to rcu/dev, applied Paul Bolle and Peter Zijlstra feedback. ]
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The CONFIG_RCU_CPU_STALL_VERBOSE Kconfig parameter causes preemptible
RCU's CPU stall warnings to dump out any preempted tasks that are blocking
the current RCU grace period. This information is useful, and the default
has been CONFIG_RCU_CPU_STALL_VERBOSE=y for some years. It is therefore
time for this commit to remove this Kconfig parameter, so that future
kernel builds will always act as if CONFIG_RCU_CPU_STALL_VERBOSE=y.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Add a "rw_lock" torture test to stress kernel rwlocks and their irq
variant. Reader critical regions are 5x longer than writers. As such
a similar ratio of lock acquisitions is seen in the statistics. In the
case of massive contention, both hold the lock for 1/10 of a second.
Signed-off-by: Davidlohr Bueso <dbueso@suse.de>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit changes rcutorture_runnable to torture_runnable, which is
consistent with the names of the other parameters and is a bit shorter
as well.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Although the test cases have been added, they must be specified explicitly
via the kvm.sh --configs argument in order to run them. This commit
therefore adds the RCU-tasks tests to the CFLIST so that they will be
run automatically by default.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds the TASKS01 and TASKS02 Kconfig fragments, along with
the corresponding TASKS01.boot and TASKS02.boot boot-parameter files
specifying that rcutorture test RCU-tasks instead of the default flavor.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
When starting a new torture run while an old one is still running, both
qemu processes can be outputting to the same console.out file. This can
cause quite a bit of confusion, so this commit checks for this situation,
which is normally indicated by nul bytes in the console output. Yes,
if your new run uses up an exact number of blocks of the file, this
check will be ineffective, but the odds are not bad.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Pranith Kumar <bobby.prani@gmail.com>
This commit specifies offstack cpumasks in TREE07 in order to catch
references to unallocated cpumask_var_t variables.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Pranith Kumar <bobby.prani@gmail.com>
Setting CONFIG_MAXSMP=y causes cpumasks to be moved offstack, which
introduces the possibility of NULL cpumask_var_t pointers. This commit
therefore enables CONFIG_MAXSMP=y in TREE01 to increase test coverage.
However, because CONFIG_MAXSMP=y implies 8192 CPUs, we need to use
the maxcpus= boot parameter to limit the number of CPUs to something
reasonable, which in turn requires updating the scripts to handle this.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Pranith Kumar <bobby.prani@gmail.com>
The current set of tests covers only cases where either all possible CPUs
are nohz_full= CPUs or none of them are. Because there have been some
recent bug escapes in cases where only some of the CPUs are nohz_full=
CPUs, this commit add a configuration where only half of the CPUs are
nohz_full= CPUs.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Pranith Kumar <bobby.prani@gmail.com>
This commit tries to get people into the correct directory before
creating the initrd directory.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Pranith Kumar <bobby.prani@gmail.com>
This commit sets the executable bit on test scripts config2frag.sh
and kvm.sh. Since #!/bin/bash is set in all the scripts, this commit
also drops it from all usage lines because the scripts can now all be
invoked directly.
Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Some of the scripts encode a default /bin/sh shell. On systems which use
dash as default shell, these scripts fail as they are bash scripts. I
encountered this while testing the sprintf() changes on a Debian system
where dash is the default shell.
This commit changes all such uses to use bash explicitly.
Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The CONFIG_PROVE_RCU_DELAY Kconfig parameter doesn't appear to be very
effective at finding race conditions, so this commit removes it.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Andi Kleen <ak@linux.intel.com>
[ paulmck: Remove definition and uses as noted by Paul Bolle. ]
Currently the post-processing complains about the lack of rcutorture
output when --buildonly is set and also emits misleading messages about
kernels being started and finishing. This commit suppresses these
complaints and messages.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The CFcommon file must now be present, which makes using the current
scripts against old kernel versions cumbersome. This commit therefore
makes the CFcommon file be optional, so that old kernel versions can be
used with current torture scripts.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The current scripting only keeps track of the git SHA-1 of the current
HEAD. This can cause confusion in cases where testing ran in a git
tree where changes had not yet been checked in. This commit therefore
also records the output of "git diff HEAD" to provide the information
needed to reconstruct the source tree that was tested.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit ensures that RCU-sched primitives are tested in
TREE_PREEMPT_RCU kernels, a combination that was previously omitted.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
The kvm-test-1-run.sh currently counts "sleep 1" commands to detect
hangs. This can fail spectacularly on busy systems, where "sleep 1"
might take far longer than one second to complete. This commit
therefore changes hang detection to use elapsed time measurements.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
The reaction of kvm-recheck.sh is obscure at best, and easy to miss
completely. This commit therefore prints "BUG: Build failed" in the
summary at the end of a run. This commit also adds the line of dashes
in cases where performance info is not available, and also avoids
printing nonsense diagnostics in cases where some of the normal test
output is not available. In addition, this commit saves off the .config
file even when the build fails.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds a CONFIG_RCU_TRACE=y version of TREE02 for debugging
purposes.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Currently, the scripts hard-code arch/x86/boot/bzImage, which does not
work well for other architectures. This commit therefore provides a
identify_boot_image function that selects the correct bzImage location
relative to the top of the Linux source tree. This commit also adds a
--bootimage argument that allows selecting some other file, for example,
"vmlinux".
This change requires that the definition of the QEMU variable be
computed earlier in order to identify where to look for the boot image
when it comes time to copy it to the results directory.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit applies quotes to permit multi-word --qemu-args and
--bootargs arguments.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
The current script does record qemu diagnostics, but the user has to
know where to look for them. This commit therefore puts them into the
Warnings file so that kvm-recheck.sh will display them. This change is
especially useful if you are in the habit of killing the qemu process
when you realize that you messed something up, but then later on wonder
why the process terminated early.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
In a normal torture-test run, the script inherits its environment
variables, but this does not work when producing a script that is
to run later. Therefore, definitions and exports are prepended to
a dryrun script but not to a script that is run immediately. This
commit reconciles this by placing definitions and exports at the
beginning of the script in both cases.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Conflicts:
tools/testing/selftests/rcutorture/bin/kvm.sh
The scripts produced by kvm.sh's "--dryrun script" argument were intended
for debugging rather than to run, but it is easier to debug if the script
output matches exactly what is run. This commit therefore makes this
script runnable.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
The torture tests need to set specific values for their respective
Kconfig options (e.g., CONFIG_LOCK_TORTURE_TEST), and must therefore
filter any conflicting definitions from the Kconfig fragment
file. Unfortunately, the code in kvm-build.sh was looking only for
CONFIG_RCU_TORTURE_TEST. This commit therefore handles the general case
of CONFIG_[A-Z]*_TORTURE_TEST.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit makes the torture scripts a bit more RCU-independent.
It also removes a redundant export of this same shell variable.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit makes the torture scripts a bit more RCU-independent.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit makes the torture scripts a bit more RCU-independent.
It also drops an redundant "export" statement.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit makes the torture scripts a bit more RCU-independent.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit makes the torture scripts a bit more RCU-independent by
changing RCU_BUILDONLY to TORTURE_BUILDONLY. It also removes an
unnecessary export command.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit makes the torture scripts a bit more RCU-independent.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit makes the torture scripts a bit more RCU-independent by
changing RCU_KMAKE_ARG to TORTURE_KMAKE_ARG. It also removes the
unnecessary export command.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Some environments require some variation on "make defconfig" to initialize
the .config file. This commit therefore adds a --defconfig argument to
allow this to be specified. The default value is of course "defconfig".
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit makes the torture scripts a bit more RCU-independent.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
It can be a bit jarring to see a locking test complain about RCU, so
this commit renames parse-rcutorture.sh to parse-torture.sh and makes
the messages it emits more generic.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit makes the output of "--dryrun sched" more user-friendly,
clearly indicating the batch starts.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
The --builddir and --relbuilddir options were initially intended to handle
parallel tests. However, since commit 43e38ab3d5 (Enable concurrent
rcutorture runs), the script manages multiple build directories as
needed for parallel testing. This commit therefore removes these two
obsolete options.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit logs the progress text that kvm.sh outputs, improving
after-the-fact troubleshooting.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
The output of each kvm-test-1-run.sh script is placed into a file
whose name parallels that of the build directory. This means that
the kvm-test-1-run.sh output is overwritten by later run. This
commit therefore places the kvm-test-1-run.sh output into the
per-test-case directory in the "res" hierarchy.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
It used to be that:
git ls-files "*Kconfig*"
would find all Kconfig files and would only find Kconfig files. This
commit renames TREE_RCU-Kconfig.txt to TREE_RCU-kconfig.txt so that this
is once again true.
Signed-off-by: Paul Bolle <pebolle@tiscali.nl>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit adds the kvm-recheck-lock.sh plug-in for locktorture to
print out lock-specific progress statistics.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit adds a trivial set of configuration files for lock
torturing.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit uses the standard software ploy of introducing another
level of indirection below the configs directory. This allows each
torture-test suite to have its own set of Kconfig files, boot parameters,
and version-specific scripts. Initially, we have only rcu, but lock
will follow soonish.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
The kvm-test-1-rcu.sh is not specific to RCU, so this commit renames it
to kvm-test-1-run.sh.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
The current set of functions in ver_functions.sh have APIs that are
specific to RCU. This commit therefore makes an RCU-independent function
that outputs version-specific boot arguments. This has the benefit that
a test-type-independent call in kvm-test-1-rcu.sh can now handle any type
of test, given a test-type-specific set of files in a configs directory.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Currently, CONFIG_RCU_TORTURE_TEST=y is hardcoded into the
kvm-test-1-rcu.sh script and CONFIG_PRINTK_TIME=y is mentioned in each
and every configs file. This commit creates a CFcommon file for these
two Kconfig parameters, and modifies kvm-test-1-rcu.sh to copy this new
file into the .config file during the build. This change will allow
these scripts to operate on torture types other than just rcutorture.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit adds a pair of files in the configs directory to allow
test-the-test runs of rcutorture via a "--configs BUSTED" argument to
the kvm.sh script.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit creates a plug-in to allow kvm-recheck.sh to process
non-rcutorture console output.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit prevents the results directory from being created for
dryruns. However, a script generated from a dryrun will create
the results directory should it be run.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org
This commit adds a "(!)" flag after the number of CPUs required by a
given test if that test requires more than the available number of CPUs.
Note that these flags appear only when the number of CPUs is specified
using the --cpus argument. In the absence of a --cpus argument, no
tests are flagged.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Running the standard set of rcutorture tests on 24 CPUs results in
the following sub-optimal schedule:
----start batch----
TREE07 16
----start batch----
TREE08 16
SRCU-P 8
----start batch----
TREE01 8
TREE02 8
TREE03 8
----start batch----
TREE04 8
TREE05 8
TREE06 8
----start batch----
SRCU-N 4
TINY01 1
TINY02 1
TREE09 1
If one of the eight-CPU runs were to be moved into the first batch,
the test suite would complete in four batches rather than five.
This commit therefore uses a greedy algorithm to re-order the test
entries so that the sequential batching will produce an optimal schedule
in this case:
----start batch----
TREE07 16
SRCU-P 8
----start batch----
TREE08 16
TREE01 8
----start batch----
TREE02 8
TREE03 8
TREE04 8
----start batch----
TREE05 8
TREE06 8
SRCU-N 4
TINY01 1
TINY02 1
TREE09 1
Please note that this is still not an optimal bin-packing algorithm,
however, it does produce optimal solutions for most common scenarios.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit fixes handling numbering of multiple runs of the same test
so as to disambiguate output.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Actual rcutorture tests take considerable time and machine resources,
so it is inconvenient to actually do an rcutorture run when optimizing
the bin-packing algorithm. This commit therefore adds a --dryrun
argument, which defaults to doing a run, but for which "sched"
says to simply print the run schedule and "script" dumps the script
without running it.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
The message complains about a build directory when it should instead
be complaining about the results directory, so this commit fixes it.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
The rcutorture tests run by default range from using one CPU to using
sixteen of them. Therefore, rcutorture testing could be sped up
significantly simply by running the kernels in parallel. Building
them in parallel is not all that helpful: "make -j" is usually a
better bet. So this commit takes a new "--cpus" argument that
specifies how many CPUs rcutorture is permitted to use for its
parallel runs. The default of zero does sequential runs as before.
The bin-packing is minimal, and will be grossly suboptimal for
some configurations. However, powers of two work reasonably well.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Both SRCU-P and SRCU-N specify eight CPUs, which results in four
iterations for a parallel run on 32 CPUs. This commit reduces SRCU-N
to four CPUs (but leaving SRCU-P at eight) to speed up parallel runs,
while maintaining essentially the same test coverage.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Currently, most qemu flags are calculated in kvm-test-1-rcu.sh,
except that -nographics is set up by kvm.sh. This commit promotes
one-stop shopping by consolidating the determination of qemu flags into
kvm-test-1-rcu.sh.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Parallel rcutorture runs is valuable on large systems, but it is not a
good idea to do (say) five builds in parallel if each build believes it
has the whole system at its disposal, especially if the system is shared.
It is also bad to restrict the build to (say) a single CPU just because
the corresponding rcutorture run uses only a single CPU. This commit
therefore adds a kvm-test-1-rcu.sh ability to pause after the build
completes, which will allow kvm.sh to do a number of builds serially
(with each build thus having the full system at its disposal), then
allow the rcutorture runs to proceed in parallel.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Currently, most boot flags are calculated in kvm-test-1-rcu.sh, except
that rcutorture.test_no_idle_hz and rcutorture.verbose are set up by
kvm.sh. This commit promotes one-stop shopping by consolidating the
determination of boot flags into kvm-test-1-rcu.sh.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Although the script name and arguments are logged in the results directory,
it is more convenient to see it in the output. This commit therefore
adds the output of this information.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Sometime problems can manifest themselves as unusually slow grace periods.
This commit therefore prints the number of rcutorture updates during the
test and the number per second. These statistics are harvested from the
config.out and qemu-cmd files, and are silently omitted if these files
are not available, as would be the case if there was a build failure or
a boot-time hang.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
All of the rcutorture scripts has the usual GPL header, which contains
a long-obsolete postal address for FSF. To avoid the need to track the
FSF office's movements, this commit substitutes the URL where GPL may
be found.
Reported-by: Greg KH <gregkh@linuxfoundation.org>
Reported-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The output of the rcutorture scripts often requires interpretation, so
this commit simplifies this interpretation by tagging messages as
BUGs (colored red) or WARNINGs (colored yellow).
Reported-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Repeatedly running a given test, for example, by repeating the name
as in "--configs "TREE08 TREE08 TREE08" records the results only of
the last run of this test. This is because the earlier results are
overwritten by the later results.
This commit therefore checks for earlier results, using numbered
file extensions to distinguish multiple runs. The earlier example
would therefore create directories TREE01, TREE01.2, and TREE01.3.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
The commit causes kvm.sh to invoke kvm-recheck.sh at the end of each
run, and causes kvm-recheck.sh to print only the name of the test, not
the full path to the corresponding Kconfig file.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
The TREE08 Kconfig fragment does not enable tracing, which is appropriate
for its test case. However, this can be inconvenient in cases where
TREE08 locates RCU bugs. This commit therefore adds a TREE08-T that
differs from TREE08 only in enabling CONFIG_RCU_TRACE.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
This commit adds the --kmake-arg to kvm.sh, which allows passing in
things like "V=1" to see the build commands, as well as enabling the
CROSS_COMPILE= make macro used for cross-building.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
This commit adds the --no-initrd argument to kvm.sh, which permits
initrd to be contained in a root partition specified by the --bootargs
argument. Without --no-initrd, the kernel build expects an initrd
directory in the same rcutorture directory that contains bin and configs.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
This commits adds the --qemu-args argument to kvm.sh that is required
to pass boot devices down through to qemu.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
This commit allows easy specification of trace_event lists, among other
things.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
This commit adds --buildonly, which does the builds specified by the
--configs argument, but does not boot or test the resulting kernels.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Don't grab the configuration fragment from the configs directory because
it might well have been changed since the test was run. Instead, use
the ConfigFragment file that was placed in the results directory.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
As it stands, the default kernel boot parameters generated from
the Kconfig fragment will override any supplied with the .boot
file that can optionally accompany a Kconfig fragment. Rearrange
ordering to permit the specific .boot arguments to override those
generated by analyzing the Kconfig fragment.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
This commit expands the checks for what architecture is running to generate
additional qemu-system- commands, then uses the resulting qemu-system-
command name to choose different qemu arguments as needed for different
architectures.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
The --rcu-kvm argument was intended to allow the scripts to live in
an alternate location. Unfortunately, this prevents the kvm.sh script
from using common functions until after it finished parsing arguments,
because it doesn't know where to find them until then. However, "cp -a"
and "ln -s" work pretty well, so lack of an --rcu-kvm argument can be
easily worked around.
This commit therefore removes this argument.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
The qemu -name argument doesn't seem to be useful in this environment,
so this commit removes it.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
The task of working out which flavor of qemu to use gets more complex
as more types of CPUs are supported. Adding Power makes three in addition
to 32-bit and 64-bit x86, so it is time to pull this out into a function.
This commit therefore creates an identify_qemu function and also adds
a --qemu-cmd command-line argument for the inevitable case where the
identify_qemu cannot figure it out.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
The commit uses configcheck.sh from within configinit.sh, replacing the
imperfect inline expansion that was there before.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
This commit drops no-longer-needed diagnostics from the output. Some of
them are retained in logfiles, in case they are ever needed.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
The TINY_RCU test cases were first put in place many years ago, and have
been incrementally modified rather than being reworked. This commit
therefore completes a long-overdue reworking of the TINY_RCU test cases.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
The TREE_RCU test cases were first put in place many years ago, and have
been incrementally modified rather than being reworked. This commit
therefore completes a long-overdue reworking of the TREE_RCU test cases.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Use .boot facility to ease inclusion of SRCU into automated testing.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
The v3.12 version of the kernel added the CONFIG_NO_HZ_FULL_SYSIDLE
Kconfig parameter, so this commit adds a version transition at that
point.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Some Kconfig fragments require rcutorture module parameters to
do optimal testing, for example, a configuration for SRCU would
need rcutorture.torture_type=srcu. This commit therefore adds a
per-Kconfig-fragment boot-parameter capability.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Different Kconfig parameters apply to different kernel versions, as
do different rcutorture module parameters. This commit allows the
rcutorture test scripts to adjust for different kernel versions.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Allow datestamp to be specified to allow tests to be broken up and run
in parallel.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
This commit adds the test framework that I used to test RCU under KVM.
This consists of a group of scripts and Kconfig fragments.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
