Under some conditions, a given run's vmlinux file will be compressed,
so that it is named vmlinux.xz rather than vmlinux. in such cases,
kvm-find-errors.sh will complain about the nonexistence of vmlinux.
This commit therefore causes kvm-find-errors.sh to check for vmlinux.xz
as well as for vmlinux.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, if the torture.sh allmodconfig step fails, this is counted as
an error (as it should be), but there is also an extraneous complaint
about a missing log file. This commit therefore adds that log file,
which is hoped to reduce confused reactions to the error report.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, torture.sh will compress the vmlinux files for KASAN and
KCSAN runs. But it will compress all of the files, including those
copied verbatim by the kvm-again.sh script. Compression takes around ten
minutes, so this is not a good thing. This commit therefore compresses
only one of a given set of identical vmlinux files, and then hard-links
it to the directories produced by kvm-again.sh.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit causes torture.sh to use the new --bootargs and --datestamp
parameters to kvm-again.sh in order to avoid redundant kernel builds
during rcuscale and refscale testing. This trims the better part of an
hour off of torture.sh runs that use --do-kasan.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds a --datestamp parameter to kvm-again.sh, which, in
contrast to the existing --rundir argument, specifies only the last
segments of the pathname. This addition enables torture.sh to use
kvm-again.sh in order to avoid redundant kernel builds.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
As it should, the kvm-recheck.sh script sets the TORTURE_SUITE bash
variable based on the type of rcutorture test being run. However,
it does not export it. Which is OK, at least until you try running
kvm-again.sh on either a rcuscale or a refscale test, at which point you
get false-positive "no success message, N successful version messages"
errors. This commit therefore causes the kvm-recheck.sh script to export
TORTURE_SUITE, suppressing these false positives.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The kvm-again.sh script, when running locally, can place the QEMU output
into kvm-test-1-run-qemu.sh.out instead of kvm-test-1-run.sh.out. This
commit therefore makes kvm-test-1-run-qemu.sh check both locations.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit drags the rcutorture scripting kicking and screaming into the
twenty-first century by making use of the BSD-derived mktemp command to
create temporary files and directories. In happy contrast to many of its
ill-behaved predecessors, mktemp seems to actually work reasonably reliably!
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The kvm-again.sh script can be used to repeat short boot-time tests,
but the kernel boot arguments cannot be changed. This means that every
change in kernel boot arguments currently necessitates a kernel build,
which greatly increases the duration of kernel-boot testing.
This commit therefore adds a --bootargs parameter to kvm-again.sh,
which allows a given kernel to be repeatedly booted, but overriding
old and adding new kernel boot parameters. This allows an old kernel
to be booted with new kernel boot parameters, avoiding the overhead of
rebuilding the kernel under test.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, kvm-check-branches.sh causes each kvm.sh invocation create a
separate date-stamped directory, then after that invocation completes,
moves it into the *-group/NNNN directory. This works, but makes it more
difficult to monitor an ongoing run. This commit therefore uses the
kvm.sh --datestamp argument to make kvm.sh put the output in the right
place to start with, and also dispenses with the additional level of
datestamping. (Those wanting datestamps can find them in the log files.)
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
A recent change to the DEBUG_INFO Kconfig option means that simply adding
CONFIG_DEBUG_INFO=y to the .config file and running "make oldconfig" no
longer works. It is instead necessary to add CONFIG_DEBUG_INFO_NONE=n
and (for example) CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y.
This combination will then result in CONFIG_DEBUG_INFO being selected.
This commit therefore updates the Kconfig options produced in response
to the kvm.sh --gdb, --kasan, and --kcsan Kconfig options.
Fixes: f9b3cd2457 ("Kconfig.debug: make DEBUG_INFO selectable from a choice")
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
If a remote system fails in certain ways, for example, if it is rebooted
without removing the contents of the /tmp directory, its remote.run file
never will be removed and the kvm-remote.sh script will loop waiting
forever. The manual workaround for this (hopefully!) rare event is to
manually remove the file, which will cause the results up to the reboot
to be collected and evaluated.
Unfortunately, to work out which system is holding things up, the user
must refer to the name of the last system whose results were collected,
then look up the name of the next system in sequence, then manually
remove the remote.run file. Even more unfortunately, this procedure can
be fooled in runs where each system handles more than one batch should
a given system take longer than expected, causing the systems to be
handled out of order.
This commit therefore causes kvm-remote.sh to print out the name of
the system it will wait on next, allowing the user to refer directly
to that name. Making the kvm-remote.sh script automatically handle
unscheduled termination of the qemu processes is left as future work.
Quite possibly deep future work.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The torture.sh script provides extra memory for scftorture and rcuscale.
However, the total memory provided is only 1G, which is less than the
2G that is required for KASAN testing. This commit therefore ups the
torture.sh script's 1G to 2G.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Now that the Tasks RCU flavors are selected by their users rather than
by the rcutorture scenarios, torture.sh fails when attempting to run
NOPREEMPT scenarios for refscale and rcuscale. This commit therefore
makes torture.sh specify CONFIG_TASKS_TRACE_RCU=y to avoid such failure.
Why not also CONFIG_TASKS_RCU? Because tracing selects this one.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
KASAN allots significant memory to track allocation state, and the amount
of memory has increased recently, which results in frequent OOMs on a
few of the rcutorture scenarios. This commit therefore provides 2G of
memory for --kasan runs, up from the 512M default.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, torture.sh saves only the build output and exit code from the
"make allmodconfig" test. This commit also saves the .config file.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
There is an extraneous "scf" in the per_version_boot_params shell function
used by scftorture. No harm done in that it is just passed as an argument
to the /init program in initrd, but this commit nevertheless removes it.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Now that CONFIG_PREEMPT_DYNAMIC=y is the default, kernels that are
ostensibly built with CONFIG_PREEMPT_NONE=y or CONFIG_PREEMPT_VOLUNTARY=y
are now actually built with CONFIG_PREEMPT=y, but are by default booted
so as to disable preemption. Although this allows much more flexibility
from a single kernel binary, it means that the current rcutorture
scenarios won't find build errors that happen only when preemption is
fully disabled at build time.
This commit therefore adds CONFIG_PREEMPT_DYNAMIC=n to several scenarios,
and while in the area switches one from CONFIG_PREEMPT_NONE=y to
CONFIG_PREEMPT_VOLUNTARY=y to add coverage of this Kconfig option.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit passes the csdlock_debug=1 kernel parameter in order to
enable CSD-lock stall reports for torture.sh scftorure runs.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The kvm-again.sh script reruns an previously built set of kernels, so
the vmlinux files are associated with that previous run, not this on.
This results in kvm-find_errors.sh reporting spurious failed-build errors.
This commit therefore omits the vmlinux check for kvm-again.sh runs.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adjusts the scftorture PREEMPT and NOPREEMPT scenarios to
account for the TASKS_RCU Kconfig option being explicitly selected rather
than computed in isolation.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, a CONFIG_PREEMPT_NONE=y kernel substitutes normal RCU for
RCU Tasks Rude and RCU Tasks Trace. Unless that kernel builds rcuscale,
whether built-in or as a module, in which case these RCU Tasks flavors are
(unnecessarily) built in. This both increases kernel size and increases
the complexity of certain tracing operations. This commit therefore
decouples the presence of rcuscale from the presence of RCU Tasks Rude
and RCU Tasks Trace.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, a CONFIG_PREEMPT_NONE=y kernel substitutes normal RCU for
RCU Tasks. Unless that kernel builds rcuscale, whether built-in or as
a module, in which case RCU Tasks is (unnecessarily) built. This both
increases kernel size and increases the complexity of certain tracing
operations. This commit therefore decouples the presence of rcuscale
from the presence of RCU Tasks.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, a CONFIG_PREEMPT_NONE=y kernel substitutes normal RCU for
RCU Tasks Rude and RCU Tasks Trace. Unless that kernel builds refscale,
whether built-in or as a module, in which case these RCU Tasks flavors are
(unnecessarily) built in. This both increases kernel size and increases
the complexity of certain tracing operations. This commit therefore
decouples the presence of refscale from the presence of RCU Tasks Rude
and RCU Tasks Trace.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, a CONFIG_PREEMPT_NONE=y kernel substitutes normal RCU for
RCU Tasks. Unless that kernel builds refscale, whether built-in or as a
module, in which case RCU Tasks is (unnecessarily) built in. This both
increases kernel size and increases the complexity of certain tracing
operations. This commit therefore decouples the presence of refscale
from the presence of RCU Tasks.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The rcutorture test suite makes double use of the rcutorture.stat_interval
module parameter. As its name suggests, it controls the frequency
of statistics printing, but it also controls the rcu_torture_writer()
stall timeout. The current setting of 15 seconds works surprisingly well.
However, given that the RCU tasks stall-warning timeout is ten -minutes-,
15 seconds is too short for TASKS02, which runs a non-preemptible kernel
on a single CPU.
This commit therefore adds checks for per-scenario specification of the
rcutorture.stat_interval module parameter.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Now that CONFIG_PREEMPT_DYNAMIC=y is the default, TASKS02 no longer
builds a pure non-preemptible kernel that uses Tiny RCU. This commit
therefore fixes this new hole in rcutorture testing by adding
CONFIG_PREEMPT_DYNAMIC=n to the TASKS02 rcutorture scenario.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Unless a kernel builds rcutorture, whether built-in or as a module, that
kernel is also built with CONFIG_TASKS_RUDE_RCU, whether anything else
needs Tasks Rude RCU or not. This unnecessarily increases kernel size.
This commit therefore decouples the presence of rcutorture from the
presence of RCU Tasks Rude.
However, there is a need to select CONFIG_TASKS_RUDE_RCU for testing
purposes. Except that casual users must not be bothered with
questions -- for them, this needs to be fully automated. There is
thus a CONFIG_FORCE_TASKS_RUDE_RCU that selects CONFIG_TASKS_RUDE_RCU,
is user-selectable, but which depends on CONFIG_RCU_EXPERT.
[ paulmck: Apply kernel test robot feedback. ]
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, a CONFIG_PREEMPT_NONE=y kernel substitutes normal RCU for
RCU Tasks. Unless that kernel builds rcutorture, whether built-in or as
a module, in which case RCU Tasks is (unnecessarily) used. This both
increases kernel size and increases the complexity of certain tracing
operations. This commit therefore decouples the presence of rcutorture
from the presence of RCU Tasks.
However, there is a need to select CONFIG_TASKS_RCU for testing purposes.
Except that casual users must not be bothered with questions -- for them,
this needs to be fully automated. There is thus a CONFIG_FORCE_TASKS_RCU
that selects CONFIG_TASKS_RCU, is user-selectable, but which depends
on CONFIG_RCU_EXPERT.
[ paulmck: Apply kernel test robot feedback. ]
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Unless a kernel builds rcutorture, whether built-in or as a module, that
kernel is also built with CONFIG_TASKS_TRACE_RCU, whether anything else
needs Tasks Trace RCU or not. This unnecessarily increases kernel size.
This commit therefore decouples the presence of rcutorture from the
presence of RCU Tasks Trace.
However, there is a need to select CONFIG_TASKS_TRACE_RCU for
testing purposes. Except that casual users must not be bothered with
questions -- for them, this needs to be fully automated. There is thus
a CONFIG_FORCE_TASKS_TRACE_RCU that selects CONFIG_TASKS_TRACE_RCU,
is user-selectable, but which depends on CONFIG_RCU_EXPERT.
[ paulmck: Apply kernel test robot feedback. ]
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, any kernel built with CONFIG_PREEMPTION=y also gets
CONFIG_TASKS_RCU=y, which is not helpful to people trying to build
preemptible kernels of minimal size.
Because CONFIG_TASKS_RCU=y is needed only in kernels doing tracing of
one form or another, this commit moves from TASKS_RCU deciding when it
should be enabled to the tracing Kconfig options explicitly selecting it.
This allows building preemptible kernels without TASKS_RCU, if desired.
This commit also updates the SRCU-N and TREE09 rcutorture scenarios
in order to avoid Kconfig errors that would otherwise result from
CONFIG_TASKS_RCU being selected without its CONFIG_RCU_EXPERT dependency
being met.
[ paulmck: Apply BPF_SYSCALL feedback from Andrii Nakryiko. ]
Reported-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>
Tested-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>
Tested-by: Zhouyi Zhou <zhouzhouyi@gmail.com>
Cc: Andrii Nakryiko <andrii@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
For consecutive numbers the lscpu command collapses the output and just
shows the range with start and end. The processors are numbered that
way on POWER8.
$ sudo ppc64_cpu --smt=8
$ lscpu | grep '^NUMA node'
NUMA node(s): 2
NUMA node0 CPU(s): 0-79
NUMA node8 CPU(s): 80-159
This causes the heuristic to detect the number threads per core, looking
for the number after the first comma, to fail, and QEMU aborts because of
invalid arguments.
$ lscpu | grep '^NUMA node0' | sed -e 's/^[^,-]*(,|\-)\([0-9]*\),.*$/\1/'
NUMA node0 CPU(s): 0-79
But the lscpu command shows the number of threads per core:
$ sudo ppc64_cpu --smt=8
$ lscpu | grep 'Thread(s) per core'
Thread(s) per core: 8
$ sudo ppc64_cpu --smt=off
$ lscpu | grep 'Thread(s) per core'
Thread(s) per core: 1
This commit therefore directly uses that value and replaces use of grep
with "sed -n" and its "p" command.
Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit weakens the checks of the kvm.sh script's --torture parameter
and the kvm-recheck.sh script's parsing so that experimental torture tests
may be created without updating these two scripts. The changes required
are to the appropriate Makefile and Kconfig file, plus a directory
whose name begins with "X" must be added to the rcutorture/configs file.
This new directory's name can then be passed in via the kvm.sh script's
--torture parameter.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The torture.sh script normally runs unattended, so there is not much
point in the "ssh" command asking for a password. This commit therefore
adds the "-o Batchmode=yes" argument to each "ssh" command to cause it
to fail rather than ask for a password.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
An "echo" slipped in between an "ssh" and the "ret=$?" that was intended
to collect its exit code, which prevents torture.sh from detecting
"ssh" failure. This commit therefore reassociates the two.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, the rcupdate.rcu_normal and rcupdate.rcu_expedited kernel
boot parameters are not regularly tested. The potential addition of
polled expedited grace-period APIs increases the amount of code that is
affected by these kernel boot parameters. This commit therefore adds a
"--do-rt" argument to torture.sh to exercise these kernel-boot options.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adjusts RUDE01 to 3 CPUs and TRACE01 to 5 CPUs in order to
test Tasks RCU's ability to handle non-power-of-two numbers of CPUs.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Thie commit adds kernel boot parameters to the SRCU-N and SRCU-P
rcutorture scenarios to cause SRCU-N to test contention-based resizing
and SRCU-P to test init_srcu_struct()-time resizing. Note that this
also tests never-resizing because the contention-based resizing normally
takes some minutes to make the shift.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit fixes a couple of typos: s/--doall/--do-all/ and
s/--doallmodconfig/--do-allmodconfig/.
[ paulmck: Add Fixes: supplied by Paul Menzel. ]
Fixes: a115a775a8 ("torture: Add "make allmodconfig" to torture.sh")
Reported-by: Paul Menzel <pmenzel@molgen.mpg.de>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The torture-test scripting's long-standing use of KVM as the environment
variable tracking the pathname of the rcutorture directory now conflicts
with allmodconfig builds due to the virt/kvm/Makefile.kvm file's use
of this as a makefile variable. This commit therefore changes the
torture-test scripting from KVM to RCUTORTURE, avoiding the name conflict.
Reported-by: Zhouyi Zhou <zhouzhouyi@gmail.com>
Tested-by: Zhouyi Zhou <zhouzhouyi@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, an obtuse compiler diagnostic can fool kvm-find-errors.sh
into believing that the build was successful. This commit therefore
adds a check for a missing vmlinux file. Note that in the case of
repeated torture-test scenarios ("--configs '2*TREE01'"), the vmlinux
file will only be present in the first directory, that is, in TREE01
but not TREE01.2.
Link: https://lore.kernel.org/lkml/36bd91e4-8eda-5677-7fde-40295932a640@molgen.mpg.de/
Reported-by: Paul Menzel <pmenzel@molgen.mpg.de>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The torture.sh scripts currently duplicates the summary lines, getting
one during the run phase and one during the summary phase of each run.
This commit therefore removes the run phase from consideration so as to
get only one summary line per run.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit ups the retries for downloading the build-product tarball
to a given remote system from once to five times, the better to handle
transient network failures.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Compressing KASAN vmlinux files reduces torture.sh res file size from
about 100G to about 50G, which is good, but the KCSAN vmlinux files
are also large. Compressing them reduces their size from about 700M to
about 100M (but of course your mileage may vary). This commit therefore
compresses both KASAN and KCSAN vmlinux files.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit further improves torture.sh run summaries by indicating
which runs' "Bugs:" counts are all KCSAN reports, and further printing
an additional end-of-run summary line when all errors reported in all
runs were KCSAN reports.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Runs having only KCSAN reports will normally print a summary line
containing only a "Bugs:" entry. However, these bugs might or might
not be KCSAN reports. This commit therefore flags runs in which all the
"Bugs:" entries are KCSAN reports.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, torture.sh lists the failed runs, but it is up to the user
to work out what failed. This is especially annoying for KCSAN runs,
where RCU's tighter definitions result in failures being reported for
other parts of the kernel. This commit therefore outputs "Summary:"
lines for each failed run, allowing the user to more quickly identify
which failed runs need focused attention.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
In a clear-cut case of "not thinking big enough", kvm.sh limits the
multipliers for torture-test scenarios to three digits. Although this is
large enough for any single system that I have ever run rcutorture on,
it does become a problem when you want to use kvm-remote.sh to run as
many instances of TREE09 as fit on a set of 20 systems with 80 CPUs each.
Yes, one could simply say "--configs '800*TREE09 800*TREE09'", but this
commit removes the need for that sort of hacky workaround by permitting
four-digit repetition numbers, thus allowing "--configs '1600*TREE09'".
Five-digit repetition numbers remain off the menu. Should they ever
really be needed, they can easily be added!
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Console logs can sometimes have trailing control-M characters, which the
forward-progress evaluation code in kvm-recheck-rcu.sh passes through
to the user output. Which does not cause a technical problem, but which
can look ugly. This commit therefore strips the control-M characters.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
With the addition of multiple callback-flood kthreads, the maximum number
of callbacks from any one of those kthreads is reported in the rcutorture
run summary. This commit changes this to report the sum of each kthread's
maximum number of callbacks in a given callback-flooding episode.
Cc: Neeraj Upadhyay <neeraj.iitr10@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit modifies the TASKS01 scenario to use four callback queues
and the TRACE01 scenario to use two queues, thus providing testing of
multiple queues by default.
Cc: Neeraj Upadhyay <neeraj.iitr10@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The echo commands following initialization of the "oldrun" variable need
to be "tee"d to $oldrun/remote-log. This commit fixes several stragglers.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The "exit 4" in kvm-remote.sh is pointlessly redirected, so this commit
removes the redirection.
Fixes: 0092eae4cb ("torture: Add kvm-remote.sh script for distributed rcutorture test runs")
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adjusts the TRACE02 scenario to use a pair of callback-flood
kthreads. This in turn forces lock contention on the single RCU Tasks
Trace callback queue, which forces use of all CPUs' queues, thus testing
this transition. (No, there is not yet any way to transition back.
Cc: Neeraj Upadhyay <neeraj.iitr10@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit enables two callback-flood kthreads for the TREE02 scenario
and 28 for the TREE10 scenario.
Cc: Neeraj Upadhyay <neeraj.iitr10@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, a transient network error can kill a run if it happens while
downloading the tarball to one of the target systems. This commit
therefore does a 60-second wait and then a retry. If further experience
indicates, a more elaborate mechanism might be used later.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit makes kvm-find-errors.sh check for and report undefined
symbols that are detected at link time.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit brings the kvm.sh script's help text up to date with recently
(and some not-so-recently) added parameters.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
All of the uses of CONFIG_RCU_FAST_NO_HZ=y that I have seen involve
systems with RCU callbacks offloaded. In this situation, all that this
Kconfig option does is slow down idle entry/exit with an additional
allways-taken early exit. If this is the only use case, then this
Kconfig option nothing but an attractive nuisance that needs to go away.
This commit therefore removes the RCU_FAST_NO_HZ Kconfig option.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
All of the rcu scenarios that mentioning CONFIG_RCU_FAST_NO_HZ disable it.
But this Kconfig option is disabled by default, so this commit removes
the pointless "CONFIG_RCU_FAST_NO_HZ=n" lines from these scenarios.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
All of the rcuscale and refscale scenarios that mention the Kconfig option
CONFIG_RCU_FAST_NO_HZ disable it. But this Kconfig option is disabled by
default, so this commit removes the pointless "CONFIG_RCU_FAST_NO_HZ=n"
lines from these scenarios.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
With CONFIG_PREEMPT_DYNAMIC=y, the kernel builds with CONFIG_PREEMPTION=y
because preemption can be enabled at runtime. This prevents any tests
of Tiny RCU or Tiny SRCU from running correctly. This commit therefore
explicitly sets CONFIG_PREEMPT_DYNAMIC=n for those scenarios.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit causes kvm-remote.sh to print the size of the tarball that
is downloaded to each of the remote systems. This size can help with
performance projections and analysis.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
By default, torture.sh allots 512M of memory for each guest OS. However,
when running scftorture with KASAN, 1G is needed. This commit therefore
causes torture.sh to provide the required 1G.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Compressing gigabyte vmlinux files can take some time, and it can be a
bit annoying to not know many more batches of compression there will be.
This commit therefore makes torture.sh print the number of files to be
compressed just before starting compression and just after compression
completes.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, the --kcsan argument to kvm.sh applies a laundry list of
Kconfig options. Now that KCSAN provides the CONFIG_KCSAN_STRICT Kconfig
option, this commit reduces the laundry list to this one option.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
It turns out that certain types of early boot bugs can result in reboot
loops, even within a guest OS running under qemu/KVM. This commit
therefore upgrades the kvm-test-1-run-qemu.sh script's hang-detection
heuristics to detect such situations and to terminate the run when
they occur.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The kvm-test-1-run-qemu.sh script logs the torture-test start time and
also when it starts getting impatient for the test to finish. However, it
does not timestamp these log messages, which can make debugging needlessly
challenging. This commit therefore adds timestamps to these messages.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
There was a time long ago when the "test" command's documentation
claimed that the "-a" and "-o" arguments did something useful.
But this documentation now suggests letting the shell execute
these boolean operators, so this commit applies that suggestion to
kvm-test-1-run-qemu.sh.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit causes kvm-test-1-run-batch.sh to use the new
kvm-assign-cpus.sh and kvm-get-cpus-script.sh scripts to create a
TORTURE_AFFINITY environment variable containing either an empty string
(for no affinity) or a list of CPUs to pin the scenario's vCPUs to.
The additional change to kvm-test-1-run.sh places the per-scenario
number-of-CPUs information where it can easily be found.
If there is some reason why affinity cannot be supplied, this commit
prints and logs the reason via changes to kvm-again.sh.
Finally, this commit updates the kvm-remote.sh script to copy the
qemu-affinity output files back to the host system.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
There is "qemu-affinity", "qemu-cmd", "qemu-retval", but also "qemu_pid".
This is hard to remember, not so good for bash tab completion, and just
plain inconsistent. This commit therefore renames the "qemu_pid" file to
"qemu-pid". A couple of the scripts must deal with old runs, and thus
must handle both "qemu_pid" and "qemu-pid", but new runs will produce
"qemu-pid".
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The jitter.sh script has some entertaining awk code to generate a
hex mask from a randomly selected CPU number, which is handed to the
"taskset" command. Except that this command has a "-c" parameter to
take a comma/dash-separated list of CPU numbers. This commit therefore
saves a few lines of awk by switching to a single-number CPU list.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
There is no way to place the vCPUs in a two-CPU rcutorture scenario to
get variable memory latency. This commit therefore upgrades the current
two-CPU rcutorture scenarios to four CPUs.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit causes the kvm-test-1-run-qemu.sh script to check the
TORTURE_AFFINITY environment variable and to add "taskset" commands to
the qemu-cmd file. The first "taskset" command is applied only if the
TORTURE_AFFINITY environment variable is a non-empty string, and this
command pins the current scenario's guest OS to the specified CPUs.
The second "taskset" command reports the guest OS's affinity in a new
"qemu-affinity" file.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, kvm-test-1-run-qemu.sh applies redirection to each and every
line of each qemu-cmd script. Only the first line (the only one that
is not a bash comment) needs to be redirected. Although redirecting
the comments is currently harmless, just adding to the comment, it is
an accident waiting to happen. This commit therefore adjusts the "sed"
command to redirect only the qemu-system* command itself.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit causes kvm.sh to use the new kvm-assign-cpus.sh and
kvm-get-cpus-script.sh scripts to create a TORTURE_AFFINITY environment
variable containing either an empty string (for no affinity) or a list
of CPUs to pin the scenario's vCPUs to. A later commit will make
use of this information to actually pin the vCPUs.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit is a first step towards pinning guest-OS vCPUs so as
to force latency differences, especially on multi-socket systems.
The kvm.sh script puts its batch-creation awk script into a temporary
file so that later commits can add the awk code needed to dole out CPUs
so as to maximize latency differences. This awk code will be used by
multiple scripts.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The last line of kvm-test-1-run-qemu.sh invokes parse-console.sh, but
kvm-test-1-run-qemu.sh is unaware of the PATH containing this script
and does not have the job title handy. This commit therefore moves
the invocation of parse-console.sh to kvm-test-1-run.sh, which has
PATH and title at hand. This commit does not add an invocation of
parse-console.sh to kvm-test-1-run-batch.sh because this latter script
is run in the background, and the information will be gathered at the
end of the full run.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, kvm-recheck.sh attempts to create a kcsan.sum file even for
build-only runs. This results in false-positive bash errors due to
there being no console.log files in that case. This commit therefore
makes kvm-recheck.sh skip creating the kcsan.sum file for build-only runs.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The kvm-remote.sh script places the datestamped directory containing
all the build artifacts in the destination systems' /tmp directories,
where they accumulate runtime artifacts such as console.log. This works,
but some systems have a habit of removing files in /tmp that have not
been recently accessed. This commit therefore runs a simple script that
periodically accesses all files in the datestamped directory.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The qemu-cmd file can contain comments that are not relevant to the
operation of kvm-recheck-lock.sh. This commit therefore strips these
comments before looking for timing information.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The qemu-cmd file can contain comments that are not relevant to the
operation of kvm-recheck-scf.sh. This commit therefore strips these
comments before looking for timing information.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, each -kcsan run in a torture.sh group of runs has its own
kcsan.sum summary. This works, but there is usually a lot of duplication
between the runs. This commit therefore also creates an overall kcsan.sum
file for the entire torture.sh run, if there was at least one -kcsan run.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The kcsan-collapse.sh script assumes that it is being run over the output
of a single kvm.sh run, which is less than helpful for torture.sh runs.
This commit therefore changes the kcsan-collapse.sh script's "ls" pattern
with a "find" command to enable a KCSAN summary across all the -kcsan
runs in a full torture.sh run.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, torture.sh accepts --doall on the one hand and --do-none
on the other, which is a bit inconsistent. This commit therefore adds
--do-all and --donone so that a fully consistent test may be used.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds three short tests of the clocksource-watchdog capability
to the torture.sh script, all to avoid otherwise-inevitable bitrot.
While in the area, fix an obsolete comment.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Fix ~12 single-word typos in RCU code comments.
[ paulmck: Apply feedback from Randy Dunlap. ]
Reviewed-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, if a torture scenario requires more CPUs than are present
on the build system, kvm.sh and friends limit the CPUs available to
that scenario. This makes total sense when the build system and the
system running the scenarios are one and the same, but not so much when
remote systems might well have more CPUs.
This commit therefore introduces a --remote flag to kvm.sh that suppresses
this CPU-limiting behavior, and causes kvm-remote.sh to use this flag.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
In a long-duration kvm-remote.sh run, almost all of the remote accesses will
be simple file-existence checks. These are thus the most likely to be caught
out by network failures, which do happen from time to time.
This commit therefore takes a first step towards tolerating temporary
network outages by making the file-existence checks repeat in the face of
such an outage. They also print a message every minute during a outage,
allowing the user to take appropriate action.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds the BUSTED-BOOST rcutorture scenario, which can be
used to test rcutorture's ability to test RCU priority boosting.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Some of the code invoked directly and indirectly from kvm.sh parses
the output of commands. This parsing assumes English, which can cause
failures if the user has set some other language. In a few cases,
there are language-independent commands available, but this is not
always the case. Therefore, as an alternative to polyglot parsing,
this commit sets the LANG environment variable to en_US.UTF-8.
Reported-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Grepping for "CPU" on lscpu output isn't always successful, depending
on the local language setting. As a result, the build can be aborted
early with:
"make: the '-j' option requires a positive integer argument"
This commit therefore uses the human-language-independent approach
available via the getconf command, both in kvm-build.sh and in
kvm-remote.sh.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, kvm-find-errors.sh assumes that if "--buildonly" appears in
the log file, then the run did builds but ran no kernels. This breaks
with kvm-remote.sh, which uses kvm.sh to do a build, then kvm-again.sh
to run the kernels built on remote systems. This commit therefore adds
a check for a kvm-remote.sh run.
While in the area, this commit checks for "--build-only" as well as
"--build-only".
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Given remote rcutorture runs, it is quite possible that the build system
will have fewer CPUs than the system(s) running the actual test scenarios.
In such cases, using the number of CPUs on the test systems can overload
the build system, slowing down the build or, worse, OOMing the build
system. This commit therefore uses the build system's CPU count to set
N in "make -jN", and by tradition sets "N" to double the CPU count.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit reduces duplicate code by making kvm.sh use the new
kvm-end-run-stats.sh script rather than taking its historical approach
of open-coding it.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit abstractst the end-of-run summary from kvm-again.sh, and,
while in the area, brings its format into line with that of kvm.sh.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The kvm-again.sh script relies on shell comments added to the qemu-cmd
file, but this means that code extracting values from the QEMU command in
this file must grep out those commment. Which kvm-recheck-rcu.sh failed
to do, which destroyed its grace-period-per-second calculation. This
commit therefore adds the needed "grep -v '^#'" to kvm-recheck-rcu.sh.
Fixes: 315957cad4 ("torture: Prepare for splitting qemu execution from kvm-test-1-run.sh")
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds a kvm-remote.sh script that prepares a tarball that
is then downloaded to the remote system(s) and executed. The user is
responsible for having set up the remote systems to run qemu, but all the
kernel builds are done on the system running the kvm-remote.sh script.
The user is also responsible for setting up the remote systems so that
ssh can be run non-interactively, given that ssh is used to poll the
remote systems in order to detect completion of each batch.
See the script's header comment for usage information.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
It is no longer possible to disable CPU hotplug in many configurations,
which means that the CONFIG_HOTPLUG_CPU=n lines in rcuscale's Kconfig
options are just a source of useless diagnostics. In addition, rcuscale
doesn't do CPU-hotplug operations in any case. This commit therefore
changes these lines to read CONFIG_HOTPLUG_CPU=y.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
It is no longer possible to disable CPU hotplug in many configurations,
which means that the CONFIG_HOTPLUG_CPU=n lines in refscale's Kconfig
options are just a source of useless diagnostics. In addition, refscale
doesn't do CPU-hotplug operations in any case. This commit therefore
changes these lines to read CONFIG_HOTPLUG_CPU=y.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit saves a few lines of code by making kvm-again.sh use the
"scenarios" file rather than the "batches" file, both of which are
generated by kvm.sh.
This results in a break point because new versions of kvm-again.sh cannot
handle "res" directories produced by old versions of kvm.sh, which lack
the "scenarios" file. In the unlikely event that this becomes a problem,
a trivial script suffices to convert the "batches" file to a "scenarios"
file, and this script may be easily extracted from kvm.sh.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds "--dryrun scenarios" to kvm.sh, which prints something
like this:
1. TREE03
2. TREE07
3. SRCU-P SRCU-N
4. TREE01 TRACE01
5. TREE02 TRACE02
6. TREE04 RUDE01 TASKS01
7. TREE05 TASKS03 SRCU-T SRCU-U
8. TASKS02 TINY01 TINY02 TREE09
This format is more convenient for scripts that run batches of scenarios.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Although "eval" was removed from torture.sh, that commit failed to
update the KCSAN instance of $* to "$@". This results in failures when
(for example) --bootargs is given more than one argument. This commit
therefore makes this change.
There is one remaining instance of $* in torture.sh, but this
is used only in the "echo" command, where quoting doesn't matter
so much.
Fixes: 197220d4a3 ("torture: Remove use of "eval" in torture.sh")
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Some versions of grep are happy to interpret a nonsensically placed "-"
within a "[]" pattern as a dash, while others give an error message.
This commit therefore places the "-" at the end of the expression where
it was supposed to be in the first place.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, kvm-again.sh updates the duration in the "seconds=" comment
in the qemu-cmd file, but kvm-transform.sh updates the duration in the
actual qemu command arguments. This is an accident waiting to happen.
This commit therefore consolidates these updates into kvm-transform.sh.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The kvm-again.sh script does not copy over the vmlinux files due to
their large size. This means that a gdb run must use the vmlinux file
from the original "res" directory. This commit therefore finds that
directory and prints it out so that the user can copy and pasted the
gdb command just as for the initial run.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Because the TORTURE_TRUST_MAKE environment variable is not recorded,
kvm-again.sh runs can result in the parse-build.sh script emitting
false-positive "BUG: TREE03 no build" messages. These messages are
intended to complain about any lack of compiler invocations when the
--trust-make flag is not given to kvm.sh. However, when this flag is
given to kvm.sh (and thus when TORTURE_TRUST_MAKE=y), lack of compiler
invocations is expected behavior when rebuilding from identical source
code.
This commit therefore makes kvm-test-1-run.sh record the value of the
TORTURE_TRUST_MAKE environment variable as an additional comment in the
qemu-cmd file, and also makes kvm-again.sh reconstitute that variable
from that comment.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
When rerunning an old run using kvm-again.sh, the jitter commands
will re-use the original "res" directory. This works, but is clearly
an accident waiting to happen. And this accident will happen with
remote runs, where the original directory lives on some other system.
This commit therefore updates the qemu-cmd commands to use the new res
directory created for this specific run.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds a --duration argument to kvm-again.sh to allow the user
to override the --duration specified for the original kvm.sh run.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds a kvm-again.sh script that, given the results directory
of a torture-test run, re-runs that test. This means that the kernels
need not be rebuilt, but it also is a step towards running torture tests
on remote systems.
This commit also adds a kvm-test-1-run-batch.sh script that runs one
batch out of the torture test. The idea is to copy a results directory
tree to remote systems, then use kvm-test-1-run-batch.sh to run batches
on these systems.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit creates a "batches" file in the res/$ds directory, where $ds
is the datestamp. This file contains the batches and the number of CPUs,
for example:
1 TREE03 16
1 SRCU-P 8
2 TREE07 16
2 TREE01 8
3 TREE02 8
3 TREE04 8
3 TREE05 8
4 SRCU-N 4
4 TRACE01 4
4 TRACE02 4
4 RUDE01 2
4 RUDE01.2 2
4 TASKS01 2
4 TASKS03 2
4 SRCU-t 1
4 SRCU-u 1
4 TASKS02 1
4 TINY01 1
5 TINY02 1
5 TREE09 1
The first column is the batch number, the second the scenario number
(possibly suffixed by a repetition number, as in "RUDE01.2"), and the
third is the number of CPUs required by that scenario. The last line
shows the number of CPUs expected by this batch file, which allows
the run to be re-batched if a different number of CPUs is available.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Although it might be unlikely that someone would name a scenario
"TORTURE_SUITE", they are within their rights to do so. This script
therefore renames the "TORTURE_SUITE" file in the top-level date-stamped
directory within "res" to "torture_suite" to avoid this name collision.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit enforces the defacto restriction on scenario names, which is
that they contain neither "/", ".", nor lowercase alphabetic characters.
This restriction avoids collisions between scenario names and the torture
scripting's files and directories.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The convention that scenario names are all uppercase has two exceptions,
SRCU-t and SRCU-u. This commit therefore renames them to SRCU-T and
SRCU-U, respectively, to bring them in line with this convention. This in
turn permits tighter argument checking in the torture-test scripting.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The cpus2use.sh script complains if the mpstat command is not available,
and instead uses all available CPUs. Unfortunately, this complaint
goes to stdout, where it confuses invokers who expect a single number.
This commit removes this error message in order to avoid this confusion.
The tendency of late has been to give rcutorture a full system, so this
should not cause issues.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit records the process IDs of the kvm-test-1-run.sh and
kvm-test-1-run-qemu.sh scripts to ease monitoring of remotely running
instances of these scripts.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Distributed runs of rcutorture will need to start and stop jittering on
the remote hosts, which means that the commands must be communicated to
those hosts. The commit therefore causes kvm.sh to place these commands
in new TORTURE_JITTER_START and TORTURE_JITTER_STOP environment variables
to communicate them to the scripts that will set this up. In addition,
this commit causes kvm-test-1-run.sh to append these commands to each
generated qemu-cmd file, which allows any remotely executing script to
extract the needed commands from this file.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, kvm-test-1-run.sh both builds and runs an rcutorture kernel,
which is inconvenient when it is necessary to re-run an old run or to
carry out a run on a remote system. This commit therefore extracts the
portion of kvm-test-1-run.sh that invoke qemu to actually run rcutorture
and places it in kvm-test-1-run-qemu.sh.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
When re-running old rcutorture builds, if the original run involved
gdb, the re-run also needs to do so. This commit therefore records the
TORTURE_KCONFIG_GDB_ARG environment variable into the qemu-cmd file so
that the re-run can access it.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit creates jitterstart.sh and jitterstop.sh scripts that handle
the starting and stopping of the jitter.sh scripts. These must be sourced
using the bash "." command to allow the generated script to wait on the
backgrounded jitter.sh scripts.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Remote rcutorture testing requires that jitter.sh continue to be
invoked from the generated script for local runs, but that it instead
be invoked on the remote system for distributed runs. This argues
for common jitterstart and jitterstop scripts. But it would be good
for jitterstart and jitterstop to control the name and location of the
"jittering" file, while continuing to have the duration controlled by
the caller of these new scripts.
This commit therefore reverses the order of the jittering and duration
parameters for jitter.sh, so that the jittering parameter precedes the
duration parameter.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Now that there is a reliable way to convince the jitter.sh scripts to
stop, the jitter_pids file is not needed, nor is the code that kills all
the PIDs contained in this file. This commit therefore eliminates this
file and the code using it.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, jitter.sh execution is controlled by a time limit and by the
"kill" command. The former allowed jitter.sh to run uselessly past
the end of a set of runs that panicked during boot, and the latter is
vulnerable to PID reuse. This commit therefore introduces a "jittering"
file in the date-stamp directory within "res" that must be present for
the jitter.sh scripts to continue executing. The time limit is still
in place in order to avoid disturbing runs featuring large trace dumps,
but the removal of the "jittering" file handles the panic-during-boot
scenario without relying on PIDs.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, the script generated by kvm.sh does a "wait" to wait on both
the current batch's guest OSes and any jitter.sh scripts. This works,
but makes it hard to abstract the jittering so that common code can be
used for both local and distributed runs. This commit therefore uses
"build.run" files in scenario directories, and these files are removed
after the corresponding scenario's guest OS has completed.
Note that --build-only runs do not create build.run files because they
also do not create guest OSes and do not run any jitter.sh scripts.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently the bN.ready and bN.wait files are placed in the
rcutorture directory, which really is not at all a good place
for run-specific files. This commit therefore renames these
files to build.ready and build.wait and then moves them into the
scenario directories within the "res" directory, for example, into
tools/testing/selftests/rcutorture/res/2021.02.10-15.08.23/TINY01.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Given large numbers of threads, the quantity of torture-test output is
sufficient to sometimes result in RCU CPU stall warnings. The probability
of these stall warnings was greatly reduced by batching the output,
but the warnings were not eliminated. However, the actual test only
depends on console output that is printed even when refscale.verbose=0.
This commit therefore causes this test to run with refscale.verbose=0.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Given large numbers of threads, the quantity of torture-test output is
sufficient to sometimes result in RCU CPU stall warnings. The probability
of these stall warnings was greatly reduced by batching the output,
but the warnings were not eliminated. However, the actual test only
depends on console output that is printed even when rcuscale.verbose=0.
This commit therefore causes this test to run with rcuscale.verbose=0.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The testid.txt file was intended for occasional in extremis use, but
now that the new "bare-metal" file references it, it might see more use.
This commit therefore labels sections of output and adds spacing to make
it easier to see what needs to be done to make a bare-metal build tree
match an rcutorture build tree.
Of course, you can avoid this whole issue by building your bare-metal
kernel in the same directory in which you ran rcutorture, but that might
not always be an option.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
In some environments, the torture-testing use of virtualization is
inconvenient. In such cases, the modprobe and rmmod commands may be used
to do torture testing, but significant setup is required to build, boot,
and modprobe a kernel so as to match a given torture-test scenario.
This commit therefore creates a "bare-metal" file in each results
directory containing steps to run the corresponding scenario using the
modprobe command on bare metal. For example, the contents of this file
after using kvm.sh to build an rcutorture TREE01 kernel, perhaps with
the --buildonly argument, is as follows:
To run this scenario on bare metal:
1. Set your bare-metal build tree to the state shown in this file:
/home/git/linux-rcu/tools/testing/selftests/rcutorture/res/2021.02.04-17.10.19/testid.txt
2. Update your bare-metal build tree's .config based on this file:
/home/git/linux-rcu/tools/testing/selftests/rcutorture/res/2021.02.04-17.10.19/TREE01/ConfigFragment
3. Make the bare-metal kernel's build system aware of your .config updates:
$ yes "" | make oldconfig
4. Build your bare-metal kernel.
5. Boot your bare-metal kernel with the following parameters:
maxcpus=8 nr_cpus=43 rcutree.gp_preinit_delay=3 rcutree.gp_init_delay=3 rcutree.gp_cleanup_delay=3 rcu_nocbs=0-1,3-7
6. Start the test with the following command:
$ modprobe rcutorture nocbs_nthreads=8 nocbs_toggle=1000 fwd_progress=0 onoff_interval=1000 onoff_holdoff=30 n_barrier_cbs=4 stat_interval=15 shutdown_secs=120 test_no_idle_hz=1 verbose=1
7. After some time, end the test with the following command:
$ rmmod rcutorture
8. Copy your bare-metal kernel's .config file, overwriting this file:
/home/git/linux-rcu/tools/testing/selftests/rcutorture/res/2021.02.04-17.10.19/TREE01/.config
9. Copy the console output from just before the modprobe to just after
the rmmod into this file:
/home/git/linux-rcu/tools/testing/selftests/rcutorture/res/2021.02.04-17.10.19/TREE01/console.log
10. Check for runtime errors using the following command:
$ tools/testing/selftests/rcutorture/bin/kvm-recheck.sh /home/git/linux-rcu/tools/testing/selftests/rcutorture/res/2021.02.04-17.10.19
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Yes, I do recall a time when 512MB of memory was a lot of mass storage,
much less main memory, but the rcuscale kvfree_rcu() testing invoked by
torture.sh can sometimes exceed it on large systems, resulting in OOM.
This commit therefore causes torture.sh to pase the "--memory 1G"
argument to kvm.sh to reserve a full gigabyte for this purpose.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
If the build fails when running multiple instances of a given rcutorture
scenario, for example, using the kvm.sh --configs "8*RUDE01" argument,
the build will be rerun an additional seven times. This is in some sense
correct, but it can waste significant time. This commit therefore checks
for a prior failed build and simply copies over that build's output.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The current jitter.sh script expects cpumask bits to fit into whatever
the awk interpreter uses for an integer, which clearly does not hold for
even medium-sized systems these days. This means that on a large system,
only the first 32 or 64 CPUs (depending) are subjected to jitter.sh
CPU-time perturbations. This commit therefore computes a given CPU's
cpumask using text manipulation rather than arithmetic shifts.
Reported-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
TREE03 tests RCU priority boosting, which is a real-time feature.
It would also be good if it tested something closer to what is
actually used by the real-time folks. This commit therefore adds
tree.use_softirq=0 to the TREE03 kernel boot parameters in TREE03.boot.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit uses the shiny new "all" and "N" cpumask options to decouple
the "nohz_full" and "rcu_nocbs" kernel boot parameters in the TREE04.boot
and TREE08.boot files from the CONFIG_NR_CPUS options in the TREE04 and
TREE08 files.
Reported-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The -lgcc command-line argument is placed poorly in the build options,
which can result in build failures, for exapmle, on ARM when uidiv()
is required. This commit therefore places the -lgcc argument after the
source files.
Fixes: b94ec36896 ("rcutorture: Make use of nolibc when available")
Tested-by: Valentin Schneider <valentin.schneider@arm.com>
Tested-by: Mark Rutland <mark.rutland@arm.com> [arm64]
Signed-off-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
RCU's rcutree.use_softirq=0 kernel boot parameter substitutes the per-CPU
rcuc kthreads for softirq, which is used in real-time installations.
However, none of the rcutorture scenarios test this parameter.
This commit therefore adds rcutree.use_softirq=0 to the RUDE01 and
TASKS01 rcutorture scenarios, both of which indirectly exercise RCU.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The sizes of vmlinux files built with KASAN enabled can approach a full
gigabyte, which can result in disk overflow sooner rather than later.
Fortunately, the xz command compresses them by almost an order of
magnitude. This commit therefore uses xz to compress vmlinux file built
by torture.sh with KASAN enabled.
However, xz is not the fastest thing in the world. In fact, it is way
slower than rotating-rust mass storage. This commit therefore also adds a
--compress-kasan-vmlinux argument to specify the degree of xz concurrency,
which defaults to using all available CPUs if there are that many files in
need of compression.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
In 2020, running KCSAN often requires careful choice of compiler.
This commit therefore adds a --kcsan-kmake-arg parameter to torture.sh
to allow specifying (for example) "CC=clang" to the kernel build process
to correctly build a KCSAN-enabled kernel.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds the command and arguments to the torture.sh log file, and
also outputs the results directory. This latter allows impatient users
to quickly find the results that are being generated by the current run.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds --configs-rcutorture, --configs-locktorture, and
--configs-scftorture arguments to torture.sh, allowing the desired
set of scenarios to be passed to each. The default for each has been
changed from a large-system-appropriate set to just CFLIST for each.
Users are encouraged to create scripts that provide appropriate settings
for their specific systems.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Now that kvm.sh puts all the relevant details in the "log" file,
there is no need for torture.sh to generate a separate "log.long"
file. This commit therefore drops this from torture.sh.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit causes torture.sh to check for zero-length runs and to take
the cowardly option of refusing to run them, logging its cowardice for
later inspection.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit causes torture.sh to use the torture.verbose_sleep_frequency
kernel boot parameter to throttle verbose refscale output on large systems.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
