qemu/tests
Max Reitz 59aec86982 iotests/297: Rewrite in Python and extend reach
Instead of checking iotests.py only, check all Python files in the
qemu-iotests/ directory.  Of course, most of them do not pass, so there
is an extensive skip list for now.  (The only files that do pass are
209, 254, 283, and iotests.py.)

(Alternatively, we could have the opposite, i.e. an explicit list of
files that we do want to check, but I think it is better to check files
by default.)

Unless started in debug mode (./check -d), the output has no information
on which files are tested, so we will not have a problem e.g. with
backports, where some files may be missing when compared to upstream.

Besides the technical rewrite, some more things are changed:

- For the pylint invocation, PYTHONPATH is adjusted.  This mirrors
  setting MYPYPATH for mypy.

- Also, MYPYPATH is now derived from PYTHONPATH, so that we include
  paths set by the environment.  Maybe at some point we want to let the
  check script add '../../python/' to PYTHONPATH so that iotests.py does
  not need to do that.

- Passing --notes=FIXME,XXX to pylint suppresses warnings for TODO
  comments.  TODO is fine, we do not need 297 to complain about such
  comments.

- The "Success" line from mypy's output is suppressed, because (A) it
  does not add useful information, and (B) it would leak information
  about the files having been tested to the reference output, which we
  decidedly do not want.

Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210118105720.14824-3-mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2021-01-26 14:36:37 +01:00
..
acceptance acceptance: switch to QMP change-vnc-password command 2021-01-23 15:55:07 -05:00
data acpi: Update _DSM method in expected files 2021-01-17 06:42:54 -05:00
decode decodetree: Improve identifier matching 2020-09-14 14:23:19 +01:00
docker tests/docker: Add dockerfile for Alpine Linux 2021-01-20 10:46:54 +01:00
fp tests/fp: Do not emit implicit-fallthrough warnings in the softfloat tests 2020-12-18 09:14:23 +01:00
guest-debug gdbstub: implement a softmmu based test 2021-01-18 10:04:31 +00:00
image-fuzzer
include tests: add missing generated sources to testqapi 2020-10-17 10:45:50 -04:00
keys tests/vm: Add Haiku test based on their vagrant images 2020-11-17 09:45:24 +01:00
migration tests: Rename PAGE_SIZE definitions 2021-01-20 10:46:54 +01:00
multiboot Remove superfluous .gitignore files 2020-10-13 12:48:17 +02:00
perf/block/qcow2
plugin plugin: Fixes compiling errors on msys2/mingw 2020-10-09 17:27:55 +01:00
qapi-schema meson: move SPHINX_ARGS references within "if build_docs" 2020-10-17 10:45:52 -04:00
qemu-iotests iotests/297: Rewrite in Python and extend reach 2021-01-26 14:36:37 +01:00
qtest tests: Fix memory leak in tpm-util.c 2021-01-20 10:46:54 +01:00
rocker
tcg tests/tcg/s390x: Fix EXRL tests 2021-01-21 11:19:45 +01:00
tsan tests/docker: Added docker build support for TSan. 2020-06-16 14:49:05 +01:00
uefi-test-tools Remove superfluous .gitignore files 2020-10-13 12:48:17 +02:00
vm tests/docker, tests/vm: remove setuptools from images 2020-11-25 12:50:48 -05:00
vmstate-static-checker-data
atomic64-bench.c qemu/atomic.h: rename atomic_ to qatomic_ 2020-09-23 16:07:44 +01:00
atomic_add-bench.c qemu/atomic.h: rename atomic_ to qatomic_ 2020-09-23 16:07:44 +01:00
benchmark-crypto-cipher.c tests: fix output message formatting for crypto benchmarks 2020-09-10 11:02:17 +01:00
benchmark-crypto-hash.c tests: fix output message formatting for crypto benchmarks 2020-09-10 11:02:17 +01:00
benchmark-crypto-hmac.c tests: fix output message formatting for crypto benchmarks 2020-09-10 11:02:17 +01:00
check-block-qdict.c tests: Use error_free_or_abort() where appropriate 2020-07-02 06:25:28 +02:00
check-block.sh tests/check-block.sh: Refuse to run the iotests with BusyBox' sed 2021-01-20 10:46:54 +01:00
check-qdict.c
check-qjson.c qobject: Drop qstring_get_try_str() 2020-12-19 10:39:16 +01:00
check-qlist.c
check-qlit.c
check-qnull.c
check-qnum.c qobject: Fix qnum_to_string() to use sufficient precision 2020-12-19 10:37:16 +01:00
check-qobject.c qobject: Make QString immutable 2020-12-19 10:39:41 +01:00
check-qom-interface.c Use DECLARE_*CHECKER* macros 2020-09-09 09:27:09 -04:00
check-qom-proplist.c qom: Clean up object_property_get_enum()'s error value 2020-09-18 13:49:48 -04:00
check-qstring.c qobject: Make QString immutable 2020-12-19 10:39:41 +01:00
crypto-tls-psk-helpers.c
crypto-tls-psk-helpers.h
crypto-tls-x509-helpers.c crypto: use a stronger private key for tests 2020-07-17 14:20:57 +02:00
crypto-tls-x509-helpers.h
dbus-vmstate-daemon.sh
io-channel-helpers.c test: Fix LGPL information in the file headers 2020-11-15 17:04:40 +01:00
io-channel-helpers.h test: Fix LGPL information in the file headers 2020-11-15 17:04:40 +01:00
iothread.c qemu/atomic.h: rename atomic_ to qatomic_ 2020-09-23 16:07:44 +01:00
iothread.h
Makefile.include build: replace ninjatool with ninja 2020-10-17 10:45:51 -04:00
meson.build meson: Propagate gnutls dependency 2021-01-12 12:38:03 +01:00
pkix_asn1_tab.c
ptimer-test-stubs.c replay: do not build if TCG is not available 2020-10-22 11:53:54 -04:00
ptimer-test.c
ptimer-test.h
qht-bench.c qemu/atomic.h: rename atomic_ to qatomic_ 2020-09-23 16:07:44 +01:00
rcutorture.c qemu/atomic.h: rename atomic_ to qatomic_ 2020-09-23 16:07:44 +01:00
requirements.txt tests/acceptance: Bump avocado requirements to 83.0 2020-12-18 12:25:42 +01:00
socket-helpers.c tests/socket-helpers: Treat EAI_NONAME as EADDRNOTAVAIL 2020-09-16 08:41:06 +02:00
socket-helpers.h
test-aio-multithread.c qemu/atomic.h: rename atomic_ to qatomic_ 2020-09-23 16:07:44 +01:00
test-aio.c
test-authz-list.c authz: Fix Lesser GPL version number 2020-10-29 09:57:37 +00:00
test-authz-listfile.c authz: Fix Lesser GPL version number 2020-10-29 09:57:37 +00:00
test-authz-pam.c authz: Fix Lesser GPL version number 2020-10-29 09:57:37 +00:00
test-authz-simple.c authz: Fix Lesser GPL version number 2020-10-29 09:57:37 +00:00
test-base64.c test: Fix LGPL information in the file headers 2020-11-15 17:04:40 +01:00
test-bdrv-drain.c block: End quiescent sections when a BDS is deleted 2020-10-27 15:26:20 +01:00
test-bdrv-graph-mod.c tests: Use error_free_or_abort() where appropriate 2020-07-02 06:25:28 +02:00
test-bitcnt.c
test-bitmap.c mingw: fix error __USE_MINGW_ANSI_STDIO redefined 2020-10-13 13:33:46 +02:00
test-bitops.c
test-block-backend.c
test-block-iothread.c tests: Use error_free_or_abort() where appropriate 2020-07-02 06:25:28 +02:00
test-blockjob-txn.c
test-blockjob.c
test-bufferiszero.c test: Fix LGPL information in the file headers 2020-11-15 17:04:40 +01:00
test-char.c tests/test-char.c: Wait for the chardev to connect in char_socket_client_dupid_test 2021-01-13 10:21:17 +01:00
test-clone-visitor.c qapi: Use QAPI_LIST_PREPEND() where possible 2020-12-19 10:20:14 +01:00
test-coroutine.c
test-crypto-afsplit.c crypt: Fix Lesser GPL version number 2020-10-29 09:57:37 +00:00
test-crypto-block.c crypt: Fix Lesser GPL version number 2020-10-29 09:57:37 +00:00
test-crypto-cipher.c crypt: Fix Lesser GPL version number 2020-10-29 09:57:37 +00:00
test-crypto-hash.c crypt: Fix Lesser GPL version number 2020-10-29 09:57:37 +00:00
test-crypto-hmac.c
test-crypto-ivgen.c crypt: Fix Lesser GPL version number 2020-10-29 09:57:37 +00:00
test-crypto-pbkdf.c test: Fix LGPL information in the file headers 2020-11-15 17:04:40 +01:00
test-crypto-secret.c crypt: Fix Lesser GPL version number 2020-10-29 09:57:37 +00:00
test-crypto-tlscredsx509.c
test-crypto-tlssession.c
test-crypto-xts.c crypt: Fix Lesser GPL version number 2020-10-29 09:57:37 +00:00
test-cutils.c
test-fdmon-epoll.c tests: add test-fdmon-epoll 2020-09-23 13:45:52 +01:00
test-hbitmap.c
test-image-locking.c
test-int128.c
test-io-channel-buffer.c io: Fix Lesser GPL version number 2020-10-29 09:57:37 +00:00
test-io-channel-command.c io: Fix Lesser GPL version number 2020-10-29 09:57:37 +00:00
test-io-channel-file.c io: Fix Lesser GPL version number 2020-10-29 09:57:37 +00:00
test-io-channel-socket.c io: Fix Lesser GPL version number 2020-10-29 09:57:37 +00:00
test-io-channel-tls.c
test-io-task.c io: Fix Lesser GPL version number 2020-10-29 09:57:37 +00:00
test-iov.c util/iov: add iov_discard_undo() 2020-09-23 13:41:58 +01:00
test-keyval.c keyval: Parse help options 2020-10-15 16:06:27 +02:00
test-logging.c qemu/atomic.h: rename atomic_ to qatomic_ 2020-09-23 16:07:44 +01:00
test-mul64.c
test-opts-visitor.c
test-qapi-util.c
test-qdev-global-props.c tests: Fixes test-qdev-global-props.c 2020-09-16 12:15:07 +02:00
test-qdist.c
test-qemu-opts.c qemu-option: warn for short-form boolean options 2021-01-23 15:55:08 -05:00
test-qga.c test-qga: fix a resource leak in test_qga_guest_get_osinfo() 2020-12-09 08:04:34 +01:00
test-qgraph.c test: Fix LGPL information in the file headers 2020-11-15 17:04:40 +01:00
test-qht-par.c
test-qht.c
test-qmp-cmds.c qapi: Add a 'coroutine' flag for commands 2020-10-09 07:08:19 +02:00
test-qmp-event.c
test-qobject-input-visitor.c qapi: enable use of g_autoptr with QAPI types 2020-09-03 09:38:36 +02:00
test-qobject-output-visitor.c qapi: Use QAPI_LIST_PREPEND() where possible 2020-12-19 10:20:14 +01:00
test-rcu-list.c qemu/atomic.h: rename atomic_ to qatomic_ 2020-09-23 16:07:44 +01:00
test-rcu-simpleq.c
test-rcu-slist.c
test-rcu-tailq.c
test-replication.c tests: test-replication disable /replication/secondary/* on msys2/mingw. 2020-09-16 08:41:06 +02:00
test-shift128.c
test-string-input-visitor.c tests: Use &error_abort where appropriate 2020-07-02 06:25:28 +02:00
test-string-output-visitor.c string-output-visitor: Fix to use sufficient precision 2020-12-19 10:37:16 +01:00
test-thread-pool.c qemu/atomic.h: rename atomic_ to qatomic_ 2020-09-23 16:07:44 +01:00
test-throttle.c
test-timed-average.c cpu-timers, icount: new modules 2020-10-05 16:41:22 +02:00
test-util-filemonitor.c tests: Fixes building test-util-filemonitor.c on msys2/mingw 2020-09-07 12:34:17 +02:00
test-util-sockets.c sockets: Make abstract UnixSocketAddress depend on CONFIG_LINUX 2020-11-03 13:17:25 +00:00
test-uuid.c test: Fix LGPL information in the file headers 2020-11-15 17:04:40 +01:00
test-visitor-serialization.c qobject: Change qobject_to_json()'s value to GString 2020-12-19 10:38:43 +01:00
test-vmstate.c vmstate: Fixes test-vmstate.c on msys2/mingw 2020-09-16 08:41:06 +02:00
test-write-threshold.c block: introduce BDRV_MAX_LENGTH 2020-12-11 17:52:40 +01:00
test-x86-cpuid.c Revert "hw/i386: Update structures to save the number of nodes per package" 2020-09-02 07:29:26 -04:00
test-xbzrle.c tests: Rename PAGE_SIZE definitions 2021-01-20 10:46:54 +01:00
vhost-user-bridge.c libvhost-user: make it a meson subproject 2020-12-08 13:48:58 -05:00