While QEMUQtestMachine closes the socket that was passed to
QEMUQtestProtocol, the python resource leak manager still
believes that the copy QEMUQtestProtocol holds is open. We
must explicitly call close to avoid this leak warnnig.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
(cherry picked from commit 6ccb48ffc1)
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
This commit is two backports squashed into one to avoid regressions.
python: *really* remove get_event_loop
A prior commit, aa1ff990, switched away from using get_event_loop *by
default*, but this is not good enough to avoid deprecation warnings as
`asyncio.get_event_loop_policy().get_event_loop()` is *also*
deprecated. Replace this mechanism with explicit calls to
asyncio.get_new_loop() and revise the cleanup mechanisms in __del__ to
match.
python: avoid creating additional event loops per thread
"Too hasty by far!", commit 21ce2ee4 attempted to avoid deprecated
behavior altogether by calling new_event_loop() directly if there was no
loop currently running, but this has the unfortunate side effect of
potentially creating multiple event loops per thread if tests
instantiate multiple QMP connections in a single thread. This behavior
is apparently not well-defined and causes problems in some, but not all,
combinations of Python interpreter version and platform environment.
Partially revert to Daniel Berrange's original patch, which calls
get_event_loop and simply suppresses the deprecation warning in
Python<=3.13. This time, however, additionally register new loops
created with new_event_loop() so that future calls to get_event_loop()
will return the loop already created.
Reported-by: Richard W.M. Jones <rjones@redhat.com>
Reported-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@21ce2ee4f2df87efe84a27b9c5112487f4670622
cherry picked from commit python-qemu-qmp@c08fb82b38212956ccffc03fc6d015c3979f42fe
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
(cherry picked from commit 85f223e5b0)
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
This method was deprecated in 3.12 because it ordinarily should not be
used from coroutines; if there is not a currently running event loop,
this automatically creates a new event loop - which is usually not what
you want from code that would ever run in the bottom half.
In our case, we do want this behavior in two places:
(1) The synchronous shim, for convenience: this allows fully sync
programs to use QEMUMonitorProtocol() without needing to set up an event
loop beforehand. This is intentional to fully box in the async
complexities into the legacy sync shim.
(2) The qmp_tui shell; instead of relying on asyncio.run to create and
run an asyncio program, we need to be able to pass the current asyncio
loop to urwid setup functions. For convenience, again, we create one if
one is not present to simplify the creation of the TUI appliance.
The remaining user of get_event_loop() was in fact one of the erroneous
users that should not have been using this function: if there's no
running event loop inside of a coroutine, you're in big trouble :)
Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@aa1ff9907603a3033296027e1bd021133df86ef1
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
(cherry picked from commit 5d99044d09)
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Based on the discussion at https://github.com/pypa/pip/issues/9726 -
even though the setuptools documentation implies that it is possible to
guard script execution with optional dependency groups, this is not true
in practice with the scripts generated by pip.
Just do the simple thing and guard the import statements.
Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@df520dcacf9a75dd4c82ab1129768de4128b554c
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
(cherry picked from commit fd0ed46d4e)
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@9c889dcbd58817b0c917a9d2dd16161f48ac8203
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
(cherry picked from commit fcaeeb7653)
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
This removes a non-idiomatic use of a "coroutine callback" in favor of
something a bit more standardized.
Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@commit 97f7ffa3be17a50544b52767d14b6fd478c07b9e
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
(cherry picked from commit 0408b8d7a0)
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Now that the minimum version is 3.7, drop some of the 3.6-specific hacks
we've been carrying. A single remaining compatibility hack concerning
3.6's lack of @asynccontextmanager is addressed in the following commit.
Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@3e8e34e594cfc6b707e6f67959166acde4b421b8
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
(cherry picked from commit f9d2e0a3bd)
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
This corrects an oversight in qmp-shell operation where new events will
not accumulate in the event queue when pressing "enter" with an empty
command buffer, so no new events show up.
Reported-by: Jag Raman <jag.raman@oracle.com>
Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@0443582d16cf9efd52b2c41a7b5be7af42c856cd
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
(cherry picked from commit 1e343714bf)
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
The new Matcher class does not have a __str__ implementation, and therefore
it prints the debugging representation of the internal object:
$ ../configure --enable-rust && make qemu-system-arm --enable-download
python determined to be '/usr/bin/python3'
python version: Python 3.13.6
mkvenv: Creating non-isolated virtual environment at 'pyvenv'
mkvenv: checking for LegacyMatcher('meson>=1.5.0')
mkvenv: checking for LegacyMatcher('pycotap>=1.1.0')
Add the method to print the nicer
mkvenv: checking for meson>=1.5.0
mkvenv: checking for pycotap>=1.1.0
Cc: qemu-stable@nongnu.org
Cc: John Snow <jsnow@redhat.com>
Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit ab85146ac4)
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Fix compilation with pip-25.2 due to missing distlib.version
Bug: https://gitlab.com/qemu-project/qemu/-/issues/3062
Signed-off-by: Sv. Lockal <lockalsash@gmail.com>
[Edits: Type "safety" whackamole --js]
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250811190159.237321-1-jsnow@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
The way editable installs work has changed at some point since Fedora 40
was released. Generally, we should be opting to use pyproject.toml
installs (PEP517/518) - but those are not fully supported until v61 of
setuptools, and CentOS Stream 9 ships v53.
Until that time, we can make use of a transitional feature in
pip/setuptools to use "legacy" editable installs, which is enough to fix
"make check-dev" on modern local workstations for now.
By using the environment variable approach to configure pip, we avoid
any problems for older versions of pip that don't recognize this option,
so it's harmless. The config-settings option first appeared in v23 of
pip. editable_mode was first supported by setuptools in v64.
(I'm not currently precisely aware of when the default behavior of '-e'
switched away from 'compat', but it appears to be a joint effect between
setuptools and pip versions.)
Version information for supported build platforms:
distro python3 pip setuptools sphinx
--------------------------------------------------------
centos_stream_9 3.9.23 21.3.1 53.0.0 3.4.3
ubuntu_22_04 3.10.12 22.0.2 59.6.0 4.3.2
** pyproject.toml installs supported as of here **
freebsd 3.11.13 23.3.2 63.1.0 5.3.0
debian_12 3.11.2 23.0.1 66.1.1 5.3.0
ubuntu_24_04 3.12.3 24.0 68.1.2 7.2.6
centos_stream_10 3.12.11 23.3.2 69.0.3 7.2.6
fedora_41 3.13.5 24.2 69.2.0 7.3.7
alpine_3_19 3.11.13 23.3.1 70.3.0 6.2.1
alpine_3_20 3.12.11 24.0 70.3.0 7.2.6
alpine_3_21 3.12.11 24.3.1 70.3.0 8.1.3
ubuntu_24_10 3.12.7 24.2 74.1.2 7.4.7
fedora_42 3.13.5 24.3.1 74.1.3 8.1.3
ubuntu_25_04 3.13.3 25.0 75.8.0 8.1.3
macports 3.13.5 25.1.1 78.1.1 8.2.3
openbsd 3.12.11 25.1.1 79.0.1 8.2.3
alpine_3_22 3.12.11 25.1.1 80.9.0 8.2.3
homebrew 3.13.5 --- 80.9.0 8.2.3
pkgsrc_current 3.12.11 25.1.1 80.9.0 8.2.3
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250715222548.198888-1-jsnow@redhat.com>
Tested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Use qom-list-get to speed up the qom-tree command.
Signed-off-by: Steve Sistare <steven.sistare@oracle.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <1752248703-217318-3-git-send-email-steven.sistare@oracle.com>
Tested-by: Markus Armbruster <armbru@redhat.com>
[Lint picked off to mollify make check-minreqs]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Thanks for your service!
Remove the old qapidoc and the option to enable the transmogrifier,
leaving the "transmogrifier" as the ONLY qapi doc generator. This in
effect also converts the QAPI test to use the new documentation
generator, too.
Update doc-good.txt output to match the new doc generator, which I
should've done exactly when we switched over to the transmogrifier, but,
uhh, oops!
Notes on the new format:
1. per-member IFCOND documentation is missing. Known issue.
2. Freeform documentation without a header is now copied through into
the output. This is a bug fix.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250618165353.1980365-4-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Fixes: b61a4eb3f3 (docs/qapidoc: support header-less freeform sections)
[Tweak commit message to say it's a bug fix, add Fixes:]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id: 20250623121845.7214-26-philmd@linaro.org
[PMM: tweaks to satisfy the python linter CI job]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Update the python tests to also check QAPI and the QAPI Sphinx
extensions. The docs/sphinx/qapidoc_legacy.py file is not included in
these checks, as it is destined for removal soon. mypy is also not
called on the QAPI Sphinx extensions, owing to difficulties supporting
Sphinx 3.x - 8.x while maintaining static type checking support. mypy
*is* called on all of the QAPI tools themselves, though.
flake8, isort and mypy use the tool configuration from the existing
python directory (in setup.cfg). pylint continues to use the special
configuration located in scripts/qapi/ - that configuration is more
permissive. If we wish to unify the two configurations, that's a
separate series and a discussion for a later date.
The list of pylint ignores is also updated, owing again to the wide
window of pylint version support: newer versions require pragmas to
occasionally silence the "too many positional arguments" warning, but
older versions do not have such a warning category and will instead yelp
about an unrecognized option. Silence that warning, too.
As a result of this patch, one would be able to run any of the following
tests locally from the qemu.git/python directory and have it cover the
QAPI tooling as well. All of the following options run the python tests,
static analysis tests, and linter checks; but with different
combinations of dependencies and interpreters.
- "make check-minreqs" Run tests specifically under our oldest supported
Python and our oldest supported dependencies. This is the test that
runs on GitLab as "check-python-minreqs". This helps ensure we do not
regress support on older platforms accidentally.
- "make check-tox" Runs the tests under the newest supported
dependencies, but under each supported version of Python in turn. At
time of writing, this is Python 3.8 to 3.13 inclusive. This test helps
catch bleeding-edge problems before they become problems for developer
workstations. This is the GitLab test "check-python-tox" and is an
optionally run, may-fail test due to the unpredictable nature of new
dependencies being released into the ecosystem that may cause
regressions.
- "make check-dev" Runs the tests under the newest supported
dependencies using whatever version of Python the user happens to have
installed. This is a quick convenience check that does not map to any
particular GitLab test.
(Note! check-dev may be busted on Fedora 41 and bleeding edge versions
of setuptools. That's unrelated to this patch and I'll address it
separately and soon. Thank you for your patience, --mgmt)
Finally, finally, finally: this means that QAPI tooling will be linted
and type-checked from the GitLab pipelines.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20250604200354.459501-5-jsnow@redhat.com
[Edited license choice per review --js]
Signed-off-by: John Snow <jsnow@redhat.com>
We pin all dependencies for the "check-minreqs" test because pip lacks a
dependency resolver that installs "the oldest possible package that
meets dependency criteria". So, in order to test our stated minimum
requirements, we pin all of our dependencies (and their dependencies,
transitively) at the oldest possible versions that still work and pass
tests; proving that our minimum requirements are correct.
(It also ensures no new features accidentally sneak in from developers
on newer platforms.)
A few transitive dependencies were omitted from the pinned dependency
file by accident; as a result, pip's dependency solver can pull in newer
dependencies, which we don't want. This patch corrects the previous
oversight and pins the missing dependencies.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20250604200354.459501-4-jsnow@redhat.com
This adds several improvements to Rust support, including
native clippy and rustdoc targets, the "objects" keyword,
and running doctests.
Require it only when Rust support is requested, to avoid
putting a strict requirement on all build platforms for the
sake of an experimental feature.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Python 3.8 went "end of life" in October 2024 and Fedora 42 dropped
this version already, so the "python" CI job is currently failing.
Thus it's time to drop support for this Python version in QEMU, too.
While we're at it, also look for "python3.13" in the configure script.
Message-ID: <20250425120710.879518-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
As of (at least) pylint 3.3.1, this code trips pylint up into believing
we are raising something other than an Exception. We are not: the first
two values may indeed be "None", but the last and final value must by
definition be a SystemExit exception.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20241101173700.965776-5-jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Newest versions of pylint complain about specifically positional
arguments in addition to too many in general. We already disable the
general case, so silence this new warning too.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20241101173700.965776-4-jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
The upcoming functional tests will require pycotap for providing
TAP output from the python-based tests. Since we want to be able
to run some of the tests offline by default, too, let's install
it along with meson in our venv if necessary (it's size is only
5 kB, so adding the wheel here should not really be a problem).
The wheel file has been obtained with:
pip download --only-binary :all: --dest . --no-cache pycotap
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20240830133841.142644-8-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Python 3.13 is in beta and Fedora 41 is preparing to make it the default
system interpreter; enable testing for it.
(In the event problems develop prior to release, it should only impact
the check-python-tox job, which is not run by default and is allowed to
fail.)
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 20240626232230.408004-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
New bleeding edge versions, new nits to iron out. This addresses the
'check-python-tox' optional GitLab test, while 'check-python-minreqs'
saw no regressions, since it's frozen on an older version of pylint.
Fixes:
qemu/machine/machine.py:345:52: E0606: Possibly using variable 'sock' before assignment (possibly-used-before-assignment)
qemu/utils/qemu_ga_client.py:168:4: R1711: Useless return at end of function or method (useless-return)
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 20240626232230.408004-2-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
This was used to bootstrap the venv with a TOML parser, after which
ensuregroup is used. Now that we expect it to be present as a system
package (either tomli or, for Python 3.11, tomllib), it is not needed
anymore.
Note that this means that, when implemented, the hypothetical "isolated"
mode that does not use any system packages will only work with Python
3.11+.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Now that Ubuntu 20.04 is not included anymore, there is no need to ship
it as part of QEMU; Ubuntu 22.04 includes it and Leap users anyway
need to install all the required dependencies from PyPI.
This mostly reverts commit ec77ee7634de123b7c899739711000fd21dab68b,
with just some changes to the wording.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Add a supportive property to access the path to the QEMU binary
Signed-off-by: Maksim Davydov <davydov-max@yandex-team.ru>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240318213550.155573-4-davydov-max@yandex-team.ru>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
We need meson v1.2.3 to build QEMU on macOS Sonoma. It
also builds fine all our CI jobs (as tested by also bumping
"accepted" in pythondeps.toml), so let's use it as our
"good enough" packaged wheel.
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1939
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20231109160504.93677-2-philmd@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
The method is not popular in iotests, we prefer use vm.qmp() and then
check success by hand. But that's not optimal. To simplify movement to
vm.cmd() let's support same interface improvements like in vm.qmp().
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20231006154125.1068348-7-vsementsov@yandex-team.ru
Signed-off-by: John Snow <jsnow@redhat.com>
Use a shorter name. We are going to move in iotests from qmp() to
command() where possible. But command() is longer than qmp() and don't
look better. Let's rename.
You can simply grep for '\.command(' and for 'def command(' to check
that everything is updated (command() in tests/docker/docker.py is
unrelated).
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-id: 20231006154125.1068348-6-vsementsov@yandex-team.ru
[vsementsov: also update three occurrences in
tests/avocado/machine_aspeed.py and keep r-b]
Signed-off-by: John Snow <jsnow@redhat.com>
Having cmd() and command() methods in one class doesn't look good.
Rename cmd() to cmd_raw(), to show its meaning better.
We also want to rename command() to cmd() in future, so this commit is
a necessary step.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20231006154125.1068348-5-vsementsov@yandex-team.ru
Signed-off-by: John Snow <jsnow@redhat.com>
We just want to ignore failure, so we don't need low level .cmd(). This
helps further renaming .command() to .cmd().
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20231006154125.1068348-3-vsementsov@yandex-team.ru
Signed-off-by: John Snow <jsnow@redhat.com>
The argument is unused, let's drop it for now, as we are going to
refactor the interface and don't want to refactor unused things.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20231006154125.1068348-2-vsementsov@yandex-team.ru
Signed-off-by: John Snow <jsnow@redhat.com>
Python 3.12 has released, so update the test infrastructure to test
against this version. Update the configure script to look for it when an
explicit Python interpreter isn't chosen.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-id: 20231006195243.3131140-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
This patch is a backport from
e03a3334b6
According to Guido in https://github.com/python/cpython/issues/104344 ,
this call was never meant to wait for the server to shut down - that is
handled synchronously - but instead, this waits for all connections to
close. Or, it would have, if it wasn't broken since it was introduced.
3.12 fixes the bug, which now causes a hang in our code. The fix is just
to remove the wait.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-id: 20231006195243.3131140-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
By using a socketpair for all of the sockets managed by the VM class and
its extensions, we don't need the sock_dir argument anymore, so remove
it.
We only added this argument so that we could specify a second, shorter
temporary directory for cases where the temp/log dirs were "too long" as
a socket name on macOS. We don't need it for this class now. In one
case, avocado testing takes over responsibility for creating an
appropriate sockdir.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20230928044943.849073-7-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Like the QMP and console sockets, begin using socketpairs for the qtest
connection, too. After this patch, we'll be able to remove the vestigial
sock_dir argument, but that cleanup is best done in its own patch.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20230928044943.849073-6-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Create a socketpair for the console output. This should help eliminate
race conditions around console text early in the boot process that might
otherwise have been dropped on the floor before being able to connect to
QEMU under "server,nowait".
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Ani Sinha <anisinha@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20230928044943.849073-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Useful if we want to use ConsoleSocket() for a socket created by
socketpair().
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Ani Sinha <anisinha@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20230928044943.849073-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
If everything has gone smoothly, we'll already have closed the socket we
gave to the child during post_launch. The other half of the pair that we
gave to the QMP connection should, likewise, be definitively closed by
now.
However, in the cleanup path, it's possible we've created the socketpair
but flubbed the launch and need to clean up resources. These resources
*would* be handled by the garbage collector, but that can happen at
unpredictable times. Nicer to just clean them up synchronously on the
exit path, here.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Ani Sinha <anisinha@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20230928044943.849073-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
This property isn't meant to do much else besides return a list of
strings, so move this setup back out into _pre_launch().
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Ani Sinha <anisinha@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20230928044943.849073-2-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Occasionally some avocado tests will fail waiting for console line
despite the machine running correctly. Console data goes missing, as can
be seen in the console log. This is due to _console_interaction calling
makefile() on the console socket each time it is invoked, which must be
losing old buffer contents when going out of scope.
It is not enough to makefile() with buffered=0. That helps significantly
but data loss is still possible. My guess is that readline() has a line
buffer even when the file is in unbuffered mode, that can eat data.
Fix this by providing a console file that persists for the life of the
console.
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com>
Message-Id: <20230912131340.405619-1-npiggin@gmail.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: John Snow <jsnow@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230914155422.426639-9-alex.bennee@linaro.org>
importlib.metadata is included in Python 3.8, so there is no
need to fallback to either importlib-metadata or pkgresources
when generating console script shims.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Debian 10 is not anymore a supported distro, since Debian 12 was
released on June 10, 2023. Our supported build platforms as of today
all support at least 3.8 (and all of them except for Ubuntu 20.04
support 3.9):
openSUSE Leap 15.5: 3.6.15 (3.11.2)
CentOS Stream 8: 3.6.8 (3.8.13, 3.9.16, 3.11.4)
CentOS Stream 9: 3.9.17 (3.11.4)
Fedora 37: 3.11.4
Fedora 38: 3.11.4
Debian 11: 3.9.2
Debian 12: 3.11.2
Alpine 3.14, 3.15: 3.9.16
Alpine 3.16, 3.17: 3.10.10
Ubuntu 20.04 LTS: 3.8.10
Ubuntu 22.04 LTS: 3.10.12
NetBSD 9.3: 3.9.13*
FreeBSD 12.4: 3.9.16
FreeBSD 13.1: 3.9.18
OpenBSD 7.2: 3.9.17
Note: NetBSD does not appear to have a default meta-package, but offers
several options, the lowest of which is 3.7.15. However, "python39"
appears to be a pre-requisite to one of the other packages we request
in tests/vm/netbsd.
Since it is safe under our supported platform policy, bump our
minimum supported version of Python to 3.8. The two most interesting
features to have by default include:
- the importlib.metadata module, whose lack is responsible for over 100
lines of code in mkvenv.py
- improvements to asyncio, for example asyncio.CancelledError
inherits from BaseException rather than Exception
In addition, code can now use the assignment operator ':='
Because mypy now learns about importlib.metadata, a small change to
mkvenv.py is needed to pass type checking.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
This reverts commit e8e4298fea.
ensuregroup allows to specify both the acceptable versions of avocado,
and a locked version to be used when avocado is not installed as a system
pacakge. This lets us install avocado in pyvenv/ using "mkvenv.py" and
reuse the distro package on Fedora and CentOS Stream (the only distros
where it's available).
ensuregroup's usage of "(>=..., <=...)" constraints when evaluating
the distro package, and "==" constraints when installing it from PyPI,
makes it possible to avoid conflicts between the known-good version and
a package plugins included in the distro.
This is because package plugins have "==" constraints on the version
that is included in the distro, and, using "pip install avocado==88.1"
on a venv that includes system packages will result in an error:
avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.
avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.
But at the same time, if the venv does not include a system distribution
of avocado then we can install a known-good version and stick to LTS
releases.
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1663
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Debian only introduced tomli in the bookworm release. Use a
vendored wheel to avoid requiring a package that is only in
bullseye-backports and is also absent in Ubuntu 20.04.
While at it, fix an issue in the vendor.py scripts which does
not add a newline after each package and hash.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Introduce a new subcommand that retrieves the packages to be installed
from a TOML file. This allows being more flexible in using the system
version of a package, while at the same time using a known-good version
when installing the package. This is important for packages that
sometimes have backwards-incompatible changes or that depend on
specific versions of their dependencies.
Compared to JSON, TOML is more human readable and easier to edit. A
parser is available in 3.11 but also available as a small (12k) package
for older versions, tomli. While tomli is bundled with pip, this is only
true of recent versions of pip. Of all the supported OSes pretty much
only FreeBSD has a recent enough version of pip while staying on Python
<3.11. So we cannot use the same trick that is in place for distlib.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
