Commit Graph

3158 Commits

Author SHA1 Message Date
Dominik Csapak
e3971865b4 enable cluster mapped USB devices for guests
this patch allows configuring usb devices that are mapped via
cluster resource mapping when the user has 'Mapping.Use' on the ACL
path '/mapping/usb/{ID}' (in addition to the usual required vm config
privileges)

for now, this is only valid if there is exactly one mapping for the
host, since we don't track passed through usb devices yet

This now also checks permissions on clone/restore, meaning a
'non-mapped' device can only be cloned/restored as root@pam user.
That is a breaking change.

Refactor the checks for restoring into a sub, so we have central place
where we can add such checks

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-By:  Markus Frank <m.frank@proxmox.com>
2023-06-16 16:24:02 +02:00
Dominik Csapak
0cf8d56c6d usb: refactor usb code and move some into USB module
similar to how we handle the PCI module and format. This makes the
'verify_usb_device' method and format unnecessary since
we simply check the format with a regex.

while doing tihs, i noticed that we don't correctly check for the
case-insensitive variant for 'spice' during hotplug, so fix that too

With this we can also remove some parameters from the get_usb_devices
and get_usb_controllers functions

while were at it, refactor the permission checks for the usb config too
and use the new 'my sub' style for the functions

also make print_usbdevice_full parse the device itself, so we don't have
to do it in multiple places (especially in places where we don't see
that this is needed)

No functional change intended

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-By:  Markus Frank <m.frank@proxmox.com>
2023-06-16 16:24:02 +02:00
Aaron Lauterer
e3aad44126 migration: add target_storage_check_available
We use this in a few places. By factoring it into its own function, we
can avoid running slightly different checks in various places.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2023-06-16 15:03:28 +02:00
Thomas Lamprecht
56d28037b5 helpers: actualy future proof and allow also checking releases
ensuring the editor state is saved helps -.-

Reported-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-16 13:53:52 +02:00
Thomas Lamprecht
e17312c99f helpers: future proof and allow also checking releases
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-16 13:26:43 +02:00
Thomas Lamprecht
9c6eabf028 fix #4784: helpers: cope with native versions in manager version check
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-16 13:22:40 +02:00
Fiona Ebner
5854213953 status: fix description of qmpstatus property
Using the word 'agent' is highly confusing here as there is no QMP
agent and thus wrongly suggests that the value is related to the
guest agent[0].

[0]: https://forum.proxmox.com/threads/123590/post-537716

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-14 13:33:56 +02:00
Alexandre Derumier
1359e23fb4 cpuconfig: add new x86-64-vX models
https://gitlab.com/x86-psABIs/x86-64-ABI/
https://lists.gnu.org/archive/html/qemu-devel/2021-06/msg01592.html
"
In 2020, AMD, Intel, Red Hat, and SUSE worked together to define
three microarchitecture levels on top of the historical x86-64
baseline:

  * x86-64:    original x86_64 baseline instruction set
  * x86-64-v2: vector instructions up to Streaming SIMD
               Extensions 4.2 (SSE4.2)  and Supplemental
               Streaming SIMD Extensions 3 (SSSE3), the
               POPCNT instruction, and CMPXCHG16B
  * x86-64-v3: vector instructions up to AVX2, MOVBE,
               and additional bit-manipulation instructions.
  * x86-64-v4: vector instructions from some of the
               AVX-512 variants.
"

This patch add new builtin model derivated from qemu64 model,
to be compatible between intel/amd.

mandatory flags from qemu-doc generator:
https://gitlab.com/qemu/qemu/-/blob/master/scripts/cpu-x86-uarch-abi.py

levels = [
    [ # x86-64 baseline
        "cmov",
        "cx8",
        "fpu",
        "fxsr",
        "mmx",
        "syscall",
        "sse",
        "sse2",
    ],
    [ # x86-64-v2
        "cx16",
        "lahf-lm",
        "popcnt",
        "pni",
        "sse4.1",
        "sse4.2",
        "ssse3",
    ],
    [ # x86-64-v3
        "avx",
        "avx2",
        "bmi1",
        "bmi2",
        "f16c",
        "fma",
        "abm",
        "movbe",
	"xsave"  #missing from qemu doc currently
    ],
    [ # x86-64-v4
        "avx512f",
        "avx512bw",
        "avx512cd",
        "avx512dq",
        "avx512vl",
    ],
]

x86-64-v1 : I'm skipping it, as it's basicaly qemu64|kvm64 -vme,-cx16 for compat Opteron_G1 from 2004
            so will use it as qemu64|kvm64 is higher are not working on opteron_g1 anyway

x86-64-v2 : Derived from qemu, +popcnt;+pni;+sse4.1;+sse4.2;+ssse3

min intel: Nehalem
min amd : Opteron_G3

x86-64-v2-AES : Derived from qemu, +aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3

min intel: Westmere
min amd : Opteron_G3

x86-64-v3 : Derived from qemu64 +aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3;+avx;+avx2;+bmi1;+bmi2;+f16c;+fma;+abm;+movbe+xsave

min intel: Haswell
min amd : EPYC_v1

x86-64-v4 : Derived from qemu64 +aes;+popcnt;+pni;+sse4.1;+sse4.2;+ssse3;+avx;+avx2;+bmi1;+bmi2;+f16c;+fma;+abm;+movbe;+xsave;+avx512f;+avx512bw;+avx512cd;+avx512dq;+avx512vl

min intel: Skylake
min amd : EPYC_v4

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2023-06-12 17:30:11 +02:00
Thomas Lamprecht
60302a968a bump version to 8.0.3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-09 12:20:45 +02:00
Fabian Grünbichler
525c6c6271 qemu: fix permission check call
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-06-09 12:10:46 +02:00
Thomas Lamprecht
77eebd880c bump version to 8.0.2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-09 10:28:56 +02:00
Thomas Lamprecht
2a93450466 d/control: bump versioned dependency for libpve-access-control-perl
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-09 10:24:05 +02:00
Fiona Ebner
814053b4c3 qm: remote migration: improve error when storage cannot be found
It can also be a permission issue, so the current error can be
a bit confusing.

Reported in the community forum:
https://forum.proxmox.com/threads/120619/post-562660

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-09 09:49:40 +02:00
Thomas Lamprecht
3f9c3a04d1 d/control: bump versioned dependency for libpve-guest-common-perl
to ensure we got the vnet access check helper available for us, and
also that the get_derived_property interface is available for our
users

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-08 17:50:50 +02:00
Thomas Lamprecht
951714ea92 restore: check bridge access when actual config is available
This was not only rather inefficient (getting the config from the
archive twice) but also wrong, as we can override options on restore,
so we can do the check only when the backed-up config and override
config got merged.

If this is to late from POV of volume deletion or the like, then the
issue is that those things happen to early, as we can only know what
to do with the actual target config, so destructive actions that
happen before that are wrong by design.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-08 17:50:50 +02:00
Thomas Lamprecht
d6deb7f6bb move helper to check bridge access out of api
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-08 17:50:50 +02:00
Alexandre Derumier
9cfd06d944 api: check bridge access for create, update, clone & restore
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
Reviewed-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-08 17:47:53 +02:00
Fiona Ebner
3dd16f4a6c config: implement method to calculate derived properties from a config
See the corresponding commit in guest-common for more information.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-08 17:47:53 +02:00
Fiona Ebner
502870a04c qmeventd: extract vmid from cgroup file instead of cmdline
This is the single remaining user of the id argument. The id argument
is a Proxmox-specific extension to QEMU, which we'd like to drop to
reduce our differences with upstream QEMU.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-07 19:37:09 +02:00
Fiona Ebner
5674d19810 remove left-over mentions of to-be-dropped, outdated QMP commands
The commands snapshot-drive and delete-drive-snapshot have been unused
by qemu-server since commit eba2b721 ("use qemu's blockdev-snapshot
functions") and are now going to be dropped in our QEMU builds too, so
get rid of these left-overs.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-07 19:36:45 +02:00
Leo Nunner
3e546c5ada cloudinit: pass through hostname via fqdn field
If no FQDN is provided, we simply set it to the current hostname. This
ensures that the hostname *really* gets set, since we encountered an
issue on Fedora and CentOS based systems where no hostname got set at
all.

When there's no FQDN set in the cloudinit config, this leads to the
following entry:

    127.0.1.1 <hostname> <hostname>

Which doesn't seem to cause any issues.

Tested on:
 - Ubuntu 23.04
 - CentOS 8
 - Fedora 38
 - Debian 11
 - SUSE 15.4

Signed-off-by: Leo Nunner <l.nunner@proxmox.com>
2023-06-07 19:33:28 +02:00
Fiona Ebner
606d9d76b2 fix #2315: api: have resize endpoint spawn a worker task
Similar to the corresponding endpoint for containers. Because disks
are involved, this can be a longer running operation, as is also
indicated by the 60 seconds timeout used in qemu_block_resize() which
is called by this endpoint.

This is a breaking API change.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-07 19:23:52 +02:00
Thomas Lamprecht
a6f14ef49b d/control: bump minimal QEMU version to 7.1
just to be safe and have this on a more recent one, as we def. do not
support 3.0 anymore

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-07 19:23:15 +02:00
Fiona Ebner
5c01f59177 fix #517: api: allow resizing qcow2 disk with snapshots
Support for this was added in QEMU 5.1 by commit 7fa140abf6 ("qcow2:
Allow resize of images with internal snapshots").

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-07 19:22:17 +02:00
Fiona Ebner
d62bdac593 fast plug options: add migrate_downtime and migrate_speed
for convenience. These options do not influence the QEMU instance
directly, but are only used for migration, so no need to keep them in
pending.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-07 18:37:51 +02:00
Fiona Ebner
f68910a05f fast plug options: order alphabetically
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-07 18:37:51 +02:00
Leo Nunner
3a704639b7 cloudinit: fix 'pending' api endpoint
This patch partially reverts commit 1b5706cd16,
by reintroducing the old format for return values (key, value, pending,
delete), but drops the "force-delete" return value. Right now, this
endpoint does not conform to its own format, because the return values
are as follows:

{
    key => {
	old => 'foo',
	new => 'bar',
    },
    […]
}

While the format specified is

[
    {
	key => 'baz',
	old => 'foo',
	new => 'bar',
    },
    […]
]

This leads to the endpoint being broken when used through 'qm' and
'pvesh'. Using the API works fine, because the format doesn't get
verified there. Reverting this change brings the advantage that we can
also use PVE::GuestHelpers::format_pending when calling the endpoint
through qm again.

Signed-off-by: Leo Nunner <l.nunner@proxmox.com>
2023-06-07 18:32:42 +02:00
Leo Nunner
efa3355d3b fix #3428: cloudinit: add parameter for upgrade on boot
up until now, we did an automatic upgrade after the first boot in our
standard cloud-init config. This has been requested to be toggleable
several times [1][2]. With this patch, "package_upgrade" is disabled by
default, and needs to be enabled manually, diverging from the previous
behaviour.

[1] https://forum.proxmox.com/threads/how-to-prevent-automatic-apt-upgrade-during-the-first-boot-with-cloud-init.68472/
[2] https://forum.proxmox.com/threads/cloud-init-ohne-package-upgrade.123841/

Signed-off-by: Leo Nunner <l.nunner@proxmox.com>
2023-06-07 18:25:46 +02:00
Fabian Grünbichler
46f3fc25d8 allow setting ipconfigX with VM.Config.Cloudinit
these config keys only affect the cloudinit drive contents (and state of the
guest inside the VM), they are not used anywhere on the hypervisor side, so
they should not require VM.Config.Network (which allows a lot more, such as
changing vNIC VLAN tags or the bridges they are connected to).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-06-07 18:23:30 +02:00
Fiona Ebner
bda7ccb1c9 schema: avoid using deprecated -no-hpet in example for 'args' property
instead use a recent example that served as a workaround in #4625.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-07 17:35:41 +02:00
Fiona Ebner
17bacc2182 cfg2cmd: replace deprecated no-hpet option with hpet=off machine flag
like the deprecation message printed by QEMU suggests.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-07 17:35:41 +02:00
Fiona Ebner
0f704640be cfg2cmd: replace deprecated no-acpi option with acpi=off machine flag
like the deprecation message printed by QEMU suggests.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-07 17:35:41 +02:00
Fiona Ebner
e35eb8766b cfg2cmd: use actual backend names instead of removed tty and paraport aliases
As described in:
https://qemu-project.gitlab.io/qemu/about/removed-features.html#chardev-backend-aliases-tty-and-parport-removed-in-8-0

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-07 17:35:41 +02:00
Wolfgang Bumiller
9d8d0be31b fixup d/changelog
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-06-07 13:50:14 +02:00
Wolfgang Bumiller
772c367bc7 bump version to 8.0.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-06-07 13:48:04 +02:00
Wolfgang Bumiller
d6d4e2675f bump common dep to 8.0.2, guest-common to 5.0.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-06-07 13:47:52 +02:00
Dominik Csapak
1572afe8ed api: switch agent api call to 'array' type
we don't want to use the '-alist' formats anymore in favor of real arrays

Acked-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2023-06-07 13:43:53 +02:00
Fiona Ebner
2e4357c537 block resize: avoid passing zero size to QMP command
Commit 7246e8f9 ("Set zero $size and continue if volume_resize()
returns false") mentions that this is needed for "some storages with
backing block devices to do online resize" and since this patch came
together [0] with pve-storage commit a4aee43 ("Fix RBD resize with
krbd option enabled."), it's safe to assume that RBD with krbd is
meant. But it should be the same situation for any external plugin
relying on the same behavior.

Other storages backed by block devices like LVM(-thin) and ZFS return
1 and the new size respectively, and the code is older than the above
mentioned commits. So really, the RBD plugin just should have returned
a positive value to be in-line with those and there should be no need
to pass 0 to the block_resize QMP command either.

Actually, it's a hack, because the block_resize QMP command does not
actually do special handling for the value 0. It's just that in the
case of a block device, QEMU won't try to resize it (and not fail for
shrinkage). But the size in the raw driver's BlockDriverState is
temporarily set to 0 (which is not nice), until the sector count is
refreshed, where raw_co_getlength is called, which queries the new
size and sets the size in the raw driver's BlockDriverState again as a
side effect. It's not known to cause any issues, but bdrv_getlength is
a coroutine wrapper starting from QEMU 8.0.0, and it's just better to
avoid setting a completely wrong value even temporarily. Just pass the
actually requested size like is done for LVM(thin) and ZFS.

[0]: https://lists.proxmox.com/pipermail/pve-devel/2017-January/025060.html

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-06 19:42:16 +02:00
Christian Ebner
bb547dcbd6 net: Skip and warn of interfaces without bridge
Handle and warn about network interfaces which are not attached to
any bridge because the user actively removed it from the VM config.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2023-06-06 18:26:27 +02:00
Stefan Hanreich
31150d20c4 qmrestore: improve description of bwlimit parameter
This makes the description consistent with the other places that
have bwlimit as a parameter as well.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
2023-06-06 17:56:34 +02:00
Fiona Ebner
2d7a026e99 fix #4737: qmeventd: gracefully handle interrupted epoll_wait call
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-24 17:11:01 +02:00
Fiona Ebner
a7547a7c9f tests: fix invoking migration tests with make
Even if between single quotes, the dollar sign needs to be escaped
here. Otherwise, there will be an error
> Search pattern not terminated at -e line 1.
and no migration tests would be run. The error did not lead to
aborting though, making it harder to notice.

Fixes: aac89f6c ("tests: avoid calling test script to get target names")
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-22 15:51:58 +02:00
Fiona Ebner
076fee16ff bump version to 8.0.0
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
 [ T: slightly rework changelog ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-19 15:08:28 +02:00
Thomas Lamprecht
3f8253e848 buildsys: rework clean target, avoid doc-gen one
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-19 15:06:46 +02:00
Thomas Lamprecht
fe61988beb buildsys: use more central dpkg packaging makefile
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-19 15:06:46 +02:00
Thomas Lamprecht
aac89f6cfa tests: avoid calling test script to get target names
As otherwise we couple *all* Makefile targets to the dependencies of
the test script, even for a simple make call (e.g., done on building
the source), so use a much simpler heuristic that just depends on
perl, which is essential in Debian.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-19 15:06:46 +02:00
Thomas Lamprecht
1edeff742d tests: simplify outputting available migration test names
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-19 15:06:46 +02:00
Thomas Lamprecht
5591892520 buildsys: ignore failure to include doc-gen file, drop useless export
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-19 15:06:46 +02:00
Fiona Ebner
201791236b buildsys: add sbuild target for convenience
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-19 15:06:46 +02:00
Fiona Ebner
ad19e46861 buildsys: create build directory atomically
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-19 15:06:46 +02:00