Commit Graph

2971 Commits

Author SHA1 Message Date
Leo Nunner
f5a88e9870 fix #4321: properly check cloud-init drive permissions
The process for editing Cloud-init drives checked for inconsistent
permissions: for adding, the VM.Config.Disk permission was needed, while
the VM.Config.CDROM permission was needed to remove a drive. The regex
in drive_is_cloudinit needed to be adapted since the drive names have
different formats before/after they are actually generated.

Due to the regex letting names fall through before, Cloud-init drives
were being checked as disks, even though they are actually treated as
CDROM drives. Due to this, it makes more sense to check for
VM.Config.CDROM instead, while also requiring VM.Config.Cloudinit, since
generating a Cloud-init drive already generates default values that are
passed to the VM.

Signed-off-by: Leo Nunner <l.nunner@proxmox.com>
2022-11-17 08:10:28 +01:00
Thomas Lamprecht
0a1c4503e5 bump version to 7.2-8
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-16 18:31:16 +01:00
Wolfgang Bumiller
1b5706cd16 drop get_pending_changes and simplify cloudinit_pending api call
- The forced-remove flag wasn't really used AFAICT and makes
  no sense IMO.
- Whether or not we care about non-MAC changes does not
  belong here, but should instead taken into account in the
  actual hotplug path recording the cloud-init state (iow.
  into $cloudinit_record_changed().)
  (This is not done here atm.)
- It seems much simpler to just have:
  * 'old' = the old value if it's not a new value
  * 'new' = the new value unless it's being deleted
  * If only one of them is set it's an addition or removal.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-16 18:17:07 +01:00
Wolfgang Bumiller
d29483147d don't call 'cleanup_config' the cloudinit section
It performs schema valdiation (and normalization).

We only ever write values into it which came from an
already validated config, and we also add an additional
"added" key which is not covered by the schema, so this
would fail.

Simply skip it.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-16 18:17:07 +01:00
Wolfgang Bumiller
f16cf6c37d record cloud-init changes in the cloudinit section
introducing an 'added' value in the cloudinit section for
values which have not been present when the cloudinit image
has been generated

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-16 18:17:07 +01:00
Wolfgang Bumiller
4b785da1a9 delay cloudinit generation in hotplug
Hotpluggieg generated a cloudinit image based on old values
in order to attach the device and later update it again, but
the update was only done if cloudinit hotplug was enabled.
This is weird, let's not.

Also introduce 'apply_cloudinit_config' which also write the
config, which, as it turns out, is the only thing we
actually need anyway, currently.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-16 18:17:07 +01:00
Wolfgang Bumiller
0337d531a0 Partially-revert "cloudinit: add cloudinit section for current generated config"
This partially reverts commit 95a5135dad.
Particularly the unprotected write to the config when
generating the cloudinit file. We leave the rest as is for
now and update the callers to deal with the config later.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-16 18:17:07 +01:00
Wolfgang Bumiller
3de134ef4a Revert "cloudinit: avoid unsafe write of VM config"
This reverts commit b137c30c3a.

In preparation of fixing the special:cloudinig section.
2022-11-16 18:16:56 +01:00
Leo Nunner
e0e036e1ba fix #4284: add read-only to non-hotpluggable disk options
Changing the read-only status of a disk is not possible through QMP, so
it needs to be exempt from the hotpluggable values as to notify the
user.

Signed-off-by: Leo Nunner <l.nunner@proxmox.com>
2022-11-16 13:16:53 +01:00
Thomas Lamprecht
1e1d6f589c write config: only write out special cloudinit config if the vm has it
this is only for the current in use CI stuff, not the actual cloud
init config itself.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-16 12:50:58 +01:00
Thomas Lamprecht
71cc2c4177 migration: cloudinit check: bump manager dependency and guard with cloudinit drive
The former to ensure the manager that depends on the newer
qemu-server is actually installed and the latter to avoid false
positives

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-16 12:47:43 +01:00
Alexandre Derumier
6622226553 net: increase max queues to 64
max supported queues tx + rx = 256, so 128 for combined
https://lists.gnu.org/archive/html/qemu-devel/2015-03/msg03917.html

But from above link it also seems that x86 only supports 80 pairs in
practice, so for now "only" quadruple the limit to 64 and see if we
get user feedback for more requested.

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
 [ T: reduce from 128 to 64 and add short rationale for that ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-16 12:14:46 +01:00
Thomas Lamprecht
cf364f9574 indentation fix
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-16 12:08:05 +01:00
Thomas Lamprecht
b137c30c3a cloudinit: avoid unsafe write of VM config
there's no guarantee that we're locked here and it also produces
unnecessary extra IO in most cases.

While at it also avoid that a special:cloudinit section is added on
start to *every* VM, which caused another bug to trigger (see prev.
commit) and is just odd for users that ain't using cloudinit

Note in two call sites that we may need to write the config indeed
out there on the caller side.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-16 12:03:53 +01:00
Thomas Lamprecht
eb9923f9b9 config: fix dropping description on parsing special cloud init section
we now always write out a new clouding special section on start (to
be fixed) independent of any cloudinit drive/config configured or
not, and thus always run into that section after a VM started with
the new qemu-server installed, which in turn set the description
always to undef.

Fixes: 95a5135 ("cloudinit: add cloudinit section for current generated config.")
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-16 11:41:34 +01:00
Thomas Lamprecht
cbfc9d753f parse config: factor out finishing reading the description comment
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-16 11:23:01 +01:00
Thomas Lamprecht
05eae0f21f cleanup validate_cpu_conf
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-15 08:49:04 +01:00
Thomas Lamprecht
74fe3d9a7b config to command: avoid line bloat, keep cmd definition near initial pushes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-15 08:34:47 +01:00
Thomas Lamprecht
326704e73f affinity: add actual example to description
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-15 08:33:04 +01:00
Thomas Lamprecht
1a67f99959 add fixme comment to replace duplicate nodename cache
that function also caches the value, and it recently was changed to
be importable, so we can just import and drop this once a new enough
pve-common is available.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-15 07:27:12 +01:00
Thomas Lamprecht
4c1d2d98c4 d/control: make pve-qemu-kvm build-dependency versioned to >= 7.1
as we use the current version for in the test bed, e.g., cfg2cmd, so
it would fail on older ones.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-13 16:46:30 +01:00
Alexandre Derumier
f314976230 test: add qemu 7.1 multiqueue netdev test
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
 [ T: fixup missing trailing backslash in test ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-13 16:43:12 +01:00
Alexandre Derumier
53ca628507 test: add qemu 7.1 default netdev rx|tx_queue_size=1024
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2022-11-13 16:42:24 +01:00
Alexandre Derumier
620d6b328f virtio-net: increase defaults rx|tx-queue-size to 1024
This is reducing packet drop on high pps, and also needed for dpdk.

Redhat already have use it by default in rhev and his openstack platform too
since 2019.

I'm using it in production since 6 months, I don't have seen performance regression.

fix: (which ask for custom option, but setting it by default seem fine for me)

https://bugzilla.proxmox.com/show_bug.cgi?id=1546
https://bugzilla.proxmox.com/show_bug.cgi?id=2349
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2022-11-13 16:42:23 +01:00
Alexandre Derumier
0c03a39035 fix #4296: virtio-net: enable packed queues for qemu 7.1
virtio 1.1 have improve virtio multiqueue performance,
with a new implementation called "packed queues".

https://www.redhat.com/en/blog/packed-virtqueue-how-reduce-overhead-virtio
https://archive.fosdem.org/2018/schedule/event/virtio/attachments/slides/2167/export/events/attachments/virtio/slides/2167/fosdem_virtio1_1.pdf

This patch enable it by default for qemu 7.1

This don't break old guests with old virtio 1.0 drivers,
as virtio device/devices are forward/backward compatible.

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2022-11-13 16:42:00 +01:00
Thomas Lamprecht
15b9ce0e9a tests: cfg2cmd: add multi-q base test for 7.0 machine version
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-13 16:41:41 +01:00
Thomas Lamprecht
989b718a45 bump version to 7.2-7
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-13 15:47:29 +01:00
Thomas Lamprecht
cd1db1b3e0 migrate test: fix some more grave indentation/whitespace errors
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-13 14:56:57 +01:00
Alexandre Derumier
73ed64967e migration : add del_nets_bridge_fdb
at the end of a live migration, we need to remove old mac entries
on source host (vm is not yet stopped), before resume vm on target host

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
 [T: resolve conflicts and rework on apply ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-13 14:56:57 +01:00
Thomas Lamprecht
4ddd2ca293 net devs: avoid registering MAC to fdb if not static
In theory we can have a config with netX records that do not specify
a `macaddr` property, we just auto-generate on in config2cmd for
startup transitively, but don't save that explicitly back to the
config; so while we could parse the /proc/$pid/cmdline or try to get
the info from QMP (not fully straight forward) it seems rather a
hassle; especially if one has in mind that this cannot happen via the
API FWICT; as there a "deletion" *saves* a newly auto generated value
out to the config, same with clone of a VM and restore of a backup.

So, in basically all reasonable cases we got the `macaddr` available,
but if we don't it makes no sense to add a FDB variable for a *newly*
generated one by the parse_net call, as the VM won't use that (well,
at least if one doesn't get "lucky" and it randomly re-generates the
same as on startup), so allow telling parse_net to skip auto
generating MACs and use that in the add-fdb-entries helper

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-13 14:56:57 +01:00
Thomas Lamprecht
1b5ba4ddc6 net devs: code cleanup new fdb mac add helper
reduce a level of indentation and modernize slightly

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-13 14:56:57 +01:00
Alexandre Derumier
21947fea42 net devs: register vNIC mac to FDB on start/resume
On plain VM start (no live migration), we can simply add MAC address
into the fdb. In case of a live migration, we add the mac address
just before the resume.

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2022-11-13 14:55:26 +01:00
Mira Limbeck
f9408af460 fix #4201: delete cloud-init disk on rollback
If the config doesn't contain the cloud-init disk anymore after the
rollback, we have to clean it up since otherwise no further disk can be
attached unless the one still existing on the storage is deleted.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Reviewed-by: Stefan Hanreich <s.hanreich@proxmox.com>
Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
2022-11-11 19:26:16 +01:00
Thomas Lamprecht
d74f424e39 test: usb: cover more ports on checking xhci 7.1+
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-11 09:10:18 +01:00
Dominik Csapak
b1099442b6 tests: add tests for various combinations of configs for usb
q35 + usb passthrough
q35 + usb3 passthrough
q35 + usb3 passthrough with new xhci controller
old machine type + new usb config error
old machine type + q35 + new usb config error
old ostype (w2k) + new usb config error

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-11 08:52:01 +01:00
Thomas Lamprecht
f6b24f427d usb: fixup: include USB config only for non-q35 again
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-11 07:43:03 +01:00
Thomas Lamprecht
342f049352 usb: small style/code cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-10 17:02:34 +01:00
Thomas Lamprecht
e68881e4a3 usb: get controllers: avoid separate loop for usb 2 devs and improve variable names
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-10 17:02:34 +01:00
Thomas Lamprecht
871ebe1775 usb: rename check_usb_index into assert_usb_index_is_useable
to better convey that this might die

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-10 17:02:34 +01:00
Dominik Csapak
c60cad61a0 fix #3271: USB: allow usb hotplugging for modern guests
same as with the extended support for more usb devices, allow
hotplugging for guests that can use the qemu-xhci controller which
require a machine type >= 7.1 and a ostype l26 or windows > 7

if no usb device was passed through on startup, dynamically add
the xhci controller (and remove if the last usb device is unplugged)
so that live migration is still possible

much of the usb hotplug code was already there, but it still needed
a few adaptions, for example we have to add a chardev when adding
a spice redir port (that gets automatically removed when the
usb-redir device gets removed)

since the spice devices use the id 'usbredirdevX' instead of 'usbX', we
have to manually map that a bit around

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-10 17:02:34 +01:00
Dominik Csapak
0c3d18ef13 USB: increase max usb devices to 14 for newer machine version and ostype
for machine versions >= 7.1 and ostype linux or windows > 7, we use the
qemu-xhci controller where we have up to 14 usable ports, so make them
available to the user

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-10 17:02:34 +01:00
Dominik Csapak
4862922a2b fix #4324: USB: use qemu-xhci for machine versions >= 7.1
going by reports in the forum (e.g. [0]) and semi-official qemu
information[1], we should prefer qemu-xhci over nec-usb-xhci

for compatibility purposes, we guard that behind the machine version,
so that guests with a fixed version don't suddenly have a different usb
controller after a reboot (which could potentially break some hardcoded
guest configs)

0: https://forum.proxmox.com/threads/proxmox-usb-connect-disconnect-loop.117063/
1: https://www.kraxel.org/blog/2018/08/qemu-usb-tips/

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-10 17:02:34 +01:00
Dominik Csapak
3deccbd7d0 USB: use machine_type_is_q35 instead of regex
we refactored that into PVE::QemuServer::Machine a while ago, so we can
use it here

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-10 17:02:34 +01:00
Dominik Csapak
b06a24927c USB: print_usbdevice_full: error out on invalid configuration
should not happen normally, but an inattentive user of that function
may forget to check the validity of the parsed device, so err
on the safe side here

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-10 17:02:34 +01:00
Dominik Csapak
238af88edc move 'windows_version' to Helpers
to avoid a cyclic dependency when we want to use that in PVE::QemuServer::USB

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-10 17:02:34 +01:00
Dominik Csapak
2b938c7d88 print_tabletdevice_full: make use of $q35 variable
just outside of context, we already save the result from
machine_type_is_q35 into the $q35 variable, but never use it.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-10 17:02:34 +01:00
Thomas Lamprecht
faf72d6cbf pci: cleanup pci: unregister mdev directly inline
not worth the hassle of a break/depends cycle

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-10 17:02:34 +01:00
Wolfgang Bumiller
c963efc882 use full path for /usr/bin/taskset
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-10 11:15:43 +01:00
Daniel Bowder
8387039819 fix #3593: add affinity to qemu
Reuse the PVE::CpuSet to validate cpuset formatting.
Add new qemu property called 'affinity' to store the cpuset.
Push taskset command in front of kvm if 'affinity' is set.

Signed-off-by: Daniel Bowder <daniel@bowdernet.com>
2022-11-10 09:39:28 +01:00
Dominik Csapak
6fa358a334 pci: make mediated device sysfs path independent of PCI id
mdevs have a host-unique UUID they are indexed with in the PCI-id
independent `/sys/bus/mdev/devices/<uuid>` path, so there is no need
to go through the PCI id for them.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-09 09:06:19 +01:00