Commit Graph

1318 Commits

Author SHA1 Message Date
Thomas Lamprecht
cbfc9d753f parse config: factor out finishing reading the description comment
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-16 11:23:01 +01:00
Thomas Lamprecht
74fe3d9a7b config to command: avoid line bloat, keep cmd definition near initial pushes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-15 08:34:47 +01:00
Thomas Lamprecht
326704e73f affinity: add actual example to description
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-15 08:33:04 +01:00
Thomas Lamprecht
1a67f99959 add fixme comment to replace duplicate nodename cache
that function also caches the value, and it recently was changed to
be importable, so we can just import and drop this once a new enough
pve-common is available.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-15 07:27:12 +01:00
Alexandre Derumier
620d6b328f virtio-net: increase defaults rx|tx-queue-size to 1024
This is reducing packet drop on high pps, and also needed for dpdk.

Redhat already have use it by default in rhev and his openstack platform too
since 2019.

I'm using it in production since 6 months, I don't have seen performance regression.

fix: (which ask for custom option, but setting it by default seem fine for me)

https://bugzilla.proxmox.com/show_bug.cgi?id=1546
https://bugzilla.proxmox.com/show_bug.cgi?id=2349
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2022-11-13 16:42:23 +01:00
Alexandre Derumier
0c03a39035 fix #4296: virtio-net: enable packed queues for qemu 7.1
virtio 1.1 have improve virtio multiqueue performance,
with a new implementation called "packed queues".

https://www.redhat.com/en/blog/packed-virtqueue-how-reduce-overhead-virtio
https://archive.fosdem.org/2018/schedule/event/virtio/attachments/slides/2167/export/events/attachments/virtio/slides/2167/fosdem_virtio1_1.pdf

This patch enable it by default for qemu 7.1

This don't break old guests with old virtio 1.0 drivers,
as virtio device/devices are forward/backward compatible.

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2022-11-13 16:42:00 +01:00
Alexandre Derumier
73ed64967e migration : add del_nets_bridge_fdb
at the end of a live migration, we need to remove old mac entries
on source host (vm is not yet stopped), before resume vm on target host

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
 [T: resolve conflicts and rework on apply ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-13 14:56:57 +01:00
Thomas Lamprecht
4ddd2ca293 net devs: avoid registering MAC to fdb if not static
In theory we can have a config with netX records that do not specify
a `macaddr` property, we just auto-generate on in config2cmd for
startup transitively, but don't save that explicitly back to the
config; so while we could parse the /proc/$pid/cmdline or try to get
the info from QMP (not fully straight forward) it seems rather a
hassle; especially if one has in mind that this cannot happen via the
API FWICT; as there a "deletion" *saves* a newly auto generated value
out to the config, same with clone of a VM and restore of a backup.

So, in basically all reasonable cases we got the `macaddr` available,
but if we don't it makes no sense to add a FDB variable for a *newly*
generated one by the parse_net call, as the VM won't use that (well,
at least if one doesn't get "lucky" and it randomly re-generates the
same as on startup), so allow telling parse_net to skip auto
generating MACs and use that in the add-fdb-entries helper

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-13 14:56:57 +01:00
Thomas Lamprecht
1b5ba4ddc6 net devs: code cleanup new fdb mac add helper
reduce a level of indentation and modernize slightly

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-13 14:56:57 +01:00
Alexandre Derumier
21947fea42 net devs: register vNIC mac to FDB on start/resume
On plain VM start (no live migration), we can simply add MAC address
into the fdb. In case of a live migration, we add the mac address
just before the resume.

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2022-11-13 14:55:26 +01:00
Dominik Csapak
c60cad61a0 fix #3271: USB: allow usb hotplugging for modern guests
same as with the extended support for more usb devices, allow
hotplugging for guests that can use the qemu-xhci controller which
require a machine type >= 7.1 and a ostype l26 or windows > 7

if no usb device was passed through on startup, dynamically add
the xhci controller (and remove if the last usb device is unplugged)
so that live migration is still possible

much of the usb hotplug code was already there, but it still needed
a few adaptions, for example we have to add a chardev when adding
a spice redir port (that gets automatically removed when the
usb-redir device gets removed)

since the spice devices use the id 'usbredirdevX' instead of 'usbX', we
have to manually map that a bit around

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-10 17:02:34 +01:00
Dominik Csapak
0c3d18ef13 USB: increase max usb devices to 14 for newer machine version and ostype
for machine versions >= 7.1 and ostype linux or windows > 7, we use the
qemu-xhci controller where we have up to 14 usable ports, so make them
available to the user

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-10 17:02:34 +01:00
Dominik Csapak
4862922a2b fix #4324: USB: use qemu-xhci for machine versions >= 7.1
going by reports in the forum (e.g. [0]) and semi-official qemu
information[1], we should prefer qemu-xhci over nec-usb-xhci

for compatibility purposes, we guard that behind the machine version,
so that guests with a fixed version don't suddenly have a different usb
controller after a reboot (which could potentially break some hardcoded
guest configs)

0: https://forum.proxmox.com/threads/proxmox-usb-connect-disconnect-loop.117063/
1: https://www.kraxel.org/blog/2018/08/qemu-usb-tips/

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-10 17:02:34 +01:00
Dominik Csapak
238af88edc move 'windows_version' to Helpers
to avoid a cyclic dependency when we want to use that in PVE::QemuServer::USB

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-10 17:02:34 +01:00
Dominik Csapak
2b938c7d88 print_tabletdevice_full: make use of $q35 variable
just outside of context, we already save the result from
machine_type_is_q35 into the $q35 variable, but never use it.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-10 17:02:34 +01:00
Thomas Lamprecht
faf72d6cbf pci: cleanup pci: unregister mdev directly inline
not worth the hassle of a break/depends cycle

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-10 17:02:34 +01:00
Wolfgang Bumiller
c963efc882 use full path for /usr/bin/taskset
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-10 11:15:43 +01:00
Daniel Bowder
8387039819 fix #3593: add affinity to qemu
Reuse the PVE::CpuSet to validate cpuset formatting.
Add new qemu property called 'affinity' to store the cpuset.
Push taskset command in front of kvm if 'affinity' is set.

Signed-off-by: Daniel Bowder <daniel@bowdernet.com>
2022-11-10 09:39:28 +01:00
Dominik Csapak
1b189121fc vm start/stop: cleanup passed-through pci devices in more situations
if the preparing of PCI devices or the start of the VM fails, we need
to cleanup the PCI devices (reservations *and* mdevs), or else it
might happen that there are leftovers which must be manually removed.

to include also mdevs now, refactor the cleanup code from
'vm_stop_cleanup' into it's own function, and call that instead of
only 'remove_pci_reservation'

also simplifies the code, such that it now removes all PCI ids
reserved for that VMID, since we cannot have multiple VMs with the
same VMID anyway

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-09 08:49:45 +01:00
Matthias Heiserer
d80ad18c67 fix #3890 - GUI: warn for unlikely iothread config
Previously, only a plaintext line in the task log showed something was off.
Now, the GUI will show it as a warning.

Reviewed-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Matthias Heiserer <m.heiserer@proxmox.com>
2022-11-08 17:49:51 +01:00
Alexandre Derumier
94ec5e7c30 add cloudinit hotplug
This allow to regenerate config drive if pending values exist
when we change vm options.

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2022-11-08 17:31:59 +01:00
Alexandre Derumier
9687287bb3 api2: add cloudinit_update
This allow to regenerate the config drive with 1 api call.

This also avoid to delete drive first, and recreate it again.

As it's a readonly drive, we can simply live update it,
and eject/replace it with qemu monitor

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2022-11-08 17:31:59 +01:00
Alexandre Derumier
71d9006b7c cloudinit: make cloudnit options fastplug
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2022-11-08 17:23:30 +01:00
Alexandre Derumier
c930f99e57 generate cloudinit drive on offline plug
Currently when only generate it at vm start

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2022-11-08 17:23:30 +01:00
Alexandre Derumier
95a5135dad cloudinit: add cloudinit section for current generated config.
Instead using vm pending options for pending cloudinit generated config,

write current generated cloudinit config in a new [special:cloudinit] SECTION.

Currently, some options like vm name, nic mac address can be hotplugged,
so they are not way to know if the cloud-init disk is already updated.

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2022-11-08 17:23:30 +01:00
Thomas Lamprecht
0d31845364 change cpu shares: drop superfluous parameter
See the related commit in pve-common.

Originally-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-08 16:20:24 +01:00
Fiona Ebner
6b7ef5e57e move get_cpuunits helper to guest-common
while making it take the value directly instead of the config.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-08 16:15:31 +01:00
Fabian Grünbichler
0c19d8fbcf schema: move 'pve-targetstorage' to pve-common
for proper re-use in pve-container.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2022-11-07 16:15:29 +01:00
Fiona Ebner
e31762f25b fix #4099: disable io_uring for virtual disks on CIFS storages
Since kernel 5.15, there is an issue with io_uring when used in
combination with CIFS [0]. Unfortunately, the kernel developers did
not suggest any way to resolve the issue and didn't comment on my
proposed one. So for now, just disable io_uring when the storage is
CIFS, like is done for other storage types that had problematic
interactions.

It is rather easy to reproduce when writing large amounts of data
within the VM. I used
dd if=/dev/urandom of=file bs=1M count=1000
to reproduce it consistently, but your mileage may vary.

Some forum reports about users running into the issue [1][2][3].

[0]: https://www.spinics.net/lists/linux-cifs/msg26734.html
[1]: https://forum.proxmox.com/threads/109848/
[2]: https://forum.proxmox.com/threads/110464/
[3]: https://forum.proxmox.com/threads/111382/

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-10-28 10:15:19 +02:00
Fiona Ebner
34e82fa28b config2cmd: make missing EFI disk warning more visible
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-09-26 11:32:23 +02:00
Thomas Lamprecht
b4415a5253 Revert "cleanup pci devices in more situations"
irrelevant/wrong

This reverts commit 81042ce445.
2022-09-20 14:22:49 +02:00
Dominik Csapak
81042ce445 cleanup pci devices in more situations
if the preparing of pci devices or the start of the vm fails, we need
to cleanup the pci devices (reservations *and* mdevs), or else
it might happen that there are leftovers which must be manually removed.

to include also mdevs now, refactor the cleanup code from 'vm_stop_cleanup'
into it's own function, and call that instead of only 'remove_pci_reservation'

also print the errors of the cleanup steps with 'warn', otherwise we
might discard important errors

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-09-20 13:52:43 +02:00
Dominik Csapak
bbf96e0f1e automatically add 'uuid' parameter when passing through NVIDIA vGPU
When passing through an NVIDIA vGPU via mediated devices, their
software needs the qemu process to have the 'uuid' parameter set to the
one of the vGPU. Since it's currently not possible to pass through multiple
vGPUs to one VM (seems to be an NVIDIA driver limitation at the moment),
we don't have to take care about that.

Sadly, the place we do this, it does not show up in 'qm showcmd' as we
don't (want to) query the pci devices in that case, and then we don't
have a way of knowing if it's an NVIDIA card or not. But since this
is informational with QEMU anyway, i'd say we can ignore that.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-08-12 13:42:33 +02:00
Matthias Heiserer
bd49ecb435 Qemu-Server: fix wrong quotation
Before, the two strings were one single string each, rather than multiple
separated by newlines.

In the docs, this looked very strange as there were linebreaks and the
dots were shown. Can be seen e.g. in api-viewer /nodes/{node}/qemu/{vmid}/config.

Signed-off-by: Matthias Heiserer <m.heiserer@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-30 12:32:25 +02:00
Dominik Csapak
2dda626d1e fix #4119: give namespace parameter to live-restore
we forgot to give the namespace parameter here, so do that.
while we're at it, give the pbs options as a hash instead of adding
another parameter.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-29 13:07:12 +02:00
Thomas Lamprecht
6884a7d7fa fix #4115: enable option to name QEMU threads after their main purpose
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-17 14:25:49 +02:00
Wolfgang Bumiller
21a9ec2ad3 support pbs namespaces
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-12 13:45:37 +02:00
Thomas Lamprecht
8884a8bfb9 virgl: library check: refactor/code style nits
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-02 17:20:39 +02:00
Stoiko Ivanov
9f979d9f31 check prerequisites for virtio-gl display
and exit early if they are not met.
The necessary libraries were taken from Thomas' post in our community
forum:
https://forum.proxmox.com/threads/.61801/post-466767 (ff)

The /dev/dri/renderD.* check is based on util/drm.c in the current
qemu source code.

Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2022-05-02 17:15:42 +02:00
Fabian Ebner
13d121d79b fix #3861: migrate: fix live migration when cloud-init changes storage
Generalizes fd95d780 ("migrate: send updated TPM state volid to target
node") to also handle other offline migrated disks appearing in the
VM config, which currently should only be cloud-init.

Breaks migration new -> old under similar (edge-case-)conditions as
fd95d780 did.

Keep sending the 'tpmstate0' STDIN parameter to avoid breaking new ->
old in the scenario fd95d780 fixed.

Keep parsing the vm_start 'tpmstate0' STDIN parameter to avoid
breaking old -> new, and to be able to keep sending it.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-28 18:29:12 +02:00
Thomas Lamprecht
f8ea1b30ce enable spice also for virtio-gpu display
same spirit as commit 3591b62b85

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-28 17:47:07 +02:00
Fabian Ebner
202a2a0bee api: create: allow overriding non-disk options during restore
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-28 08:43:09 +02:00
Thomas Lamprecht
463bb05f93 enable spice also for virtio-gl display
It can be used with that and the agent features can make it a nicer
experience.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-28 08:07:14 +02:00
Alexandre Derumier
c70e4ec397 memory: enable balloon free-page-reporting for auto-memory reclaim
Allow balloon device  driver to report hints of guest free pages to
the host, for auto memory reclaim

https://lwn.net/Articles/759413/
https://events19.linuxfoundation.org/wp-content/uploads/2017/12/KVMForum2018.pdf

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
[ T: fixup tests ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-27 11:08:50 +02:00
Thomas Lamprecht
baa4f62de0 fix QemuServer module
fixes d1e7b92289 ("parse vm config: remove "\s*" from multi-line
comment regex")

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-27 11:08:50 +02:00
Stefan Sterz
d1e7b92289 parse vm config: remove "\s*" from multi-line comment regex
To be consistent with PBS's implementation of multi-line comments
remove "\s*" here too. Since the regex isn't lazy .* matches
everything \s* would anyway. (Note that new lines occurs after "$").

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2022-04-27 08:17:21 +02:00
Fabian Ebner
e8b07b29ee restore: also deactivate/destroy cloud-init disk upon error
by re-using the same hash that's used when allocating/activating the
disks in the helpers doing the opposite.

Also in preparation to allow skipping certain disks upon restore.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-25 18:19:17 +02:00
Fabian Ebner
ff86112c14 restore deactivate volumes: never die
Such an error shouldn't abort the whole operation.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-25 18:19:17 +02:00
Fabian Ebner
e60afe8273 restore destroy volumes: remove check for absolute path
Only a result from vdisk_alloc is assigned as a volid and that's never
an absolute path.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-25 18:19:17 +02:00
Fabian Ebner
01a4377fc1 restore: cleanup oldconf: also clean up snapshots from kept volumes
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-25 18:19:17 +02:00