Commit Graph

2681 Commits

Author SHA1 Message Date
Stefan Reiter
f9dde219f2 fix #3075: add TPM v1.2 and v2.0 support via swtpm
Starts an instance of swtpm per VM in it's systemd scope, it will
terminate by itself if the VM exits, or be terminated manually if
startup fails.

Before first use, a TPM state is created via swtpm_setup. State is
stored in a 'tpmstate0' volume, treated much the same way as an efidisk.

It is migrated 'offline', the important part here is the creation of the
target volume, the actual data transfer happens via the QEMU device
state migration process.

Move-disk can only work offline, as the disk is not registered with
QEMU, so 'drive-mirror' wouldn't work. swtpm itself has no method of
moving a backing storage at runtime.

For backups, a bit of a workaround is necessary (this may later be
replaced by NBD support in swtpm): During the backup, we attach the
backing file of the TPM as a read-only drive to QEMU, so our backup
code can detect it as a block device and back it up as such, while
ensuring consistency with the rest of disk state ("snapshot" semantic).

The name for the ephemeral drive is specifically chosen as
'drive-tpmstate0-backup', diverging from our usual naming scheme with
the '-backup' suffix, to avoid it ever being treated as a regular drive
from the rest of the stack in case it gets left over after a backup for
some reason (shouldn't happen).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-10-05 06:51:02 +02:00
Fabian Grünbichler
d2ceac56b5 api: template: invert lock and fork
like for other API calls, repeat the cheap checks done for early abort
before forking and without locks after forking and obtaining the lock,
and only hold the flock in the forked worker instead of across the fork.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Fabian Ebner <f.ebner@proxmox.com>
2021-10-04 09:46:57 +02:00
Fabian Grünbichler
b297918ce2 api: return UPID in template call
as reported on the forum, this is currently missing, making status
queries via the API impossible:

https://forum.proxmox.com/threads/create-vm-via-api-interface.95942/#post-416084

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Fabian Ebner <f.ebner@proxmox.com>
2021-10-04 09:46:52 +02:00
Fabian Grünbichler
3e07c6d54b vm_destroy: remove pending volumes as well
if a volume is only referenced in the pending section of a config it was
previously not removed when removing the VM, unless the non-default
'remove unreferenced disks' option was enabled.

keeping track of volume IDs which we attempt to remove gets rid of false
warnings in case a volume is referenced both in the config and the
pending section, or multiple times in the config for other reasons.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-09-30 17:37:27 +02:00
Thomas Lamprecht
243af50d49 bump version to 7.0-14
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-09-22 09:32:25 +02:00
Thomas Lamprecht
f8830c4d6e migrate: code style, use up to 100cc if it helps to reduce line-bloat
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-09-22 09:26:18 +02:00
Thomas Lamprecht
95b3583b5e migrate: simplify code and add comment
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-09-22 09:25:53 +02:00
Fabian Ebner
d213ba299d migrate: use correct target storage id for checks
The '--targetstorage' parameter does not apply to shared storages.

Example for a problem solved with the enabled check: Given a VM with
images only on a shared storage 'storeA', not available on the target
node (i.e. restricted by the nodes property). Then using
'--targetstorage storeB' would make offline migration suddenly
"work", but of course the disks would not be accessible and then
trying to migrate back would fail...

Example for a problem solved with the content type check: if a
VM had a shared ISO image, and there was a '--targetstorage storeA'
option, availablity of the 'iso' content type is checked for
'storeA', which is wrong as the ISO would not be moved to that
storage.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-09-22 08:57:35 +02:00
Thomas Lamprecht
a8d0fec3c2 whitespace/indentation fixes & cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-09-09 17:50:21 +02:00
Fabian Grünbichler
9a66c311ac fix #3608: unbreak removal of scsi controller
the assumption that the index of the controller matches that of the last
removed drive only holds for virtio-scsi-single controller, which makes
the old code print a warning when removing the last drive of a
non-virtio-scsi-single controller except when the indices line up by
chance.

we can simply only call a simplified qemu_iothread_del when removing a
scsi disk of a VM with the virtio-scsi-single controller, and skip the
call for the other controllers which don't support io-threads anyway.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-09-09 17:46:02 +02:00
Constantin Herold
ae776a6288 fix #3581: pass size via argument for memory-backend-ram qmp call
Signed-off-by: Constantin Herold <proxmox8914@herold.me>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-08-18 10:41:04 +02:00
Fabian Grünbichler
0d519cfc76 bump version to 7.0-13
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-08-05 14:03:33 +02:00
Dominik Csapak
a2e22f9fb2 api2: only add ide drives for non-legacy bootorders
@bootorder only contains entries for non-legacy bootorder entries,
but the default one contains all cdroms anyway, and if the user
explicitely disabled cdroms, it is ok to not add them back
for the new cdrom drive.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-08-05 13:58:35 +02:00
Thomas Lamprecht
94e307177f bump version to 7.0-12
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-30 16:53:48 +02:00
Dominik Csapak
5170f6282d bootorder: fix double entry on cdrom edit
We unconditionally added an entry into the bootorder whenever we
edited the drive, even if it was already in there. Instead we only want to do
that if the bootorder list does not contain it already.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-30 15:07:10 +02:00
Dominic Jäger
8717d89d92 Fix #3371: parse ovf: Allow dots in VM name
Dots are allow in PVE VM names, so they should not be dropped during import.

Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
2021-07-29 17:17:39 +02:00
Thomas Lamprecht
619a43992a bump version to 7.0-11
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-23 11:08:53 +02:00
Mira Limbeck
104f47a9f8 fix #2563: allow live migration with local cloud-init disk
The content of the ISO should be the same on both nodes, so offline
migrate the ISO, but don't regenerate it on VM start on the target node.

This way even with snippets the content will not change during live
migration.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2021-07-23 11:04:22 +02:00
Stefan Reiter
deb734e348 api: always add new CD drives to bootorder
Attaching an ISO image to a VM is usually/often done for two reasons:
* booting an installer image
* supplying additional drivers to an installer (e.g. virtio)

Both of these cases (the latter at least with SeaBIOS and the Windows
installer) require the disk to be marked as bootable.

For this reason, enable the bootable flag for all new CDROM drives
attached to a VM by adding it to the bootorder list. It is appended to
the end, as otherwise it would cause new drives to boot before already
existing boot targets, which would be a more grave (and IMO bad)
behaviour change.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-07-23 11:04:19 +02:00
Stefan Reiter
55c7f9cf66 live-restore: fail early if target storage doesn't exist
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-07-23 11:04:15 +02:00
Thomas Lamprecht
78a3ada744 lvm: avoid the use of IO uring
there may be a kernel issue or a bug in how QEMU uses io_uring, but
we have users that report crashes which f.ebner could see on some
workloads, not really deterministic though and it seems that in newer
kernel versions (5.12+) the crash becomes a hang

While we're closing in on the actual issue here (which could be the
same as for RBD) let's disable io_uring for LVM.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-23 10:55:20 +02:00
Thomas Lamprecht
e83dd50a36 nic: support e1000e
That bit of code seems to be enough here, tested with

qm set VMID --net1 e1000e=EA:93:42:22:10:D8,bridge=vmbr0

on a Alpine Linux and a Windows Server 2016 VM.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-16 19:10:01 +02:00
Thomas Lamprecht
f7bc17ca6d nic: one per line and sort
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-16 18:57:38 +02:00
Thomas Lamprecht
b236d33e4a bump version to 7.0-10
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-07 14:56:40 +02:00
Fabian Ebner
ec5d198e5b cfg2cmd: avoid io_uring with LVM and write{back, through} cache
Reported in the community forum[0]. Also tried with LVM-thin, but it
doesn't seem to be affected.

See also 628937f53a for the same fix for
krbd.

[0]: https://forum.proxmox.com/threads/after-upgrade-to-7-0-all-vms-dont-boot.92019/post-401017

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-07-07 14:55:17 +02:00
Thomas Lamprecht
d2428c7713 bump version to 7.0-9
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-07 12:07:39 +02:00
Thomas Lamprecht
d3f9db4d7a fix cpuunits defaults regression
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-07 12:07:39 +02:00
Thomas Lamprecht
60ac42335a bump version to 7.0-8
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-07 10:28:01 +02:00
Thomas Lamprecht
67498860a4 conf: cpuunits: adapt description and defaults for cgroup v2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-07 10:27:05 +02:00
Thomas Lamprecht
6c71a52acd cpu weight: clamp to maximum for cgroup v2
In v2 the range is [1, 10000], but the API allows the old limits from
2 to 262144, so clamp the upper for v2.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-07 09:36:20 +02:00
Alexandre Derumier
4a5cb613d3 api2: fix vmconfig_apply_pending errors handling
commit
https://git.proxmox.com/?p=qemu-server.git;a=commit;h=eb5e482ded9ae6aeb6575de9441b79b90a5de531

have introduced error handling for offline pending apply,

-               PVE::QemuServer::vmconfig_apply_pending($vmid, $conf, $storecfg, $running);
+               PVE::QemuServer::vmconfig_apply_pending($vmid, $conf, $storecfg, $running, $errors);

 sub vmconfig_apply_pending {
-    my ($vmid, $conf, $storecfg) = @_;
+    my ($vmid, $conf, $storecfg, $errors) = @_;

but they was wrong nonused $running param, so currently $errors are not correctly handled

Fixes: eb5e482ded ("vmconfig_apply_pending: add error handling")
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
Tested-by: Oguz Bektas <o.bektas@proxmox.com>
2021-07-06 12:40:43 +02:00
Thomas Lamprecht
1a947e67fe bump version to 7.0-7
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-05 20:49:55 +02:00
Thomas Lamprecht
738dc81cba further improve on #3329, ensure write-back is used over write-around
Suggested-by: Rick Altherr <kc8apf@kc8apf.net>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-05 20:47:50 +02:00
Thomas Lamprecht
e43b19109e bump version to 7.0-6
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-02 09:45:10 +02:00
Thomas Lamprecht
9de049b0ad live-restore: add another comment for efidisk special case just to be sure
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-01 12:57:11 +02:00
Stefan Reiter
6f94e1625b live-restore: preload efidisk before starting VM
The efidisk never got restored correctly before, since we don't use the
generic print_drive_commandline_full for it, and as such it didn't get a
backing image attached. This not only causes the efidisk data to be lost
on restore, but also an error at the end, since we try to remove a
non-existing PBS blockdev.

Since it is attached differently to a regular drive, adding PBS backing
would be more difficult, but not to worry: an efidisk is small enough
that it doesn't hurt performance to just restore it via the regular
mechanism before starting the VM, and simply excluding it from the live
restore entirely.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-07-01 12:54:32 +02:00
Stefan Reiter
628937f53a cfg2cmd/drive: don't use io_uring for krbd with wb/wt cache
As reported here and locally reproduced:
https://forum.proxmox.com/threads/efi-vms-wont-start-under-7-beta-with-writeback-cache.91629/

This configuration is currently broken. Until we figure out how to fix
it properly, we can just have this (luckily very narrow) config pattern
fall back to aio=threads as it used to.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-07-01 09:40:51 +02:00
Thomas Lamprecht
defa8e21be bump version to 7.0-5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-24 18:58:22 +02:00
Dominik Csapak
2c44ec4974 fix #2175: PVE/API2/Qemu: update_vm_api: check old drive for permissions too
otherwise a user with only VM.Config.CDROM can detach a disk from a VM
by updating it to a cdrom drive

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-24 18:56:04 +02:00
Dominik Csapak
bb660bc3ce PVE/API2/Qemu/update_vm_api: refactor drive permission check
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-24 18:56:04 +02:00
Thomas Lamprecht
6185586ebd bump version to 7.0-5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-24 18:45:57 +02:00
Thomas Lamprecht
5620282fbd cfg2cmd: add btrfs-store and test for cache mode
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-24 18:42:55 +02:00
Wolfgang Bumiller
0fe779a62c don't default to O_DIRECT on btrfs without nocow
otherwise it'll produce a whole lot of checksum errors

and while this would be nice as a storage feature check,
it's hard to be 100% accurate there anyway since a directory
storage can point anywhere, like for instance a btrfs
directory, causing the same issue...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-06-24 17:15:31 +02:00
Thomas Lamprecht
07dfe5391e bump version to 7.0-4
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-23 12:57:31 +02:00
Fabian Grünbichler
85fcf79e21 template: add -snapshot to KVM command
this allows effectively setting ALL volumes as read-only, even if the
disk controller does not support it. without it, IDE and SATA disks
with (base) volumes which are marked read-only/immutable on the storage
level prevent the template VM from starting for backup purposes.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-06-23 12:37:40 +02:00
Fabian Grünbichler
2c53ff94fa test: add template drive read-only tests
ensuring the current behaviour:

templates will pass readonly=on to Qemu, except for SATA and IDE drives
which don't support that flag.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-06-23 12:37:40 +02:00
Fabian Grünbichler
b4dc647557 template: mark efidisk as read-only
otherwise backups of templates using UEFI fail with storages like LVM
thin, where the volumes are not writable. disk controllers like IDE and
SATA that don't support being read-only are still broken for UEFI.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
[ drop the readonly=off when not required, resolve merger conflict
  from Dominik's EFI disk cache mode fix ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-23 12:36:20 +02:00
Fabian Grünbichler
75748d4492 drive: factor out read-only helper
we also need it for efidisks.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-06-23 12:27:54 +02:00
Fabian Grünbichler
75c430cee8 test: unbreak restore_config_test
for unprivileged users (and possibly some root setups). reading from
pmxcfs now results in a hard error for unprivileged users, so there
might be some more of these lurking somewhere..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-06-23 12:27:54 +02:00
Fabian Ebner
872cfcf5bc api: update vm: correctly handle warnings status for delayed task
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-06-23 12:26:53 +02:00