Commit Graph

3158 Commits

Author SHA1 Message Date
Fabian Ebner
7344af7bf8 clone disk: pass in efi vars size rather than config
It's confusing that the config associated to the destination is
actually a reference to the source config for both existing callers.

Also, disk import will need to base the calculation on the passed-in
drive parameters and not just the current config, so this change is in
preparation for that too.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-03-11 10:45:59 +01:00
Fabian Ebner
1196086ff1 clone disk: group source and target parameters
to make the interface more digestible.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-03-11 10:45:59 +01:00
Fabian Ebner
31b1db61c2 clone disk: remove check for min QEMU version 2.7
Upgrading a cluster node entails re-starting or migrating VMs and even
PVE 6.0 already had QEMU 4.0.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-03-11 10:45:59 +01:00
Fabian Ebner
367e6bf4b7 api: update: pass correct config when creating disks
While the new options should be written to the pending config, the
decisions (currently only one) in create_disks needs to be made for
the current config.

Seems to fix EFI disk creation, but actually, it's only
future-proofing, because, currently, the same OVMF_VARS file is
used independently of $smm.

The correct config is also needed to determine the correct size for
the EFI disk for the upcoming import-from feature.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-03-11 10:45:59 +01:00
Fabian Ebner
fe19840a56 api: create disks: always activate/update size when attaching existing volume
For creation, activation and size update never triggered, because the
passed in $conf is essentially the same as the creation $settings, so
the disk was always detected to be the same as the "existing" one. But
actually, all disks are new, so it makes sense to do it.

For update, activation and size update nearly always triggered,
because only the pending changes are passed in as $conf. The case
where it didn't trigger is when the same pending change was made twice
(there are cases where hotplug isn't done, but makes it even more
unlikely).

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-03-11 10:45:59 +01:00
Fabian Ebner
52b361afd6 device unplug: verify that unplugging scsi disk completed
Avoids the error
  adding drive failed: Duplicate ID 'drive-scsi1' for drive
that could happen when switching over to a new disk (e.g. via qm set),
if unplugging wasn't fast enough.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-03-11 10:45:59 +01:00
Oguz Bektas
6ab41628b2 api: vm_start: 'force-cpu' is for internal migration use only
'force-cpu' parameter was introduced to allow live-migration of VMs with
custom CPU models; it does not need to be allowed for general use on
vm_start for regular users, since they would be able to set arbitrary
cpu types or cpuid parameters that aren't supported.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2022-03-08 15:16:09 +01:00
Fabian Ebner
d11391ff30 qmp client: increase timeout for thaw
Using a loop of freeze, sleep 5, thaw, sleep 5, an idling Windows 11
VM with 4 cores and 8GiB RAM once took 54 seconds for thawing. It took
less than a second about 90% of the time and maximum of a few seconds
for the majortiy of other cases, but there can be outliers where 10
seconds is not enough.

And there can be hookscripts executed upon thaw, which might also not
complete instantly.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-03-03 12:12:14 +01:00
Fabian Ebner
f7765dd72e restore: remove duplicate check
The refactoring in 36d4bdcb86 missed
this. The check is already done as part of the following check_storage
call.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-02-25 09:58:16 +01:00
Moayad Almalat
39abafc8e7 fix #3733: add 20 seconds timeout when VM backup stopped 2022-02-24 13:33:45 +01:00
Thomas Lamprecht
36d4bdcb86 restore: factor out common early checks into closure
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-24 13:29:42 +01:00
Thomas Lamprecht
9f8ba326ce restore: early check cloud init target AllocateSpace permission
Originally-by: Matthias Heiserer <m.heiserer@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-24 13:24:01 +01:00
Matthias Heiserer
62af60cd3d fix #3886: QEMU restore: verify storage allows images before writing
When restoring a backup and the storage the disks would be created on
doesn't allow 'images', the process errors without cleanup.
This is the same behaviour we currently have when the storage is
disabled.

Signed-off-by: Matthias Heiserer <m.heiserer@proxmox.com>
Reviewed-by: Fabian Ebner <f.ebner@proxmox.com>
Tested-by: Fabian Ebner <f.ebner@proxmox.com>
2022-02-24 13:19:49 +01:00
Fabian Ebner
5f5aba2578 clone disk: force raw format for TPM state
The format can't be qcow2 for TPM state, because swtpm reads the file
directly expecting raw data.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-02-17 14:32:51 +01:00
Thomas Lamprecht
64cc310f0a fix #3845: also clamp cpu units to cgroup dependent valid range on hotplug
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-11 09:31:13 +01:00
Thomas Lamprecht
0c6b941e5b config2cmd: drop unused get_cpuunits call
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-11 09:30:22 +01:00
Thomas Lamprecht
e65e35ca6d cpu units: lower minimum for accessing full cgroupv2 range
also clamp that up in the v1 case for compat.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-11 09:29:52 +01:00
Thomas Lamprecht
f43d7f67bf cpu units: handle clamping more centrally
preparation for also clamping on hotplug and lower the minimum in the
schema so that the full v2 range can be used.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-11 09:28:35 +01:00
Thomas Lamprecht
4d47a9f510 api: qga file-write: drop the check for base64
it's potentially expensive to check and the user already needs to
explicitly turn auto-encoding off, besides QEMU/QGA should handle
that and just error out gracefully on bogus base64 values.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-10 16:27:57 +01:00
Thomas Lamprecht
2b607ad1ab api: qga file-write: code/style cleanups/refactoring
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-10 16:26:53 +01:00
Dominik Csapak
0cb151937f fix #3683: agent file-write: enable user to encode the content themselves
by adding an optional parameter 'encode' (enabled by default). When it
is disabled, the content must be base64 encoded already. This
way, users can send a binary file to the vm by base64 encoding it
themselves

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-02-10 16:18:20 +01:00
Fabian Grünbichler
e594231bf1 migrate: move tunnel-helpers to pve-guest-common
besides the log calls these don't need any parts of the migration state,
so let's make them generic and re-use them for container migration and
replication in the future.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-02-09 18:49:55 +01:00
Fabian Grünbichler
5668463b69 nbd alloc helper: allow passing in explicit format
and make $volname optional, to support remote migration usage without
code duplication.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-02-09 18:48:46 +01:00
Fabian Grünbichler
c9c32c1ba4 update_vm: allow simultaneous setting of boot-order and dev
else this fails if we check 'boot' before the device was put into
the config or pending section.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-02-09 18:48:29 +01:00
Fabian Grünbichler
ad5812d8fb parse_config: optional strict mode
when passing a config from one cluster to another, we want to be strict
when parsing - it's better to fail the migration early and upgrade the
target node instead of failing the migration later (when significant
work for transferring disks and/or state has already been done) or not
at all, but silently lose config settings that the target doesn't
understand.

this also might be helpful in other cases - e.g. when restoring from a
backup.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-02-09 18:47:56 +01:00
Fabian Grünbichler
dbdcc5cde0 schema: use pve-bridge-id
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-02-09 18:47:43 +01:00
Thomas Lamprecht
e0981f8b8b d/control: bump versioned (build-)dependency of libpve-common-perl
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-09 18:47:03 +01:00
Fabian Grünbichler
82a0367149 move map_storage to PVE::JSONSchema::map_id
since we are going to reuse the same mechanism/code for network bridge
mapping and pve-container.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-02-09 18:46:20 +01:00
Thomas Lamprecht
b6d59e3efb d/control: bump (build-)dependency of libpve-guest-common-perl
to get new helpers for snapshot

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-09 18:45:50 +01:00
Thomas Lamprecht
c0acd873f5 agent: code style fix
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-09 18:45:14 +01:00
Fabian Ebner
fe2c506926 snapshot: implement __snapshot_activate_storages
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-02-08 10:43:03 +01:00
Fabian Grünbichler
4df8fe45b2 clone: repeat more checks after locking
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-02-08 09:00:31 +01:00
Fabian Ebner
3b53c47155 clone disk: don't modify drive parameter
While existing callers are not using the parameter after the call,
the modification is rather unexpected and could lead to bugs quickly.

Also avoid setting an undef value in the hash, but use delete instead.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-01-31 13:10:33 +01:00
Fabian Ebner
bdf6ba1e7d api: move disk: fork before locking
using the familiar early+repeated checks pattern from other API calls.
Only intended functional changes are with regard to locking/forking.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-01-31 13:10:33 +01:00
Fabian Ebner
dbecb46f2a api: clone: fork before locking
using the familiar early+repeated checks pattern from other API calls.
Only intended functional changes are with regard to locking/forking.

For a full clone of a running VM without guest agent, this also fixes
issuing vm_{resume,suspend} calls for drive mirror completion.
Previously, those just timed out, because of not getting the lock:

> create full clone of drive scsi0 (rbdkvm:vm-104-disk-0)
> Formatting '/var/lib/vz/images/105/vm-105-disk-0.raw', fmt=raw
> size=4294967296 preallocation=off
> drive mirror is starting for drive-scsi0
> drive-scsi0: transferred 2.0 MiB of 4.0 GiB (0.05%) in 0s
> drive-scsi0: transferred 635.0 MiB of 4.0 GiB (15.50%) in 1s
> drive-scsi0: transferred 1.6 GiB of 4.0 GiB (40.50%) in 2s
> drive-scsi0: transferred 3.6 GiB of 4.0 GiB (90.23%) in 3s
> drive-scsi0: transferred 4.0 GiB of 4.0 GiB (100.00%) in 4s, ready
> all 'mirror' jobs are ready
> suspend vm
> trying to acquire lock...
> can't lock file '/var/lock/qemu-server/lock-104.conf' - got timeout
> drive-scsi0: Cancelling block job
> drive-scsi0: Done.
> resume vm
> trying to acquire lock...
> can't lock file '/var/lock/qemu-server/lock-104.conf' - got timeout

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-01-31 13:10:33 +01:00
Fabian Ebner
d6cdfae417 drive mirror monitor: warn when suspend/resume/freeze/thaw calls fail
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-01-31 13:10:33 +01:00
Nicholas Sherlock
d806b017ac pci: allow override of PCI vendor/device ids
This allows mobile- and vGPUs to be presented to the guest as if they
were the original desktop variants of the card. It also allows
device-ID variants that guests don't know about to be renamed to
match compatible sibling devices the guest does have drivers for
(e.g. to remove manufacturer-specific vendor ID variants that prevent
the use of a device which would otherwise have a supported chipset)

e.g. hostpci0: 03:00,vendor-id=0x8086,device-id=0x10f6

Signed-off-by: Nicholas Sherlock <n.sherlock@gmail.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>
2022-01-25 10:59:23 +01:00
Mira Limbeck
ea18b60455 fix #3792: cloudinit: use of uninitialized value
With the patch adding vendor-data support to cloud-init, a use of
uninitialized value was introduced. This can be fixed by setting it to
an empty string if no vendor-data is defined.

vendor-data can only be set via --cicustom and is optional.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2021-12-21 15:45:18 +01:00
Oguz Bektas
a644de298c avoid writing the config if there are no pending changes to apply
We drop properties which we do not understand and we call
`vmconfig_apply_pending` on stop and before start, so if a user tried
to edit the config or downgraded qemu-server they may get stuff
dropped from the config just by doing a stop/start, which may be a
bit too confusing, also the write is just unnecessary then.

we also have the same skipping logic when starting vms, this way we
avoid calling 'write_config' when there are no present changes to
commit.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2021-12-02 19:24:29 +01:00
Thomas Lamprecht
e3e34a10e7 bump version to 7.1-4
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-22 17:07:17 +01:00
Fabian Grünbichler
fd95d780a2 migrate: send updated TPM state volid to target node
The volid may change if local-storage migration is involved, we need
to tell the target node the new one and update the in-memory config
for starting the target VM accordingly.

Reported here: https://forum.proxmox.com/threads/99906/#post-431345

this possibly breaks migration new -> old iff
- spice is not used (else the explicit ticket wins because it comes
  later)
- a local TPM state volume is used
- that local TPM state volume has a different volume id on the target
  node (switched storage, volname already taken, ..)

because the target node will then mis-interpret the tpmstate0 line as
spice ticket and set it accordingly. if the old tpm state volume ID does
not exist on the target node, migration will fail. if it exists by
chance, it might work albeit with a wrong spice ticket (new because of
this patch) and tpm state volume (pre-existing breakage).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-11-22 16:55:17 +01:00
Fabian Grünbichler
399ca0d66e migrate: improve start STDIN-parameter parsing
only do the compat fallback if no explicit spice ticket was given, and
warn on unknown parameters on STDIN.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-11-22 16:53:09 +01:00
Thomas Lamprecht
dce63c70c8 bump version to 7.1-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-16 14:04:49 +01:00
Fabian Grünbichler
41c8671e78 migrate: skip tpmstate for NBD migration
This patch fixes the wrong attempt of setting up an NBD server for
the replicated TPM state volume, in contrast to the other volumes the
TPM state is managed by swtpm and isn't available to QEMU for
block-migration/bitmap tracking.

Note that we do migrate the state volume via a storage migration
anyway if necessary.

This code path was only triggered for replicated VMs with TPM.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-16 14:03:01 +01:00
Thomas Lamprecht
248fab7a08 bump version to 7.1-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-15 16:59:27 +01:00
Dominik Csapak
1319908f5d exclude efidisk and tpmstate for boot disk selection
else we cannot create a vm without a disk but with a tpmstate/efidisk,
since the api tries to generate the default bootorder with them included

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-15 16:57:52 +01:00
Thomas Lamprecht
701a86f267 bump version to 7.1-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-15 16:27:06 +01:00
Dominik Csapak
9c85548fa1 pci: do not reserve pci-ids for mediated devices
else a user cannot use more than one mdev per card per host.
We do not need to reserve them at all, since sysfs will error out
on creation/reuse anyway

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-15 15:01:12 +01:00
Thomas Lamprecht
ce3fbcd456 api: update: fix missing newline in background-delayed task error
this error path is mostly used for re-attaching disks and the like,
and the "check if task is already done" part uses a method to read
the task status that will never include a trailing newline, so add it
our self to avoid "... at /usr/share/perl5/PVE/API2/Qemu.pm line
1480. (500)"

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-15 09:21:50 +01:00
Oguz Bektas
bec8742495 cfg2cmd: disable SMM when display=none and SeaBIOS is both used
issue reported in community forum [0][1], like "serial[n]" display we
also need to set this option for "none", otherwise we get a boot
loop.

[0]: https://forum.proxmox.com/threads/99508
[1]: https://forum.proxmox.com/threads/97310/post-427129

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
Reviewed-by: Fabian Ebner <f.ebner@proxmox.com>
Tested-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-11 14:16:09 +01:00