Commit Graph

2971 Commits

Author SHA1 Message Date
Fabian Ebner
5f9575929a clone disk: assert that drive name is the same for drive-mirror on single VM
because when the VM ID of target and source are the same,
qemu_drive_mirror_monitor() switches the QEMU device node over to the
new backing image. The planned import-from functionality makes it
possible to run into this, although for an a bit unusual use case.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-01 10:24:59 +02:00
Fabian Ebner
f9be9137fc api: create/modify: add content type checks
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-01 09:58:01 +02:00
Fabian Grünbichler
8b8893c3c4 fix #3957: spell 'occurred' correctly
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-03-25 09:06:58 +01:00
Dominik Csapak
d8a7e9e881 PCI: allow longer pci domains
some systems[0] have pci domains longer than the default ('0000') of 4
characters, so change the regex to allow at least 4.

0: https://forum.proxmox.com/threads/problem-with-gpu-passthrough-in-a-virtual-machine.105720/

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-03-16 18:03:35 +01:00
Thomas Lamprecht
11f9264fed cpu config: code format/whitespace fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-03-16 15:53:00 +01:00
Fabian Grünbichler
1770b70fdf api: snapshot delete: prefix lock-fail error
otherwise users might get confused if they just get a message about a
migrate lock not being available..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-03-15 13:28:27 +01:00
Fabian Ebner
fdbbed2fac fix #3424: api: snapshot delete: wait for active replication
A to-be-deleted snapshot might be actively used by replication,
resulting in a not (or only partially) removed snapshot and locked
(snapshot-delete) VM. Simply wait a few seconds for any ongoing
replication.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-03-15 12:06:21 +01:00
Fabian Ebner
7979bbcd43 api: factor out check/cleanup for drive params
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-03-14 16:28:55 +01:00
Fabian Ebner
a23d57d596 image convert: allow block device as source
Necessary to import from an existing storage using block-device
volumes like ZFS.

Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
[split into its own patch]
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-03-14 16:28:55 +01:00
Fabian Ebner
84c253e947 parse ovf: untaint path when calling file_size_info
Prepare for calling parse_ovf via API, where the -T switch is used.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-03-11 10:45:59 +01:00
Fabian Ebner
6e55f5799e schema: add pve-volume-id-or-absolute-path
Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
[split into its own patch + style fixes]
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-03-11 10:45:59 +01:00
Fabian Ebner
ff84f0e36b efivars size: allow overriding efidisk parameter
For disk import, it should be based on the disk properties that are
passed in rather than on those of a possibly pre-existing disk in the
config.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-03-11 10:45:59 +01:00
Fabian Ebner
25166060ae clone disk: allow cloning from an unused or unreferenced disk
and also when source and target drivename are different. In those
cases, it is done via qemu-img convert/dd.

In preparation to allow import from existing PVE-managed disks.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-03-11 10:45:59 +01:00
Fabian Ebner
7344af7bf8 clone disk: pass in efi vars size rather than config
It's confusing that the config associated to the destination is
actually a reference to the source config for both existing callers.

Also, disk import will need to base the calculation on the passed-in
drive parameters and not just the current config, so this change is in
preparation for that too.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-03-11 10:45:59 +01:00
Fabian Ebner
1196086ff1 clone disk: group source and target parameters
to make the interface more digestible.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-03-11 10:45:59 +01:00
Fabian Ebner
31b1db61c2 clone disk: remove check for min QEMU version 2.7
Upgrading a cluster node entails re-starting or migrating VMs and even
PVE 6.0 already had QEMU 4.0.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-03-11 10:45:59 +01:00
Fabian Ebner
367e6bf4b7 api: update: pass correct config when creating disks
While the new options should be written to the pending config, the
decisions (currently only one) in create_disks needs to be made for
the current config.

Seems to fix EFI disk creation, but actually, it's only
future-proofing, because, currently, the same OVMF_VARS file is
used independently of $smm.

The correct config is also needed to determine the correct size for
the EFI disk for the upcoming import-from feature.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-03-11 10:45:59 +01:00
Fabian Ebner
fe19840a56 api: create disks: always activate/update size when attaching existing volume
For creation, activation and size update never triggered, because the
passed in $conf is essentially the same as the creation $settings, so
the disk was always detected to be the same as the "existing" one. But
actually, all disks are new, so it makes sense to do it.

For update, activation and size update nearly always triggered,
because only the pending changes are passed in as $conf. The case
where it didn't trigger is when the same pending change was made twice
(there are cases where hotplug isn't done, but makes it even more
unlikely).

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-03-11 10:45:59 +01:00
Fabian Ebner
52b361afd6 device unplug: verify that unplugging scsi disk completed
Avoids the error
  adding drive failed: Duplicate ID 'drive-scsi1' for drive
that could happen when switching over to a new disk (e.g. via qm set),
if unplugging wasn't fast enough.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-03-11 10:45:59 +01:00
Oguz Bektas
6ab41628b2 api: vm_start: 'force-cpu' is for internal migration use only
'force-cpu' parameter was introduced to allow live-migration of VMs with
custom CPU models; it does not need to be allowed for general use on
vm_start for regular users, since they would be able to set arbitrary
cpu types or cpuid parameters that aren't supported.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2022-03-08 15:16:09 +01:00
Fabian Ebner
d11391ff30 qmp client: increase timeout for thaw
Using a loop of freeze, sleep 5, thaw, sleep 5, an idling Windows 11
VM with 4 cores and 8GiB RAM once took 54 seconds for thawing. It took
less than a second about 90% of the time and maximum of a few seconds
for the majortiy of other cases, but there can be outliers where 10
seconds is not enough.

And there can be hookscripts executed upon thaw, which might also not
complete instantly.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-03-03 12:12:14 +01:00
Fabian Ebner
f7765dd72e restore: remove duplicate check
The refactoring in 36d4bdcb86 missed
this. The check is already done as part of the following check_storage
call.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-02-25 09:58:16 +01:00
Moayad Almalat
39abafc8e7 fix #3733: add 20 seconds timeout when VM backup stopped 2022-02-24 13:33:45 +01:00
Thomas Lamprecht
36d4bdcb86 restore: factor out common early checks into closure
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-24 13:29:42 +01:00
Thomas Lamprecht
9f8ba326ce restore: early check cloud init target AllocateSpace permission
Originally-by: Matthias Heiserer <m.heiserer@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-24 13:24:01 +01:00
Matthias Heiserer
62af60cd3d fix #3886: QEMU restore: verify storage allows images before writing
When restoring a backup and the storage the disks would be created on
doesn't allow 'images', the process errors without cleanup.
This is the same behaviour we currently have when the storage is
disabled.

Signed-off-by: Matthias Heiserer <m.heiserer@proxmox.com>
Reviewed-by: Fabian Ebner <f.ebner@proxmox.com>
Tested-by: Fabian Ebner <f.ebner@proxmox.com>
2022-02-24 13:19:49 +01:00
Fabian Ebner
5f5aba2578 clone disk: force raw format for TPM state
The format can't be qcow2 for TPM state, because swtpm reads the file
directly expecting raw data.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-02-17 14:32:51 +01:00
Thomas Lamprecht
64cc310f0a fix #3845: also clamp cpu units to cgroup dependent valid range on hotplug
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-11 09:31:13 +01:00
Thomas Lamprecht
0c6b941e5b config2cmd: drop unused get_cpuunits call
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-11 09:30:22 +01:00
Thomas Lamprecht
e65e35ca6d cpu units: lower minimum for accessing full cgroupv2 range
also clamp that up in the v1 case for compat.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-11 09:29:52 +01:00
Thomas Lamprecht
f43d7f67bf cpu units: handle clamping more centrally
preparation for also clamping on hotplug and lower the minimum in the
schema so that the full v2 range can be used.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-11 09:28:35 +01:00
Thomas Lamprecht
4d47a9f510 api: qga file-write: drop the check for base64
it's potentially expensive to check and the user already needs to
explicitly turn auto-encoding off, besides QEMU/QGA should handle
that and just error out gracefully on bogus base64 values.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-10 16:27:57 +01:00
Thomas Lamprecht
2b607ad1ab api: qga file-write: code/style cleanups/refactoring
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-10 16:26:53 +01:00
Dominik Csapak
0cb151937f fix #3683: agent file-write: enable user to encode the content themselves
by adding an optional parameter 'encode' (enabled by default). When it
is disabled, the content must be base64 encoded already. This
way, users can send a binary file to the vm by base64 encoding it
themselves

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-02-10 16:18:20 +01:00
Fabian Grünbichler
e594231bf1 migrate: move tunnel-helpers to pve-guest-common
besides the log calls these don't need any parts of the migration state,
so let's make them generic and re-use them for container migration and
replication in the future.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-02-09 18:49:55 +01:00
Fabian Grünbichler
5668463b69 nbd alloc helper: allow passing in explicit format
and make $volname optional, to support remote migration usage without
code duplication.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-02-09 18:48:46 +01:00
Fabian Grünbichler
c9c32c1ba4 update_vm: allow simultaneous setting of boot-order and dev
else this fails if we check 'boot' before the device was put into
the config or pending section.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-02-09 18:48:29 +01:00
Fabian Grünbichler
ad5812d8fb parse_config: optional strict mode
when passing a config from one cluster to another, we want to be strict
when parsing - it's better to fail the migration early and upgrade the
target node instead of failing the migration later (when significant
work for transferring disks and/or state has already been done) or not
at all, but silently lose config settings that the target doesn't
understand.

this also might be helpful in other cases - e.g. when restoring from a
backup.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-02-09 18:47:56 +01:00
Fabian Grünbichler
dbdcc5cde0 schema: use pve-bridge-id
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-02-09 18:47:43 +01:00
Thomas Lamprecht
e0981f8b8b d/control: bump versioned (build-)dependency of libpve-common-perl
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-09 18:47:03 +01:00
Fabian Grünbichler
82a0367149 move map_storage to PVE::JSONSchema::map_id
since we are going to reuse the same mechanism/code for network bridge
mapping and pve-container.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-02-09 18:46:20 +01:00
Thomas Lamprecht
b6d59e3efb d/control: bump (build-)dependency of libpve-guest-common-perl
to get new helpers for snapshot

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-09 18:45:50 +01:00
Thomas Lamprecht
c0acd873f5 agent: code style fix
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-09 18:45:14 +01:00
Fabian Ebner
fe2c506926 snapshot: implement __snapshot_activate_storages
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-02-08 10:43:03 +01:00
Fabian Grünbichler
4df8fe45b2 clone: repeat more checks after locking
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-02-08 09:00:31 +01:00
Fabian Ebner
3b53c47155 clone disk: don't modify drive parameter
While existing callers are not using the parameter after the call,
the modification is rather unexpected and could lead to bugs quickly.

Also avoid setting an undef value in the hash, but use delete instead.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-01-31 13:10:33 +01:00
Fabian Ebner
bdf6ba1e7d api: move disk: fork before locking
using the familiar early+repeated checks pattern from other API calls.
Only intended functional changes are with regard to locking/forking.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-01-31 13:10:33 +01:00
Fabian Ebner
dbecb46f2a api: clone: fork before locking
using the familiar early+repeated checks pattern from other API calls.
Only intended functional changes are with regard to locking/forking.

For a full clone of a running VM without guest agent, this also fixes
issuing vm_{resume,suspend} calls for drive mirror completion.
Previously, those just timed out, because of not getting the lock:

> create full clone of drive scsi0 (rbdkvm:vm-104-disk-0)
> Formatting '/var/lib/vz/images/105/vm-105-disk-0.raw', fmt=raw
> size=4294967296 preallocation=off
> drive mirror is starting for drive-scsi0
> drive-scsi0: transferred 2.0 MiB of 4.0 GiB (0.05%) in 0s
> drive-scsi0: transferred 635.0 MiB of 4.0 GiB (15.50%) in 1s
> drive-scsi0: transferred 1.6 GiB of 4.0 GiB (40.50%) in 2s
> drive-scsi0: transferred 3.6 GiB of 4.0 GiB (90.23%) in 3s
> drive-scsi0: transferred 4.0 GiB of 4.0 GiB (100.00%) in 4s, ready
> all 'mirror' jobs are ready
> suspend vm
> trying to acquire lock...
> can't lock file '/var/lock/qemu-server/lock-104.conf' - got timeout
> drive-scsi0: Cancelling block job
> drive-scsi0: Done.
> resume vm
> trying to acquire lock...
> can't lock file '/var/lock/qemu-server/lock-104.conf' - got timeout

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-01-31 13:10:33 +01:00
Fabian Ebner
d6cdfae417 drive mirror monitor: warn when suspend/resume/freeze/thaw calls fail
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-01-31 13:10:33 +01:00
Nicholas Sherlock
d806b017ac pci: allow override of PCI vendor/device ids
This allows mobile- and vGPUs to be presented to the guest as if they
were the original desktop variants of the card. It also allows
device-ID variants that guests don't know about to be renamed to
match compatible sibling devices the guest does have drivers for
(e.g. to remove manufacturer-specific vendor ID variants that prevent
the use of a device which would otherwise have a supported chipset)

e.g. hostpci0: 03:00,vendor-id=0x8086,device-id=0x10f6

Signed-off-by: Nicholas Sherlock <n.sherlock@gmail.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>
2022-01-25 10:59:23 +01:00