Commit Graph

2302 Commits

Author SHA1 Message Date
Fabian Grünbichler
a4c7029dd8 api/destroy: repeat early checks after lock
to protect against concurrent changes

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-05-04 17:33:53 +02:00
Fabian Grünbichler
8284121451 QemuServer: drop unused imported locking functions
lock_file is used by PVE::QemuServer::Memory, but it does properly 'use
PVE::Tools ...' itself so we can drop them in the main module.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-05-04 17:33:32 +02:00
Thomas Lamprecht
1bd6fe132f vzdump: assemble: fix skipping all pending and snapshot config entries
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-04 17:29:09 +02:00
Fabian Grünbichler
fa38a81217 bump version to 6.1-17
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-05-04 14:12:21 +02:00
Fabian Grünbichler
1e68f42199 d/control: bump build-depend on pve-storage
with refactored archive/compressor info and RE

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-05-04 14:09:36 +02:00
Alwin Antreich
f43a4f127c Fix #2124: Add support for zstd
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-05-04 11:19:35 +02:00
Alwin Antreich
c6d517835a restore: replace archive format/compression
regex to reduce the code duplication, as archive_info and
decompressor_info provides the same information as well.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-05-04 11:19:35 +02:00
Thomas Lamprecht
cb521f2d88 backup: never try to freeze in stop mode backup
VM was can be true for stop mode backup, we cannot check the "is VM
currently running" as that doesn't tells us anything (could be the
backup process), so check the mode also..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-03 11:13:38 +02:00
Thomas Lamprecht
d60cbb97ec _do_vm_stop: only warn if QMP command failed but VM is actually stopped
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-03 11:12:54 +02:00
Thomas Lamprecht
0433e98f23 bump version to 6.1-16
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-04-29 16:23:26 +02:00
Thomas Lamprecht
aa6ebf6a4b qm nbdstop: cope graceful with errors
as the nbd server could have been stopped by something else.
Further, it makes no sense to die and mark the migration thus as
failed, just because of a NBD server stop issue.

At this point the migration hand off to the target was done already,
so normally we're good, if it fails we have other (followup) problems
anyway.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-04-29 16:22:33 +02:00
Thomas Lamprecht
3e802221e1 migrate: only stop NBD if we got a NBD url from the target
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-04-29 16:22:33 +02:00
Fabian Ebner
0b953b8e34 cleanup: get rid of unnecessary closures
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-04-29 13:49:46 +02:00
Thomas Lamprecht
0091afc795 bump version to 6.1-15
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-04-29 13:37:40 +02:00
Fabian Ebner
ae180b8f08 Include vmstate and unused volumes in foreach_volid
and refactor the test_volid closure. Like this get_replicatable_volumes doesn't
need a separate loop for unused volumes anymore. For get_vm_volumes, which is used
for activation/deactivation of volumes at migration and deactivation in vm_stop_cleanup,
includes those volumes now. For migration it's an improvement, because those volumes
might need to be migrated and for vm_stop_cleanup it shouldn't hurt. The last user
of foreach_volid is check_vm_disks_local used by migrate_vm_precondition,
where information about the additional volumes doesn't hurt either.

Note that replicate is (still) set by default, so the behavior for
get_replicatable_volumes for unused volumes should not change.

Hibernation vmstate files are now also included and recognized as 'is_vmstate'.
The 'size' attribute will not be overwritten by subsequent iterations for the
same volid anymore (a volid may appear both in the config and in snapshots),
so the size from the current config is now preferred.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-04-29 12:14:40 +02:00
Fabian Ebner
b24f07d406 Fix test_volid call for vmstate and fix check for snapshots on migration
by excluding vmstate. It is referenced by snapshots, but
is not a volume containing a snapshot.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-04-29 12:14:40 +02:00
Thomas Lamprecht
5364990d26 restore: small code cleanup for parse_backup_hints
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-04-29 11:05:19 +02:00
Dominik Csapak
30a1155658 restore: use correct storage for format check for cloudinit drives
when a backup includes a cloudinit disk on a non-existent storage,
the restore fails with 'storage' does not exist

this happens because we want to get the format of the disk, by
checking the source storage

we fix this by using the target storage first and only the source as
fallback

this will still fail if neither storage exists
(which is ok, since we cannot restore then anyway)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-04-29 10:33:45 +02:00
Dominik Csapak
48caa553ab importovf: die with error when disk file is missing
Also add missing '\n' at the end of error messages

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-04-27 13:13:10 +02:00
Dominik Csapak
afc910a829 importovf: fix import of ovfs without default namespaces
Some OVF files to not declare 'rasd' as a default namespace (in the
top level Envelope element), but inline in each element (e.g.
<rasd:HostResource xmlns:rasd="foo">...</rasd:HostResource>)

This trips up our relative findvalue with
> XPath error : Undefined namespace prefix

To avoid this, search in the global XPathContext (where we register
those namespaces ourselves) and pass the item_node as context
parameter.

This works then for both cases

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-04-27 13:12:49 +02:00
Dominik Csapak
39322a9341 test: add test for OVF with missing default rasd namespace
sometimes vendors do not put the 'rasd' namespaces in the top level
Envelope, but in every 'rasd' element this adds a test for this

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-04-27 13:09:51 +02:00
Dominik Csapak
31bf5a0f2b test: print more info when OVF parsing fails
When one of the ovf tests fails to parse at all, we just get the
'die' message of the failing component, but not which file actually
failed to parse.

To get better output, convert the parsing also to a test and ok() and
fail() respectively and then printing the error.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-04-27 13:08:58 +02:00
Fabian Grünbichler
4fb85adc65 api/resume: make nocheck root-only
this is only used for migration via 'qm mtunnel', regular users should
never need to resume a VM that does not logically belong to the node it
is running on

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-04-27 13:07:54 +02:00
Fabian Grünbichler
90ff65b63a migrate: simplify replicated_volume loop
(no change compared to previous iteration except for readability)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-04-20 11:24:23 +02:00
Fabian Ebner
cee620e671 Fix live migration with replicated unused volumes
by counting only local volumes that will be live-migrated via qemu_drive_mirror,
i.e. those listed in $self->{online_local_volumes}.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-04-20 11:12:56 +02:00
Thomas Lamprecht
38311a1d17 migrate: workaround issues with format switch on storage live migration
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-04-17 15:27:38 +02:00
Thomas Lamprecht
b01de19922 cfg2cmd: factor out audio device generation
and version guard it with 4.0 machines, just do be sure

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-04-17 15:20:13 +02:00
Aaron Lauterer
940e2a3a06 cfg2cmd: Add the audiodev property to audio devs
With Qemu 4.2 a new `audiodev` property was introduced [0] to explicitly
specify the backend to be used for the audio device. This is accompanied
with a warning that the fallback to the default audio backend is
deprecated.

[0] https://wiki.qemu.org/ChangeLog/4.2#Audio

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-04-17 14:23:16 +02:00
Thomas Lamprecht
818c3b8d91 cfg2cmd: ovmf: code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-04-15 18:30:53 +02:00
Fabian Grünbichler
89b0461af1 bump version to 6.1-14
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-04-09 09:36:41 +02:00
Fabian Grünbichler
4fcfd15d52 d/control: depend on updated guest-common/pve-storage
pve-storage: new storage_migrate with allow_rename
guest-common: update_volume_ids

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-04-09 09:36:41 +02:00
Fabian Ebner
ea5b400812 sync_disks: log output of storage_migrate
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-04-08 22:11:54 +02:00
Fabian Ebner
49a5a0d84b sync_disks: be more verbose if storage_migrate fails
If storage_migrate dies, the error message might not include the
volume ID or the target storage ID, but those might be good to know.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-04-08 22:11:54 +02:00
Fabian Ebner
cc1a3820db sync_disks: use allow_rename to avoid collisions on the target storage
This makes it possible to migrate a VM with volumes store1:vm-123-disk-0
store2:vm-123-disk-0 to some targetstorage. Also prevents migration failure
when there is an orphaned disk with the same volid on the target.

To avoid confusion, the name should not change for 'vmstate'-volumes.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-04-08 22:11:54 +02:00
Fabian Ebner
97ece9ddce Update volume IDs in one go
Use 'update_volume_ids' for the live-migrated disks as well.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-04-08 22:11:54 +02:00
Fabian Ebner
bcd5401acc Allow specifying targetstorage for offline migration
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-04-08 22:11:54 +02:00
Fabian Ebner
37666e4caa Take note of changes to the volume IDs when migrating and update the config
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-04-08 22:11:54 +02:00
Fabian Ebner
092538833e Allow parsing vmstate entries
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-04-08 22:11:54 +02:00
Fabian Ebner
1f726e0a85 Use new storage_migrate interface
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-04-08 22:11:54 +02:00
Fabian Ebner
912792e245 Switch to using foreach_volume instead of foreach_drive
It was necessary to move foreach_volid back to QemuServer.pm

In VZDump/QemuServer.pm and QemuMigrate.pm the dependency on
QemuConfig.pm was already there, just the explicit "use" was missing.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-04-08 22:11:54 +02:00
Thomas Lamprecht
e5118d9b91 bump version to 6.1-13
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-04-08 17:44:38 +02:00
Thomas Lamprecht
2f6f002c04 config_to_command: indentation, whitspace and newline fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-04-08 17:44:37 +02:00
Stefan Reiter
b516c84871 Ignore version checks when using QEMU -rc releases
Upstream marks these as having a micro-version of >=90, unfortunately the
machine versions are bumped earlier so testing them is made unnecessarily
difficult, since the version checking code would abort on migrations etc...

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
[ Thomas: do so refactor ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-04-08 17:44:37 +02:00
Thomas Lamprecht
da75afe5c0 d/control: bump versioned to libpve-common-perl
mainly for the new idmap and storage map format

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-04-08 17:13:18 +02:00
Thomas Lamprecht
8cea210f99 cpu config: add upcoming EPYC-Rome CPU type
will be included with QEMU 5.0

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-04-08 17:06:52 +02:00
Fabian Grünbichler
e1cfa02e62 config: harmonize bridge pattern
so that pve-container and qemu-server use the same one, in preparation
for moving it to JSONSchema and having a bridgepair format.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-04-08 14:11:33 +02:00
Stefan Reiter
5bc084707b cfg2cmd: add test cases for custom CPU models
Requires a mock CPU-model config, which is given as a raw string to also
test parsing capabilities. Also tests defaulting behaviour.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Reviewed-By: Fabian Ebner <f.ebner@proxmox.com>
Tested-By: Fabian Ebner <f.ebner@proxmox.com>
2020-04-07 17:27:58 +02:00
Stefan Reiter
9f9792d3dc fix #2318: allow phys-bits CPU setting
Can be specified for a particular VM or via a custom CPU model (VM takes
precedence).

QEMU's default limit only allows up to 1TB of RAM per VM. Increasing the
physical address bits available to a VM can fix this.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-04-07 17:27:58 +02:00
Stefan Reiter
c4581b9cc5 Rework get_cpu_options and allow custom CPU models
If a cputype is custom (check via prefix), try to load options from the
custom CPU model config, and set values accordingly.

While at it, extract currently hardcoded values into seperate sub and add
reasonings.

Since the new flag resolving outputs flags in sorted order for
consistency, adapt the test cases to not break. Only the order is
changed, not which flags are present.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Reviewed-By: Fabian Ebner <f.ebner@proxmox.com>
Tested-By: Fabian Ebner <f.ebner@proxmox.com>
2020-04-07 17:27:58 +02:00
Stefan Reiter
45619185a6 Add helpers to better structure CPU option handling
To avoid hardcoding even more CPU-flag related things for custom CPU
models, introduce a dynamic approach to resolving flags.

resolve_cpu_flags takes a list of hashes (as documented in the
comment) and resolves them to a valid "-cpu" argument without
duplicates. This also helps by providing a reason why specific CPU flags
have been added, and thus allows for useful warning messages should a
flag be overwritten by another.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Reviewed-By: Fabian Ebner <f.ebner@proxmox.com>
Tested-By: Fabian Ebner <f.ebner@proxmox.com>
2020-04-07 17:27:58 +02:00