Commit Graph

2971 Commits

Author SHA1 Message Date
Thomas Lamprecht
a83b2330fc bump version to 7.0-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-21 11:18:39 +02:00
Fabian Ebner
24b84b4766 migrate: enforce that image content type is available
and use it for the vdisk_list call too. This avoids scanning (and picking up
volumes from!) storages that are not even configured to hold images.

Previously, the content type was only enforced when a storage map was present.

Also serves a bit as a preparation to enforce content type on guest startup,
because now migration failure happens early and not only when trying to start
the guest on the remote node.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-06-21 11:17:48 +02:00
Fabian Ebner
0d2db08414 prefer storage_check_enabled over storage_check_node
storage_check_enabled simply checks for the 'disable' option and then calls
storage_check_node.

While not strictly necessary for a second call where only the storage differs,
e.g. in case of clone, it is more future-proof: if support for a target storage
is added at some point, it might be easy to miss adapting the call.

For the migration checks, the situation is improved by now always catching
disabled (target) storages.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-06-21 11:17:48 +02:00
Fabian Ebner
cc1cdadbf4 test: fix restore config test as unprivileged user
after upgrading to bullseye, the cfs_read_file call within
restore_update_config_line() results in an error:
    Is a directory!
when done as an unprivileged user.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-06-18 17:26:24 +02:00
Fabian Ebner
8a0addab87 vmstatus: don't set PID when VM is not running
by avoiding int(undef)

Reported-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-06-18 14:05:35 +02:00
Thomas Lamprecht
a200af1084 config: limit description/comment length to 8 KiB
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-18 13:11:07 +02:00
Fabian Ebner
ad2cad72be vm status: force int where appropriate
to avoid potential problems with stringified numbers in Javascript and
elsewehere.

The vmid was not always an integer as the return schema expects, namely
when there was an opt_vmid argument, because the 'ne' comparision coerced the
vmid to be a string then.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-06-17 14:47:35 +02:00
Fabian Ebner
ef88eaaa58 avoid setting lun number for drives when pvscsi controller is used
Reported in the community forum[0].

In QEMU's hw/scsi/vmw_pvscsi.c in the SCSIBusInfo struct, the max_lun property
is set to 0. This means that in our stack, one cannot have multiple disks and
use 'scsihw: pvscsi' currently, as kvm would fail with
    bad scsi device lun: 1

Instead of increasing the lun number, increase the scsi-id, as we already do for
lsi.* (in hw/scsi/lsi53c895a.c the max_lun property is also 0).

[0]: https://forum.proxmox.com/threads/kvm-bad-scsi-device-lun-1.84318/

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Reviewed-by: Stefan Reiter <s.reiter@proxmox.com>
Tested-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-16 20:26:27 +02:00
Thomas Lamprecht
9da0feb5e5 cfg2cmd: add test for efidisk rbd cache handling
I don't think this is something which will get broken by accident but
still nice to "document" this behavior in a regression test

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-16 15:24:57 +02:00
Thomas Lamprecht
26d717252a followup; shorter code for efidisk rbd cache handling
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-16 15:18:03 +02:00
Dominik Csapak
6aaad2306d fix #3329: turn on cache=writeback for efidisks on rbd
on slower ceph clusters, the write pattern of the ovmf booting process
slows down the boot of the vm, so we turn on caching by default

it seems no other storage (until now) behaves like this. if it does in
the future, we can still add them too, or add a 'cache' property for
the efidisk

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-16 15:14:30 +02:00
Fabian Ebner
16e66777a0 vm destroy: do not remove unreferenced disks by default
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Reviewed-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-08 14:56:39 +02:00
Fabian Ebner
9a8ba1272c scan volids: remove superfluous parameter
The only caller that didn't use 'images' was removed as part of the migration
refactoring in commit 62a4c963b8, so this is not
even a breaking change as the 'PVE 7' comment might've suggested.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Reviewed-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-08 14:56:30 +02:00
Fabian Ebner
692f604bb0 Revert "revert spice_ticket prefix change in 7827de4"
This reverts commit ff09c795ed. We wanted to wait
until PVE 7.0 for the change to not break migration new -> old until then.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Reviewed-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-08 14:56:10 +02:00
Thomas Lamprecht
e41fb93e17 buildsys: change upload dist to bullseye
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-08 09:47:35 +02:00
Fabian Grünbichler
9bf522bc1e vzdump: add master key support
running outdated VMs without master key support will generate a warning
but proceed with a backup without encrypted key upload.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-06-02 16:49:00 +02:00
Thomas Lamprecht
7908e50263 vzdump: drop legacy fallback logging for dirty-bitmap
Users need to reboot at least once for the upgrade to 7.0, so any VM
running is then using a new enough QEMU...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-02 16:48:18 +02:00
Thomas Lamprecht
30aa458010 bump version to 7.0-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-28 13:08:50 +02:00
Thomas Lamprecht
daf829ecae live-restore: merge snapshot/repo log lines into one
to many lines make task log harder to read

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-28 12:14:37 +02:00
Stefan Reiter
88cabb624d live-restore: add more logging
To bring it better in line with regular restore, also log the
repository, the snapshot and the target for each drive.

While at it, adjust capitalization of existing log line and clean up
repeated '$1' use.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-28 12:12:50 +02:00
Fabian Ebner
3ab0f9252a destroy VM: also check if unused volumes are base images
It's arguably not likely in practice that only an unused volume is still in use
as a base image, but do it for completeness sake.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-05-28 12:12:50 +02:00
Fabian Ebner
ba1a198481 destroy VM: always remove (referenced) VM state volumes
With --destroy-unreferenced-disks 0 they were not removed yet, but no use in
keeping them around.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-05-28 12:12:50 +02:00
Stefan Reiter
2dc0eb61e8 qm: assume correct VNC setup in 'vncproxy', disallow passwordless
The QMP 'change' command is no longer available since QEMU 6.0, so this
cannot work - instead of replacing it, we can just remove it however.

The 'if' branch would only set the VNC socket path anew and enable
password mode, which is always set and enabled on startup already.
The 'else' branch was intended for certificate login (?), which
according to the FIXME comment is long gone anyway - simply forbid
'vncproxy' without the PVE ticket environment variable set.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-28 11:31:15 +02:00
Stefan Reiter
378ad769dd cfg2cmd: use long form QEMU parameters to avoid warning in 6.0
QEMU warns us about this:

kvm: -chardev socket,id=qmp,path=/var/run/qemu-server/100.qmp,server,nowait: warning: short-form boolean option 'server' deprecated
Please use server=on instead
kvm: -chardev socket,id=qmp,path=/var/run/qemu-server/100.qmp,server,nowait: warning: short-form boolean option 'nowait' deprecated
Please use wait=off instead
kvm: -vnc unix:/var/run/qemu-server/100.vnc,password: warning: short-form boolean option 'password' deprecated
Please use password=on instead

The new syntax is backwards compatible to at least QEMU 4.0.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-28 11:31:15 +02:00
Fabian Ebner
75a2a42395 vmstatus: make template property optional
to avoid printing 'template: ' with 'qm status <id> --verbose' if it's false.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-05-26 17:40:28 +02:00
Lorenz Stechauner
1cb23b87b4 api: clone: sort vm disks to keep numbers consistent
reported by user in forum:
https://forum.proxmox.com/threads/problem-when-copying-template-with-2-discs.89851/

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
2021-05-26 17:32:52 +02:00
Thomas Lamprecht
c09667bc9d bump version to 7.0-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-13 19:11:22 +02:00
Thomas Lamprecht
b818a0504c debian: update compat, copyright, ..
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-13 19:09:29 +02:00
Thomas Lamprecht
c948e143d8 bump version to 6.4-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-29 16:16:51 +02:00
Fabian Grünbichler
3dc33a728a fix #2862: allow sata/ide template backups
for IDE and SATA, setting the whole drive into readonly mode is not
possible. skip the readonly flag for such drives as a workaround until
we find a better solution.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-04-29 16:15:53 +02:00
Dominik Csapak
30664f14ff fix bootdisk_size for new bootorder config scheme
Previously, we ever only had a single boot *disk*, while possibly
having multiple cdroms/nics in the boot order

e.g. the config:

 boot: dnc
 bootdisk: scsi0
 ide0: media=cdrom,none
 scsi0: xxx
 net0: ...

would return the size of scsi0 even though it would first boot
from cdrom/network.

When editing the bootorder with such a legacy config, we
remove the 'bootdisk' property and replace the legacy notation
with an explicit order, but we only search the first disk
for the size now.

Restore that behaviour by iterating over all disks in the boot
order property string until we get one that is not a cdrom
and has a size.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-29 16:15:33 +02:00
Thomas Lamprecht
8f43ac4893 Revert "migration: do not set default speed limit"
The default was changed for 5.2, so while it is not 32 MiB/s anymore,
it is still 128 MiB/s which I did not notice on my 1 Gbps (or < 125
MiB/s) setup. For users with links faster than one gigabit it now did
some limiting - so setup a very high limit so than even 100G should
not max this out.

This reverts commit a89bd10084.
2021-04-29 15:48:21 +02:00
Thomas Lamprecht
ba2a06fda2 bump version to 6.4-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-23 16:26:59 +02:00
Fabian Ebner
9938d24df2 migrate: fix memory migration start time
The variable is only ever used for calculating the average speed of memory
migration, but it was set before disk mirroring already. But the disk
sizes are not included in the calculation, resulting in (very) wrong values.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-04-23 15:00:44 +02:00
Fabian Ebner
6629f976ac qemu_img_convert: add missing newline for progress output
which was accidentally removed by b5e9d97bdf.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-04-22 11:37:00 +02:00
Dylan Whyte
ebce523987 fix #3369: auto-start vm after failed stopmode backup
Fixes an issue in which a VM/CT fails to automatically restart after a
failed stop-mode backup.

Also fixes a minor typo in a comment

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
Reviewed-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-04-22 11:35:20 +02:00
Stefan Reiter
f755117071 live-restore: hold 'create' lock during operation
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-21 17:46:05 +02:00
Stefan Reiter
fefd65a1d9 live-restore: don't remove VM on error
Potentially an admin can still recover some data, or wants to inspect
the state.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-21 17:46:05 +02:00
Stefan Reiter
020dd358f9 qmrestore: add live-restore option
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-21 17:46:05 +02:00
Fabian Ebner
ab5b97d8a8 drive: volume in-use check: remove unused closure parameter
and simplify the calling iteration.

Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-04-19 22:09:17 +02:00
Thomas Lamprecht
b68a957b2e migration: keep log rate steady if polling gets more frequent
Either we're done in a few seconds anyway, or if the VM dirties lots
of pages we need quite a bit of time, and then it does not help to
output roughly the same status 10 times a second...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-19 22:08:19 +02:00
Thomas Lamprecht
0fca250af0 migration: rework logging to more humand friendly, less spammy
* use render_bytes where possible, to get quick to read and grasp
  units printed
* xbzrle is only interesting if actually pages/bytes are send using
  it, so only log in that case
* log if VM dirties more than we send
* log current speed we get from QEMU

In general there are less lines logged and huge integers are avoided.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-19 21:54:37 +02:00
Thomas Lamprecht
e693c49190 migration: factor out variable + code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-19 21:51:21 +02:00
Thomas Lamprecht
7de328c629 migration: log: s/migration_caps/migration capabilities/
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-19 21:48:31 +02:00
Thomas Lamprecht
a89bd10084 migration: do not set default speed limit
the claim that QEMU limits this to 32M otherwise is bogus, at least
with any current QEMU version..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-19 21:46:52 +02:00
Thomas Lamprecht
6539865a9d migration: refactor and tidy-up code
Use an early die so that the rest can loose an indentation level for
the actual migration status reporting code

Extract common used members of the stat hash for shorter code.

use `git show -w --word-diff=color --word-diff-regex='\w+'` for
getting a better view of actual changes

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-19 14:59:54 +02:00
Fabian Ebner
0783c3c271 migration: move finishing block jobs to phase2 for better/uniform error handling
avoids the possibility to die during phase3_cleanup and instead of needing to
duplicate the cleanup ourselves, benefit from phase2_cleanup doing so.

The duplicate cleanup was also very incomplete: it didn't stop the remote kvm
process (leading to 'VM already running' when trying to migrate again
afterwards), but it removed its disks, and it didn't unlock the config, didn't
close the tunnel and didn't cancel the block-dirty bitmaps.

Since migrate_cancel should do nothing after the (non-storage) migrate process
has completed, even that cleanup step is fine here.

Since phase3 is empty at the moment, the order of operations is still the same.

Also add a test, that would complain about finish_tunnel not being called before
this patch. That test also checks that local disks are not already removed
before finishing the block jobs.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-04-18 18:30:41 +02:00
Fabian Ebner
a6be63ac9b migration: split out replication from scan_local_volumes
and avoid one loop over the config, by extending foreach_volid to include the
drivename.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-04-18 18:30:41 +02:00
Fabian Ebner
4b26ffbfa5 migration: keep track of replicated volumes via local_volumes
by extending filter_local_volumes.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-04-18 18:30:41 +02:00
Fabian Ebner
efe0d457c6 migration: use storage_migration for checks instead of online_local_volumes
Like this we don't need to worry about auto-vivifaction.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-04-18 18:30:41 +02:00