Commit Graph

2337 Commits

Author SHA1 Message Date
Thomas Lamprecht
48343b3f1d add runs_at_least_qemu_version to check if we can backup IOThread disks
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-23 10:47:45 +02:00
Thomas Lamprecht
19e9b30895 introduce version_cmp helper for qemu_machine_feature_enabled
will be reused for a "running KVM/QEMU version is at least" helper in
a next patch

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-23 10:38:16 +02:00
Thomas Lamprecht
d610b14591 [no-change] sort and group module use
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-23 10:36:46 +02:00
Thomas Lamprecht
8266bc59db Revert "fix #1071: VMs with IOThread enabled disks can now be backed up"
This reverts commit 6b4b369fe3.
2019-10-23 09:31:51 +02:00
Thomas Lamprecht
93a7923868 bump version to 6.0-12
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-22 16:31:53 +02:00
Thomas Lamprecht
24d1f93a84 fixup: vmstate: pass volid not resolved path to vollist
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-22 16:31:16 +02:00
Thomas Lamprecht
929bb379c9 bump version to 6.0-11
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-22 12:50:23 +02:00
Thomas Lamprecht
d321c4a921 followup: iterate over pending changes sorted
for a more deterministic behavior, should not change things in
practice

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-22 12:47:19 +02:00
Oguz Bektas
fb4d1ba27e pending apply/hotplug: don't hard code force to true
Each pending options has a hash value which has the 'force'
information encoded as entry. But, this can be { force => 1 } or
{ force => 0 }, so we actually need to check the value and not just
set force to the hash directly, as else we have force always truthy..

fixes a bug where 'detach' caused disks to be destroyed immediately,
because $force parameter was always true since hash is true.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-22 12:45:51 +02:00
Thomas Lamprecht
6b4b369fe3 fix #1071: VMs with IOThread enabled disks can now be backed up
Thanks to Dietmars patch[0] those VMs can now be backed up
successfully, so remove this aborting check.

[0]: https://git.proxmox.com/?p=pve-qemu.git;a=commit;h=69cb18950a705b54f438f4659b603b3f52901c2f

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Tested-By: Dominik Csapak <d.csapak@proxmox.com>
2019-10-22 11:53:58 +02:00
Thomas Lamprecht
edcbf953ab fixup: VM statefile: pass volid not resolved path to vollist
We cannot activate a path, only volume IDs with activate_volumes
(duh)

fixes commit 5c1d42b7f8

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-22 11:53:58 +02:00
Thomas Lamprecht
9142d2e3f0 actually bump version to 6.0-10
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-18 22:04:59 +02:00
Thomas Lamprecht
90c6002536 bump version to 6.0-10
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-18 21:58:19 +02:00
Mira Limbeck
9a13f0fed3 cloudinit: fix vm start hanging with disk on ZFS
With the changes to pve-storage in commit 56362cf the startup hangs for
5 minutes on ZFS if the cloudinit disk does not exist. Instead of
calling activate_volume followed by file_size_info we now call
volume_size_info. This should work reliably on all storages that support
cloudinit disks.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2019-10-18 21:40:34 +02:00
Mira Limbeck
21e1ee7b32 fix #2344: ignore cloudinit in replication check
When adding a cloudinit disk it does not contain media=cdrom until it is
actually created. This means the check in check_replication fails to
detect cloudinit and it is recognized as normal disk. Then parse_volname
fails because it does not match the vm-$vmid-XYZ format. To fix this we
now check explicitly if the volname matches cloudinit and if so, return
early.

Additionally 2 small cleanups replacing cloudinit regexes with the
same check for volname matches cloudinit.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2019-10-18 21:39:05 +02:00
Christian Ebner
d9123ef5b9 fix #1291: add option purge for vm_destroy api call
When destroying a VM, we intentionally did not remove all related
configs such as backup or replication jobs.
The intention of this flag is to allow the removal of references to
the VM being removed from such configs on destroy.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-18 21:22:51 +02:00
Thomas Lamprecht
69f2907c79 fixup: renamed conf_table_with_pending to config_with_pending_array
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-18 21:00:27 +02:00
Oguz Bektas
59ef70033c api: use guesthelper method for vm_pending path
we can use the shared conf_table_with_pending guesthelper to produce the
config table with the extra delete and pending columns.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2019-10-18 18:45:27 +02:00
Oguz Bektas
5c39708eb3 cli: use guesthelper for pending
use the shared format_pending method from guesthelpers

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2019-10-18 18:45:27 +02:00
Oguz Bektas
98bc3aeb92 use new config helpers from guest-common for pending changes
most of the pending changes related code has been moved into
AbstractConfig, so we have to call them as class methods from QemuConfig instead.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2019-10-18 18:45:27 +02:00
Oguz Bektas
d3179e1c36 api: use shared methods in config GET
in config GET call, we can now use the new shared methods from
guest-common, namely load_current_config and load_snapshot_config.

the correct method is called depending on the parameters 'current' or
'snapshot'

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2019-10-18 18:45:27 +02:00
Thomas Lamprecht
f73ed6d10f fixup: QemuConfig->write_config doesn't takes the raw config
Thanks to Fabian for the quick notice

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-18 11:44:15 +02:00
Oguz Bektas
90ec74eac1 use print_snapshot_tree guest helper for qm listsnapshot
moved code to GuestHelpers for feature parity with pct

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2019-10-18 11:30:41 +02:00
Thomas Lamprecht
0c040cfee2 d/control: bump version dependency to libpve-guest-common-perl
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-18 11:29:50 +02:00
Thomas Lamprecht
5da072fb84 QemuServer: sort and group used perl modules
group by:
* external
* pve, other package
* pve, same package

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-18 11:24:26 +02:00
Thomas Lamprecht
3361d09901 destroy_vm: use write_config from our Config module to set an "empty" config
brings us more in line with what we do in pve-container, also it's
good to not use file_set_contents directly if we have all those nice
wrapper interface methods to do things in a safe and guaranteed way.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-18 11:22:07 +02:00
Thomas Lamprecht
5172770df7 followup: use new base config provided destroy_config method
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-18 11:20:52 +02:00
Dominic Jäger
3e8e214d73 Fix #2412: Missing VMs in pools
Between calling vm_destroy and removing the ID from user.cfg (remove_vm_access)
creating a new VM with this ID was possible. VMs could go missing from pools as
a consequence.

Adding a lock solves this for clones from the same node. Additionally,
unlinking must happen at the very end of the deletion process to avoid that
other nodes use the ID in the meanwhile.

Co-developed-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
2019-10-17 19:23:49 +02:00
Thomas Lamprecht
5c1d42b7f8 Fix #2171: vm_start: volid based statefiles were not activated
So, while we could just make this a special case before the
config_to_command call and set the $conf->{vmstate} to the statefile
for the case were it's a valid volumeid, the special case handling
get's much easier when we do this outside of that method.

So it's basically a trade-off, and after looking far to long at all
nice revisions Alwin made for me and Fabians request, and even trying
out different approaches, it was never perfect.

But having slight code duplication over the movement mess I proposed
(as I did not had the full picture then, sorry Alwin) felt like the
slightly nicer trade off, as all worked I just use this one now, it
has very clear semantics, easy to understand and that now three lines
are duplicated is IMO irrelevant.

Co-developed-by: Alwin Antreich <a.antreich@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-17 19:20:32 +02:00
Thomas Lamprecht
8f899d734e cfg2cmd: push vmstate to volid list to ensure it gets also deactivated
the volume id list is only used to activate before real start and
deactivate later, so use it for the vmstate file too.

This not only makes config_to_command have less side effects, it also
ensures that the vmstate is deactivated again

Co-developed-by: Alwin Antreich <a.antreich@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-17 19:10:52 +02:00
Dominik Csapak
af1f1ec038 fix #2395: refactor qemu_img_convert to accept files as source
and use it also for efidisk creation and importdisk
this way we correctly handle zfs-over-iscsi options for those cases

also write tests for it

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-10-17 13:57:21 +02:00
Dominik Csapak
458a2b2640 add tests for qemu_img_convert
Add tests for the qemu_img_convert parameters to the resulting
'qemu-img convert' call

we mock the 'run_command' and extract the 'cmd' parameter to
compare with what we expect

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-10-17 13:57:21 +02:00
Thomas Lamprecht
a447e92c09 vm_start: don't reuse migrate_port variable for storage migration
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-14 13:49:30 +02:00
Thomas Lamprecht
9066fd2635 config_to_command: remove unused variable
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-14 13:49:30 +02:00
Stefan Reiter
3d8d2e8dad fix #2402: allow 1GB hugepages if 2MB is unavailable
As reported in bug #2402, a system started with "default_hugepagesz=1G
hugepagesz=1G" does not have a /sys/kernel/mm/hugepages/hugepages-2048kB
directory.

To fix, ignore the missing directory in hugepages_mount (since it might
not be needed anyway), and correctly check if the requested hugepage
size is available in hugepages_size instead.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-10-10 15:32:46 +02:00
Thomas Lamprecht
503be1f71e test cfg2cmd: add spice enhancement test
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-09 16:51:03 +02:00
Thomas Lamprecht
69f4bd3434 test: cfg2cmd: do NOT sort expected/actual commands
In general it matters where a command line options is positioned
inside a QEMU command, so we want to actually also check the order in
the cfg2cmd test

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-09 08:08:24 +02:00
Thomas Lamprecht
311e92935a cfg2cmd: sort PCI bridges when adding them for stabillity
In general it matters where a command line options is positioned
inside a QEMU command, so we want to actually also check the order in
the cfg2cmd test, to do so we need to avoid false positives like this
added.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-09 07:51:13 +02:00
Aaron Lauterer
4d316a63c7 cfg2cmd: fix serial-bus for spice foldersharing
Thanks to Gilberto Nunes for finding a bug where the VM would not start
with foldersharing enabled and the qemu agent option disabled [0].

The cause was that the device org.spice-space.webdav.0 would not find a
virtio-serial-bus in this situation.

Since we always create a virtio-serial-bus for the spice vdagent it
seems sensible to use that also for the foldersharing device by moving
it in front of the other spice devices.

[0]: https://pve.proxmox.com/pipermail/pve-devel/2019-October/039441.html

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2019-10-08 18:15:00 +02:00
Stefan Reiter
03b8d4a72d cfg2cmd: fix descriptions of cfg2cmd test cases
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-10-02 08:35:27 +02:00
Alexandre Derumier
faf80f2566 qemu 4.0 : add Cascadelake-Server && KnightsMill cpu models
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2019-09-30 16:44:51 +02:00
Mira Limbeck
7d6c99f0a0 fix #2217: don't copy cloudinit disk on clone
This removes the cloudinit disk from the list of drives to clone. As the
cloudinit disk is recreated on every VM start, it's not necessary to
clone it.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2019-09-26 18:13:26 +02:00
Thomas Lamprecht
f1619a3b42 bump version to 6.0-9
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-09-26 12:02:14 +02:00
Thomas Lamprecht
733234be04 cfg2cmd: support USB 3 SPICE ports with 4.0 machine feature
The reason for why we did not do this in the first place was the fact
that the "usb3" flag could be set in older qemu-server versions, we
just ignored it but not filtered it out of the config..

That means there can be VMs out there which would now become a
different HW layout, and issue for migration and live-snapshot
restore.

But, actually, while the "usb3" property could be set it allowed to
start the VM in only if an additional USB devices was added to the VM
with USB2, or the VM uses "q35" based machine - as else no "ehci" was
available, and thus the "ignored" USB3 - SPICE could not get attached
anywhere -> QEMU chickened out.

And if a user had a configuration where this could started we have
still a bit luck, live-migration was not possible as the "can't
migrate VM which uses local devices:" check still hit, as in
qemu-server older than 6.0-8 we explicitly checked for "spice" when
seeing what usb device were not local, so a "spice,usb3=X" was always
(luckily) wrongly detected as local device -> migration was blocked.

So we only have one case left: restoring a live-snapshot. Here sadly
there seems no way out, it was possible to do with a "spice,usb3=1"
usb device, and thus all Snapshots taken on such VMs after they had a
clean restart on PVE 6 (to have a machine version >= 4.0) are broken
- but can be easily fixed by removing the "usb3=1" from the
problematic snapshot config.
As restoring a snapshot can be repeated more than once even on
failure without rendering the snapshot or VM permanently unusable,
this should be a reasonable compromise.

I strongly believe that the chance is so small that no one is
affected in practice and the property description mentioned that it
was not supported. If anybody is affected on snapshot restore we can
help them on a case-per-case basis.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-09-26 11:54:53 +02:00
Thomas Lamprecht
fce8336a00 bump version to 6.0-8
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-09-25 17:07:08 +02:00
Aaron Lauterer
a022e3fdab tree-wide trailing whitespace cleanup
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2019-09-25 16:55:53 +02:00
Mira Limbeck
a82348eb34 fix #2382: delete cloudinit disk before restoring
The fix introduced in commit bf4a933 did not work as intended. We're
iterating over the $oldconf, not over $virtdev_hash. This means
$drive->{is_cloudinit} is always undefined. Instead use the $exclude_cloudinit
parameter from drive_is_cdrom().

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2019-09-25 16:53:29 +02:00
Thomas Lamprecht
2135fcdb36 make clean: also cleanup source tarball
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-09-25 14:59:03 +02:00
Thomas Lamprecht
7e7ec468a0 d/control: update dh version dependency and standard version
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-09-25 14:58:05 +02:00
Thomas Lamprecht
efd152d787 buildsys: add dsc target
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-09-25 14:51:50 +02:00