Fiona Ebner <f.ebner@proxmox.com> says:
Record the created fleecing images in the VM configuration to be able
to remove left-overs after hard failures.
Adds a new special configuration section 'fleecing', making special
section handling more generic as preparation, as well as fixing some
corner cases in configuration parsing and adding tests.
Fiona Ebner (16):
migration: remove unused variable
test: avoid duplicate mock module in restore config test
test: add parse config tests
parse config: be precise about section type checks
test: add test case exposing issue with unknown sections
parse config: skip unknown sections and warn about their presence
vzdump: anchor matches for pending and special sections
vzdump: skip all special sections
config: make special section handling generic
test: parse config: test config with duplicate sections
parse config: warn about duplicate sections
check type: require schema as an argument
config: add fleecing section
fix#5440: vzdump: better cleanup fleecing images after hard errors
migration: attempt to clean up potential left-over fleecing images
destroy vm: clean up potential left-over fleecing images
Link: https://lore.proxmox.com/20250127112923.31703-1-f.ebner@proxmox.com
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Collect special sections below a common 'special-sections' key in
preparation to introduce a new special section.
The special 'cloudinit' section was added in the top-level of the
configuration structure, but it's cleaner to group special sections
more similar to snapshots.
The 'cloudinit' key was already initialized, so having the new
'special-sections' key be always initialized should not cause issues
after checking and adapting all usages of 'cloudinit' which this patch
attempts to do.
Add compat handling for remote migration which might receive the
configuration hash from a node that does not yet have the changes.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Link: https://lore.proxmox.com/20250127112923.31703-10-f.ebner@proxmox.com
First, the provider is asked about what restore mechanism to use.
Currently, only 'qemu-img' is possible. Then the configuration files
are restored, the provider gives information about volumes contained
in the backup and finally the volumes are restored via
'qemu-img convert'.
The code for the restore_external_archive() function was copied and
adapted from the restore_proxmox_backup_archive() function. Together
with restore_vma_archive() it seems sensible to extract the common
parts and use a dedicated module for restore code.
The parse_restore_archive() helper was renamed, because it's not just
parsing.
While currently, the format for the source can only be raw, do an
untrusted check for the source for future-proofing. Still serves as a
basic sanity check currently.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
[WB: fix 'bwlimit' typo]
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Tested-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Reviewed-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Link: https://lore.proxmox.com/20250404133204.239783-18-f.ebner@proxmox.com
When starting a VM, drives with an attached ISO image require an
explicit `media` parameter. Setting `media=disk` on such drives makes
them writable. This breaks the user expectation that ISO images are
immutable.
Hence, require an explicit `media=cdrom` parameter on VM updates via
the API for drives that have a volume with type `iso` attached. If
some users rely on the ability to set `media=disk` on such drives, we
may make this available in the future (and require high privileges).
Until then, affected users can manually edit the VM config.
Suggested-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Friedrich Weber <f.weber@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Allow users with the Mapping.Use privilege on the /mapping/hwrng path to
configure /dev/hwrng as an entropy source for VirtIO RNG devices.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
Allow non-root users with the VM.Config.HWType privilege to configure
/dev/urandom & /dev/random as an entropy source for a VirtIO RNG device.
/dev/hwrng remains restricted to the root user.
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
This patch is for enabling AMD SEV-SNP support.
Where applicable, it extends support for existing SEV(-ES) variables
to SEV-SNP. This means that it retains no-debug and kernel-hashes
options, but the no-key-sharing option is removed.
The default policy value is identical to QEMU’s, and the therefore
required option has been added to configure SMT support.
The code was tested by running a VM without SEV, with SEV, SEV-ES,
SEV-SNP. Each configuration was tested with and without an EFI disk
attached. For SEV-enabled configurations it was also verified that the
kernel actually used the respective feature.
Signed-off-by: Philipp Giersfeld <philipp.giersfeld@canarybit.eu>
Tested-by: Markus Frank <m.frank@proxmox.com>
Reviewed-by: Markus Frank <m.frank@proxmox.com>
So that we can show a proper warning in the migrate dialog and check it
in the bulk migrate precondition check.
The returned 'unavailable_storages' and 'unavailable-resources' should
be the same as before, but we now always return (not_)allowed_nodes
too.
To make the code a bit easier to read, reorganize how we construct the
(not_)allowed_nodes properties.
Also add a note that we want to redesign the return values here for a
easier to consume interface.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Switch the underlying check_local_resource method to return a hash and
adapt call sites to that change. Return that new hash in the API as
new return property for the frontend to have more info to check.
This is in preparation for enabling live-migration for VMs with
mappings that declare being capable of them.
Originally-by: Dominik Csapak <d.csapak@proxmox.com>
[TL: split out from unrelated changes and fix return schema]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Do not mark mapped devices as local resources for the offline
migration case, there it's only relevant that they have a mapping
configured on the target node, which is a different check.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
we cleaned up extracted images, but logged it even for non extracted
ones. Only log this when we actually cleaned anything up.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
in our schema for 'vga' the type is optional, so a config like
vga: memory=64
is valid. When checking the type we have to check if the type is defined
before accessing it.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
[FE: add 'termproxy/vncproxy' prefix to commit title]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Remove variable `$size`, which is not used here and likely a copy-paste
redundancy from the `create_disks` subroutine.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
Asserts whether the target storage supports storing VM images before
cloning a VM and its volumes to the target storage.
Without the check in place, a VMs volumes can be cloned to a storage,
which does not support VM images, but won't be able to start since any
attached volume must be stored on a supported storage.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
Asserts whether the target storage supports storing VM images before
moving a VM volume to the target storage.
Without the check in place, a VM volume can be moved to a storage, which
does not support VM images, but won't be able to start since any
attached volume must be stored on a supported storage.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
Commit c8ed1ac2 ("api: create disks: live import: use format from
storage layer") broke live import from an OVA containing a disk,
because a combined format like 'ova+vmdk' was used for the live-import
disk mapping, leading to failure:
> invalid format 'ova+vmdk' for 'scsi0' mapping
There was also an informational message about the confusion earlier:
> file_size_info: '/mnt/pve/dir/images/135/vm-135-disk-0.vmdk': \
> falling back to 'raw' from unknown format 'ova+vmdk
Fixes: c8ed1ac2 ("api: create disks: live import: use format from storage layer")
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
While no problem is known with having an empty machine string in the
configuration and it was already possible setting an empty machine
manually via 'qm set', the behavior changed because of commit 919e69d0
("machine: add check_and_pin_machine_string() helper") and there is
potential for problematic edge cases. Restore the previous behavior.
Pinning is also required when there is no given machine type, so the
call to check_and_pin_machine_string() should stay unconditional.
For update, pinning was recently added by commit 7a9570f3 ("api:
update vm config: pin machine version when switching to windows os
type"), so bring that in-line with the behavior for create too.
Another idea would've been to just return the default machine in
check_and_pin_machine_string(), but that would also be a change in
behavior. In particular, the default depends on the arch, so an empty
machine type will pick the default machine for the currently
configured arch even when the arch is later changed.
Reported-by: Daniel Herzig <d.herzig@proxmox.com>
Fixes: 919e69d0 ("machine: add check_and_pin_machine_string() helper")
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
During virtual machine creation, the machine version is pinned when
the guest OS is Windows. The same should be done when the guest OS
type is newly set to Windows for consistency.
NOTE RFC
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Extract the logic for guest OS-type dependent machine version pinning
into a dedicated helper, so it can be re-used.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Like this, it can be used by modules that cannot depend on
QemuServer.pm without creating a cyclic dependency.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Add an export, since the function is rather commonly used (in
particular inlined in function calls, where prefixing with the module
name would hurt readability) and there won't be much potential for
confusion name-wise.
This was the only user of stat(), so remove the File::stat include.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
The commit 37a1f42a introduced a conversion step for newly allocated
volume disks to base volumes for VM templates. This wrongly excluded
efidisk and tmpstate volumes from this conversion, even though those
should be converted to base volumes too.
Therefore, include efidisks and tpmstate volumes to be converted to base
volumes when newly allocating them.
Suggested-by: Fiona Ebner <f.ebner@proxmox.com>
Fixes: 37a1f42a ("fix #5301: convert added volume disks to base images for templates")
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
when creating new ones, we already force raw as format, but adding existing
volumes as tpmstate0 had no such checks.
Suggested-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
since there is no reliable way to check whether a linked clone would end up
being something other than a raw file, and the volumes are tiny anyway.
otherwise on directory storages, the following sequence of events could happen:
- linked clone using raw file as base and qcow2 as overlay
- swtpm_setup interprets qcow2 file as raw
- swtpm_setup fails to find TPM state and overwrites it with a new one
- file is now no longer a linked clone, but a raw file with a qcow2 extension
- move disk and migration fail because of the format mismatch
the downside is that storages that actually support raw linked clones (ZFS,
RBD, LVM-thin) now use more space for fully cloned TPM state volumes...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
The file_size_info() will return the size of the image based on
guessing the format. When importing via API, the correct size is
already known, so it's better to pass it in. The root-only 'qm'
commands for disk import and OVF import will still use auto-detection
for backwards compatibility. It might make sense to be able to
explicitly specify the format there too to get the correct size in all
cases.
New callers should detect the size according to the appropriate format
first.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Keep format auto-detection for backwards compatibility. Only root is
allowed to use such images as source images and the untrusted checks
will be done here.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Use the same logic as the actual import later.
Has the nice side effect of getting rid of the unnecessary manual
dispatch to the plugin too.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
With auto-detection, there might be false positives for raw images
that contain data that looks like another image format, but are not
problematic, because they will always be treated as raw images.
Since commit "image convert: use volume format from storage layer",
actual importing will happen using the storage layer format, so use
that here too.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Avoid that file_size_info() does auto-detection via qemu-img.
This also adapts to the new argument order.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Since commit "print drive commandline: improve format detection" such
mismatches will lead to being unable to start the VM, so catch the
issue early.
Suggested-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
as it may be the only cause of the clone incompatibility
Example:
# qm clone 101 102 --full --snapname foo
Before:
> Full clone feature is not supported for 'local-zfs:base-100-disk-2/vm-101-disk-2' (tpmstate0)
After:
> Full clone feature is not supported for a snapshot of 'local-zfs:base-100-disk-2/vm-101-disk-2' (tpmstate0)
Signed-off-by: Alexander Zeidler <a.zeidler@proxmox.com>
So far, the error message only contained the name of the disk
(tpmstate0, efidisk0, ...), which can also lead to the assumption that a
specific disk type is the problem. Now the volume ID is primarily
listed.
Example:
# qm clone 101 102 --full --snapname foo
Before:
> Full clone feature is not supported for drive 'tpmstate0'
After:
> Full clone feature is not supported for 'local-zfs:base-100-disk-2/vm-101-disk-2' (tpmstate0)
Signed-off-by: Alexander Zeidler <a.zeidler@proxmox.com>
When cloning was repeatedly attempted, the error message indicated a
different unsupported volume each time. The hash is now sorted to always
mention the same volume as long as it has not been fixed.
Signed-off-by: Alexander Zeidler <a.zeidler@proxmox.com>
[FE: replace old-style 'foreach' with 'for' while at it]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
the ui and api talks about 'import working storage' but the error here
still said 'for extraction'. Improve the message by unifiying the
wording and adding the storage name to it too.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
This reverts commit fca0ba5d77, quoting
Fiona in verbatim:
> Regarding the patch "schema: add fleecing-images config property",
> Fabian off-list suggested using a config section "special:fleecing"
> instead of a property, so that it is truly internal-only. If we go for
> that, the commit should be reverted. Which approach do you prefer?
-- https://lore.proxmox.com/pve-devel/5126c251-64fd-44fe-b1a6-fda9074eb9a1@proxmox.com/
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
The function checks for resources that cannot be migrated, snapshoted,
or suspended.
To run this function while the snapshot lock is active, the
pve-guest-common patch 'AbstractConfig: add abstract method to check for
resources preventing a snapshot.' is required.
Signed-off-by: Markus Frank <m.frank@proxmox.com>
this is to override the target extraction storage for the option disk
extraction for 'import-from'. This way if the storage does not
supports the content type 'images', one can give an alternative one.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
when 'import-from' contains a disk image that needs extraction
(currently only from an 'ova' archive), do that in 'create_disks'
and overwrite the '$source' volid.
Collect the names into a 'delete_sources' list, that we use later
to clean it up again (either when we're finished with importing or in an
error case).
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
creating non-raw disk images with arbitrary content is only possible with raw
access to the storage, but checking for references to external files doesn't
hurt, in case for non pve-managed volumes.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
[ DC: removed problematic checks for pve-managed volumes ]
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
