Explicitly allows changing other properties than the cputype, even if
the currently set cputype is not accessible by the user. This way, an
administrator can assign a custom CPU type to a VM for a less privileged
user without breaking edit functionality for them.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
and move the lock call and decision logic closer together
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Tested-by: Fabian Ebner <f.ebner@proxmox.com>
This reverts commit b5490d8a98.
When resizing a volume of a running VM, a qmp block_resize command
is issued. This is non-blocking, so the size on the storage immediately
after issuing the command might still be the old one.
This is part of the issue reported in bug #2621.
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
this is only used for migration via 'qm mtunnel', regular users should
never need to resume a VM that does not logically belong to the node it
is running on
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
It was necessary to move foreach_volid back to QemuServer.pm
In VZDump/QemuServer.pm and QemuMigrate.pm the dependency on
QemuConfig.pm was already there, just the explicit "use" was missing.
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Just like with live-migration, custom CPU models might change after a
snapshot has been taken (or a VM suspended), which would lead to a
different QEMU invocation on rollback/resume.
Save the "-cpu" argument as a new "runningcpu" option into the VM conf
akin to "runningmachine" and use as override during rollback/resume.
No functional change with non-custom CPU types intended.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
This is required to support custom CPU models, since the
"cpu-models.conf" file is not versioned, and can be changed while a VM
using a custom model is running. Changing the file in such a state can
lead to a different "-cpu" argument on the receiving side.
This patch fixes this by passing the entire "-cpu" option (extracted
from /proc/.../cmdline) as a "qm start" parameter. Note that this is
only done if the VM to migrate is using a custom model (which we can
check just fine, since the <vmid>.conf *is* versioned with pending
changes), thus not breaking any live-migration directionality.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
both where previously missing. the existing 'check_storage_access'
helper is not applicable here since it operates on a full set of VM
config options, not just storage IDs.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
the syntax is backwards compatible, providing a single storage ID or '1'
works like before. the new helper ensures consistent behaviour at all
call sites.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
as preparation for refactoring it further. remote migration will add
another 1-2 parameters, and it is already unwieldly enough as it is.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
on storages where the minimum size of images is bigger than the real
OVMF_VARS.fd file, they get padded to their minimum size
when using such an image, qemu maps it fully to the vm, but the efi
does not find the vars region and creates a file on the first efi
partition it finds
this breaks some settings in the ovmf, such as resolution
to fix this, we have to specify the size for the pflash, so that
qemu only maps the first n bytes in the vm (this only works for
raw files, not for qcow2)
we also have to use the correct size when converting between storages
in 'clone_disk' (used for move disk and cloning vms) and when
live migrating to different storages
when we now expect that the source image is always correctly used/created
(e.g. raw with size=x in pflash argument) then we always create the
target correctly
when encountering users which have a non-valid image (e.g. a efidisk
moved from zfs to qcow2 before this patch), we have to tell them to
recreate the efidisk and the settings on it
we have to version_guard it to 4.1+pve2 (since we haven't bumped yet
since the change to pve2)
also add 2 tests, one for the old version and one for the new
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Stefan Reiter <s.reiter@proxmox.com>
Reviewed-by: Stefan Reiter <s.reiter@proxmox.com>
[ Thomas: rebased to master ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
by only checking for replicatable volumes when a replication job is
defined, and passing only actually replicated volumes to the target node
via STDIN, and back via STDOUT.
otherwise this can pick up theoretically replicatable, but not actually
replicated volumes and treat them wrong.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
For secure live migration with local disks via NBD over a unix socket,
we have to somehow communicate from the source node to the target node
if it supports it. This is because there can only be one NBD server with
exactly one socket bound.
The source node passes that information via STDIN. Support for
'spice_ticket: (...)' is added in addition to 'nbd_protocol_version:
<version>'. As old source nodes send the spice ticket without a prefix,
we still have to have a fallback for this case. New information should
always be passed via a prefix that is matched, otherwise it will be
recognized as spice ticket.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
With Qemu 4.2 we encountered a problem with unix sockets and SSH socket
forwarding for drive-mirror. It seems the socket gets reopened again and
again after it closes for some reason. This can be worked around by
specifying 'block-job-cancel' instead of 'block-job-complete' when we're
not interested in swapping the disks again from NBD to their original
protocol. This is always the case when we use drive-mirror for live
migrating a VM.
qemu_drive_mirror is used for migration and for clone_disk. All in all
we have 3 cases to handle. Either the 'skip' case which skips the
completion of the job. The 'wait' case which was the default before and
still is when $completion is undefined. And the new 'wait_noswap' case
which is used for the live migration.
If 'wait_noswap' is specified, we issue a 'block-job-cancel' once the block
job is in 'ready' state. This completes the block job without swapping the
disks.
clone_disk always uses 'block-job-cancel' via the qemu_blockjobs_cancel
sub.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
The initialization for the drive keys in $confdesc is changed
to be a single for-loop iterating over the keys of $drivedesc_hash and
the initialization of the unusedN keys is move to directly below it.
To avoid the need to change all the call sites, functions with more than
a few callers are exported from the submodule and imported into QemuServer.pm.
For callers of the now imported functions within QemuServer.pm, the prefix
PVE::QemuServer is dropped, because it is unnecessary and now even confusing.
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
The http-server has a 64KB payload limit for post requests, so note
that explicit even if it's a theoretical maximum as the reamainig
params also need some space in the request
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
'input-data' can be used to pass arbitrary data to a guest when running
an agent command with 'guest-exec'. Most guest-agent implementations
treat this as STDIN to the command given by "path"/"arg", but some go as
far as relying solely on this parameter, and even fail if "path" or
"arg" are set (e.g. Mikrotik Cloud Hosted Router) - thus "command" needs
to be made optional.
Via the API, an arbitrary string can be passed, on the command line ('qm
guest exec'), an additional '--pass-stdin' flag allows to forward STDIN
of the qm process to 'input-data', with a size limitation of 1 MiB to
not overwhelm QMP.
Without 'input-data' (API) or '--pass-stdin' (CLI) behaviour is unchanged.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Because of alignment and rounding in the storage backend, the effective
size might not match the 'newsize' parameter we passed along.
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
The description for vm_config was out of date and from the description
for vm_pending it was hard to tell what the difference to vm_config was.
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
regression introduced with commit a85ff91b
previously we set $target to undef if it's localnode or localhost, then
we check if node exists.
with regression commit, behaviour changes as we do the node check in
else, but $target may be undef. this causes an error:
no such cluster node ''
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
improved readability
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
VM.Audit can see the current config and the list of snapshots
already, so there is no real reason to disallow
the config of snapshots
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
This is the guarantee that this call operates on it's created config.
A VMID cannot be reused afterall. So only remove the guarantee at the
last step, just before throwing up the error message about the clone
failure.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
We clone the source VM firewall config before forking the "realcmd"
worker, but did not mind cleaning it up again if the clone failed
somewhere in the worker.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
wrap around code which can possibly fail in evals to handle them
gracefully, and log errors.
note: this results in a change of behavior in the API. since errors
are handled gracefully instead of "die"ing, when there is a pending
change which cannot be applied for some reason, it will get logged in
the tasklog but the vm will continue booting regardless. the
non-applied change will stay in the pending section of the
configuration.
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Do the same as for the "create" case, only trigger the "start after
create/restore" task after the locked "realcmd" was done. Else, the
start can never succeed, it also acquires a lock, but restore only
release it once outside of realcmd.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
bump versioned build-dependency, as qemu-server has tests checking
for errors, and we fixed an grammar error in pve-storage, so we need
the newer version to ensure our test go through
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
only VM.PowerMgmt is not enough, since we allocate space on a storage,
so we need VM.Config.Disk on the vm and Datastore.AllocateSpace on the storage
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Sometimes, a user wants to remove the 'suspended' state without
resuming the vm from that state. Since the vm is locked with
'suspended', this was not possible without help from root@pam
This patch allows to delete the vmstate and the suspended lock and
related config entries with it. The user still has to have the right
priviliges and the vm cannot be 'protected' for this to work
Inspired-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
we did not actually delete the state if we deleted the 'vmstate' config,
leaving stray vmstates on the disks
actually implement the removal, requiring 'VM.Config.Disk' and
'VM.PowerMgmt' privs
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
'pve-qm-machine' is auto-registered, but for re-use for a new
runningmachine we added the newer pve-qemu-machine standard option.
Use that one to avoid confusion.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
With the noerr flag set in parse_volume_id we have to check if
$volname is defined before comparing it to 'cloudinit'.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
this is useful as meta information for e.g., provisioning or config
management systems
adding the info also to the 'status' api call to make it easier to show
it in the gui
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
QMP and monitor helpers are moved from QemuServer.pm.
By using only vm_running_locally instead of check_running, a cyclic
dependency to QemuConfig is avoided. This also means that the $nocheck
parameter serves no more purpose, and has thus been removed along with
vm_mon_cmd_nocheck.
Care has been taken to avoid errors resulting from this, and
occasionally a manual check for a VM's existance inserted on the
callsite.
Methods have been renamed to avoid redundant naming:
* vm_qmp_command -> qmp_cmd
* vm_mon_cmd -> mon_cmd
* vm_human_monitor_command -> hmp_cmd
mon_cmd is exported since it has many users. This patch also changes all
non-package users of vm_qmp_command to use the mon_cmd helper. Includes
mocking for tests.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
While we may not want to copy the cloudinit disk/drive, we still need
to create+allocate the volume, else the next start complains about a
missing CI drive..
fixes commit 7d6c99f0a0.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Show storages configured for the target node and not for the current one
because they can be different.
Duplicated the `complete_storage` sub and extended it to extract the
targetnode from the parameters to pass it into the storage_check_enabled
function.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
SHA-512 crypted passwords are longer than 64 byte, and it also does
not make sense to limit passwords to such a short length. Increase
to 1024, that should be enough for a while, but still limits maximal
password payload to avoid DOS or the like.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
This brings qemu more in line with containers, and it's nicer to
allow passing the replacement config if we want to keep it, instead
of setting a "memory: 128" config.
Use that to lock it on removal before final deletion, and on legacy
tar archive restore, in between old VM destruction and new
restoration.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
reverting a nonexisting option did not work with the latest changes
in pve-guest-common, because we do not delete the pending option
in 'add_to_pending_delete' anymore
this had the effect that we had following in the config:
[pending]
option: pendingvalue
delete: option
which would do the deletion code and the pending add code
(e.g. delete the pending cloud init drive and creating it again)
to avoid that situation, we need to remove the option from the pending hash
in the 'delete loop'
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
When adding a cloudinit disk it does not contain media=cdrom until it is
actually created. This means the check in check_replication fails to
detect cloudinit and it is recognized as normal disk. Then parse_volname
fails because it does not match the vm-$vmid-XYZ format. To fix this we
now check explicitly if the volname matches cloudinit and if so, return
early.
Additionally 2 small cleanups replacing cloudinit regexes with the
same check for volname matches cloudinit.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
When destroying a VM, we intentionally did not remove all related
configs such as backup or replication jobs.
The intention of this flag is to allow the removal of references to
the VM being removed from such configs on destroy.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
we can use the shared conf_table_with_pending guesthelper to produce the
config table with the extra delete and pending columns.
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
most of the pending changes related code has been moved into
AbstractConfig, so we have to call them as class methods from QemuConfig instead.
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
in config GET call, we can now use the new shared methods from
guest-common, namely load_current_config and load_snapshot_config.
the correct method is called depending on the parameters 'current' or
'snapshot'
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
Between calling vm_destroy and removing the ID from user.cfg (remove_vm_access)
creating a new VM with this ID was possible. VMs could go missing from pools as
a consequence.
Adding a lock solves this for clones from the same node. Additionally,
unlinking must happen at the very end of the deletion process to avoid that
other nodes use the ID in the meanwhile.
Co-developed-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
This removes the cloudinit disk from the list of drives to clone. As the
cloudinit disk is recreated on every VM start, it's not necessary to
clone it.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
and add "current" as keyword, further remove the parenthesis for the
post-if, to adapt to Proxmox general perl code style
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thanks to Stefan and Thomas for the suggestions.
Changes from v1:
* update parameter description
* warn instead of die
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
otherwise a user with only VM.Config.HWType cannot
delete a 'pending' usbX: spice or serial: socket option
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
this creates a reboot request file (inspired by pve-container)
and relies on the 'qm cleanup' call by the qmeventd to detect
and restart the vm afterwards
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
for some reason not setting port results in a port of '65535' which
triggers an execption in http-server anyevent, so we set the port to 0
also, we have to read the ticket from stdin even for 'unix' type secure
migration
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
e.g. local storage was considered not allowed for offline migration
even if it is available on all nodes, this should now be fixed as it
is now considered available on all nodes if a local storage isn't
restricted to a specific subset of the available nodes. The user is
responseable to make sure that the datacenter storage config reflects
the actual setup, so there is no additional check for local storages
which aren't available on all nodes if they are not explicitly marked
at datacenter level.
Signed-off-by: Tim Marx <t.marx@proxmox.com>
Adds the path '{vmid}/cloudinit/dump' and requires the parameter 'type'
that's either 'user', 'network' or 'meta'. Returns the generated config as
string.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
The variable is used instead of the literal value so we have one single
place to change the actual value of every use.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
The restore of a backup from a VM template will first restore the VM and then
convert the restored VM back into a template.
This automatically performes the steps of the current behaviour, where the user
has to manually convert the restored VM back to a template.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
When a Cloudinit image is created, we allow choosing
the format as parameter.
So, the parameter should also be taken into account.
The default for Cloudinit will stay qcow2 on directory storages.
using the same logic like serialX: socket
users need VM.Config.HWType for usbX: spice
but only root can add/remove real devices
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
since we do not want the user to change a device which has a real
devices already set, we have to check it later when we have the config
so we do not have to give the params to vm_check_modify_config_perm anymore
also improve the regex to \d+
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
the clone API calls (target) 'storage' parameter is optional as we
simply use the source storage in this case, but we did not handle
this case when we added the bandwidth_limit abillity, address that.
This patch only pushes the storage parameter into the storage_list array
if it is defined.
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
for migrate_vm, clone_vm and move_vm_disk and extract it. The 'migrate_vm' call
passes it to PVE::QemuMigrate->migrate for handling.
Additionally the bwlimit option's description of the 'create_vm' call gets
consistent capitalization of I/O.
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
create a fixed size cloudinit disk if it is referenced in config and
does not exist. the size of the disk created when first added to the
config is reduced to 4MiB to match the one created in
commit_cloudinit_disk.
maximum file size per snippet file (network, user, meta) is increased to 1MiB.
preparation for offline migration without the cloudinit disk (that is
always regenerated on start).
also fixes#1807, although a further patch is required to change the
vmid on restore
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>
this makes it possible to give a storage for state saving, if one
wants to use a different storage than for snapshots or does not
want to save this info into the config
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Adds the 'cicustom' option to specify either or both network and user
options as property strings. Their parameters are files in a snippets
storage (e.g. local:snippets/network.yaml). If one or both are specified
they are used instead of their respective generated configuration.
This allows the use of completely custom configurations and is also a
possible solution for bug #2068 by specifying a custom user file that
contains package_upgrade: false.
Tested with Ubuntu 18.10 and cloud-init 18.4.7
Signed-off-by: David Limbeck <d.limbeck@proxmox.com>
this adds a new config option for it, and executes it on four
points in time:
'pre-start'
'post-start'
'pre-stop'
'post-stop'
on pre-start we abort if the script fails
and pre-stop will not be called if the vm crashes or if
the vm gets powered off from inside the guest
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
The qm CLI command offer the config and showcmd functions. Both of those
outputs may vary with respect to a given snapshot. This adds a switch
that shows the corresponding snapshot's config and command line.
The code needs a newer libpve-guest-common-perl, thus bumping the
dependency.
Signed-off-by: Rhonda D'Vine <rhonda@proxmox.com>
$newconf->{pending} is a reference to an empty hash, which is not falsy,
thus we always printed the warning
so check if there are actual values there and if yes,
give the names of the properties for which pending changes are found
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Some storage like rbd or lvm can't keep thin-provising after a qemu-mirror.
Call qga guest-fstrim if qga is available and fstrim_cloned_disks is enabled
after move_disk and migrate.
Co-Authored-By: Alexandre Derumier <aderumier@odiso.com>
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
writes the given content to the file
the size is at the moment limited by the max post size of the
pveproxy/daemon, so we set the maxLength to 60k
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
this api call reads a file via the guest agent,
(in 1MB chunks) but is limited to 16MiB (for now)
if the file is bigger, the output gets truncated and a
'truncated' flag is set in the return object
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
this imitates the qemu-guest-agent interface
with an 'exec' api call which returns a pid
and an 'exec-status' api call which takes a pid
the command for the exec call is given as an 'alist'
which means that when using we have to give the 'command'
parameter multiple times e.g.
pvesh create <...>/exec --command ls --command '-lha' --command '/home/user'
so that we avoid having to deal with shell escaping etc.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
this executes the guest agent command 'set-user-password'
with which one can change the password of an existing user in the vm
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Move the locking inside worker, so that the process doing the actual
work (create or restore) holds the lock, and can call functions which
do locking without deadlocking.
This mirrors the behaviour we use for containers, and allows to add
an 'autostart' parameter which starts the VM after successful
creation. vm_start needs the lock and as not the worker but it's
parents held it, it couldn't know that it was actually save to
continue...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
cloning a vm means copying the current state, not the
state of 'some time in the future, when the vm is started again'
we should not copy the pending changes, which also fixes the
issue that we got a wrong pending change on the disks,net,smbios,etc.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
since password is easily decrypted, hide it on the api
if someone needs it, they can get it directly from the
config
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
We don't leave this up to cloud-init as we don't want
un-hashed values at all in our configs.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
move: don't error out with "you can't move a cdrom"
clone: always full-clone cloud-init images
They get completely replaced anyway at the next start, so
there's no point in keeping them.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
*) always replace old cloudinit images
*) apply pending cloudinit changes when generating a new
image
For cloudinit we now always use vdisk_free before
vdisk_alloc in order to always replace old images, this
allows us to hotplug a new drive by setting it to
`none,media=cdrom` first (to eject the disk), then setting
it back to 'storage:cloudinit' to have a new image generated
after applying the currently pending changes.
