and adapt the tests
this does not impact live migration, since the order here does not
change the device layout
we want this to consistently have the readconfig first
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
from hotplug_pending we go into 'vmconfig_update_disk', where we check the
hotpluggability of options.
add 'ssd' there as a non-hotpluggable option (since we'd have to unplug/plug to
change the drive type)
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
The package will be used for custom CPU models as a SectionConfig, hence
the name. For now we simply move some CPU related helper functions and
declarations over from QemuServer to reduce clutter there.
Exports are to avoid changing all call sites, functions have useful
names on their own.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
As 'qemu_img_format' just matches a regex, this doesn't make much of
a difference, but AFAICT all other calls of 'qemu_img_format' use 'volname'.
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
since we handle errors gracefully now, we don't need to write & save
config every time we change a setting.
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
get_basic_machine_info was removed by commit 045749f2fc.
Use get_host_arch to get the default machine type instead, and
optionally allow to specify architecture as parameter.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
* query_understood_cpu_flags returns all flags that QEMU/KVM knows about
* query_supported_cpu_flags returns all flags that QEMU/KVM can use on
this particular host.
To get supported flags, a temporary VM is started with QEMU, so we can
issue the "query-cpu-model-expansion" QMP command. This is how libvirt
queries supported flags for its "host-passthrough" CPU type.
query_supported_cpu_flags is thus rather slow and shouldn't be called
unnecessarily.
Note that KVM and TCG accelerators provide different expansions for the
"host" CPU type, so we need to query both.
Currently only supports x86_64, because QEMU-aarch64 doesn't provide the
necessary querying functions.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
wrap around code which can possibly fail in evals to handle them
gracefully, and log errors.
note: this results in a change of behavior in the API. since errors
are handled gracefully instead of "die"ing, when there is a pending
change which cannot be applied for some reason, it will get logged in
the tasklog but the vm will continue booting regardless. the
non-applied change will stay in the pending section of the
configuration.
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
instead of writing the config after every change, we can do it once for
all the changes in the end to avoid redundant i/o.
we also don't need to load_config after writing fastplug changes.
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
bump versioned build-dependency, as qemu-server has tests checking
for errors, and we fixed an grammar error in pve-storage, so we need
the newer version to ensure our test go through
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
run_command only passes defined and chomped strings to the callback,
so no need to do that twice.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
QEMU usually only prints warnings and errors and stays silent otherwise,
so it makes sense to just log all of it's output.
Prefix it with '[<target_hostname>]' to indicate that the output is
coming from the remote node, so users know where to search for the
error.
Side effect is that the 'VM start' task created by the migration will
now show the "QEMU:" prefix, but it's still very readable IMHO.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
By default run_command prints the entire commandline executed when an
error occurs, but QEMU and our migrate command are not only
uninteresting to the user[*] but also annoyingly long. Hide them and only
print the exit code.
[*] Especially our migrate command, since it can't be manually executed
anyway. QEMU's commandline *might* contain something interesting, but is
so long that it's tricky to parse anyway, any a user can always call 'qm
showcmd --pretty'.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Split out 'update_disksize' from the renamed 'update_disk_config' to
allow code reuse in QemuMigrate.
Remove dots after messages to keep style consistent for migration log.
After updating in sync_disks (phase1) of migration, write out updated
config. This means that even if migration fails or is aborted in later
stages, we keep the fixed config - this is not an issue, as it would
have been fixed on the next attempt anyway, and it can't hurt to have
the correct size instead of a wrong one either way.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
only VM.PowerMgmt is not enough, since we allocate space on a storage,
so we need VM.Config.Disk on the vm and Datastore.AllocateSpace on the storage
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
if the user set a device as hostpci with the 'shorthand' syntax:
hostpciX: 00:12
we ignored it on starting and showcmd and continued.
Since the user explicitly wanted to passthrough a device, we now check
if there is actually a device with that id
for explicitly configured devices (00:12.1), we did not check if it exists,
but the kvm call failed with a non-obvious error message
now we always call 'lspci' from SysFSTools to check if it actually exists,
and fail if not. With this, we can drop the workaround for adding
'0000' if no domain was given, since lspci does it already for us
this fixes#2510, an issue with using mediated devices where the users did not have
the domain in the config, since we forgot to add the default domain there
the only issue with this patch is that it changes the behaviour of
'showcmd' slightly, as in now, we die if the device was explicitly
given, but did not exists (we showed the commandline, now we fail)
this also slightly changes the commandline for qemu (adding always
the domain), which is not a problem since we cannot live migrate
or snapshot such vms, but we have to adapt the tests
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
some storage backends have bigger granularity than the default 128k
size from the EFIVARS template file, so we actually need to poll the
real created disk size, as it will be used to create the target
volume for local storage migration on running VMs, if it's to small
the target will be to small and migration will fail.
Just a fix for newly created EFIDISKS, for others we need to rescan
the size after we've got the migrate lock and write the updated info
out, so that the target node has the correct one (protected from
migrate lock).
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
if a user removed the vmstate from the config for whatever reason,
a vmstart did not remove the 'suspended' lock
so always delete it and delete the vmstate only if it really was there
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
while it's a disk from our storage POV, in QEMU it's a pflash, and
those cannot be hot-plugged
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
With our QEMU 4.1.1 package we can pass a additional internal version
to QEMU's machine, it will be split out there and ignored, but
returned on a QMP 'query-machines' call.
This allows us to use it for increasing the granularity with which we
can roll-out HW layout changes/additions for VMs. Until now we
required a machine version bump, happening normally every major
release of QEMU, with seldom, for us irrelevant, exceptions.
This often delays rolling out a feature, which would break
live-migration, by several months. That can now be avoided, the new
"pve-version" component of the machine can be bumped at will, and
thus we are much more flexible.
That versions orders after the ($major, $minor) version components
from an stable release - it can thus also be reset on the next
release.
The implementation extends the qemu-machine REGEX, remembers
"pve-version" when doing a "query-machines" and integrates support
into the min_version and extract_version helpers.
We start out with a version of 1.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Reviewed-by: Stefan Reiter <s.reiter@proxmox.com>
if we don't know which format the source volume/file has, let qemu-img
decide.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
or any other variant of the word 'pending'.
note that we can actually allow this snapshot after PVE 7.0, since
pending section and snapshots will be properly namespaced.
([pve:pending] and [snap:$snapname] or similar).
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
this is useful as meta information for e.g., provisioning or config
management systems
adding the info also to the 'status' api call to make it easier to show
it in the gui
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
...into:
* PVE::QemuServer::Helpers::min_version: check a major.minor version
string with a given major/minor version (this is equivalent to calling
the old qemu_machine_feature_enabled with only $kvmver)
* PVE::QemuServer::Machine::extract_version: get major.minor version
string from arbitrary machine type (e.g. pc-q35-4.0, ...)
* PVE::QemuServer::Machine::machine_version: helper to call
extract_version automatically before min_version
Includes a cfg2cmd test case with pinned machine version.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
...PVE::QemuServer::Machine.
qemu_machine_feature_enabled is exported since it has a *lot* of users
in PVE::QemuServer and a long enough name as it is.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
QMP and monitor helpers are moved from QemuServer.pm.
By using only vm_running_locally instead of check_running, a cyclic
dependency to QemuConfig is avoided. This also means that the $nocheck
parameter serves no more purpose, and has thus been removed along with
vm_mon_cmd_nocheck.
Care has been taken to avoid errors resulting from this, and
occasionally a manual check for a VM's existance inserted on the
callsite.
Methods have been renamed to avoid redundant naming:
* vm_qmp_command -> qmp_cmd
* vm_mon_cmd -> mon_cmd
* vm_human_monitor_command -> hmp_cmd
mon_cmd is exported since it has many users. This patch also changes all
non-package users of vm_qmp_command to use the mon_cmd helper. Includes
mocking for tests.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
vm_exists_on_node in PVE::QemuConfig checks if a config file for a vmid
exists
vm_running_locally in PVE::QemuServer::Helpers checks if a VM is running
on the local machine by probing its pidfile and checking /proc/.../cmdline
check_running is left in QemuServer for compatibility, but changed to
simply call the two new helper functions.
Both methods are also correctly mocked for testing snapshots.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
parse_cmdline is required for upcoming changes related to custom CPU
types and live migration, and this way we can re-use existing code.
Provides the necessary infrastructure to parse QEMU /proc/.../cmdline.
Changing the single user (check_running) is trivial too.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Also remove unused $confdir variable in QemuConfig, but leave it and
$lock_dir there, since those paths should only be used with
cfs_config_path anyway.
nodename() is still called in multiple places, but since it's cached by
INotify it doesn't really matter.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
found no use with tree-wide search, so remove:
* nic_models
* os_list_description
Both were introduced before the import to SVN happened.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Out code behaves like either l26 or other when the ostype is
undefined, both are not common as our webinterface _always_ sets the
ostype.
If one configured QXL with a VM as output device but does not has an
ostype set, and that works without "max_outputs=4" it really should
work with none too.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
with pve-qemu-4.0.1-3 or higher it was not possible in a spice remote
session to enable more displays on the fly in linux guests.
Adding the `max_outputs` parameter to the qxl device manually restores
the functionality.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
While we may not want to copy the cloudinit disk/drive, we still need
to create+allocate the volume, else the next start complains about a
missing CI drive..
fixes commit 7d6c99f0a0.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
This adds an extra field to agent_fmt that specifes the type of guest
agent connection to use. Currently there is no choice, and defaults to
virtio-serial. Since qemu-ga also runs over isa-serial, this allows OSes
such as NetBSD and OpenBSD, which do not have support for virtio-serial,
to run a qemu-ga.
This is an optional field, which leaves the default as virtio-serial. As
it doesn't change the default, it will require no change to older
configuration files.
Signed-off-by: Matt Dunwoodie <ncon@noconroy.net>
Show storages configured for the target node and not for the current one
because they can be different.
Duplicated the `complete_storage` sub and extended it to extract the
targetnode from the parameters to pass it into the storage_check_enabled
function.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
since PVE::Cluster::get_local_migration_ip does not exist anymore. this
is basically an inlined version, since this is the only remaining caller
that we actually want to keep.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
We are in the QemuServer package not in LXC, so use the correct
package for the Config, namely QemuConfig
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
the reboot request is only cleaned in the vm_start path, so if reboot
fails for some reason, the request still exists. this causes an
unintentional reboot when a shutdown/stop/hibernate is called.
to mitigate, we can just clear the reboot request in case of an error.
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
This brings qemu more in line with containers, and it's nicer to
allow passing the replacement config if we want to keep it, instead
of setting a "memory: 128" config.
Use that to lock it on removal before final deletion, and on legacy
tar archive restore, in between old VM destruction and new
restoration.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
it has some potential semantic change too, i.e., the Storage
vdisk_list call is not wrapped by eval anymore, put as
we did some (unguarded) storage things before that call I'd say that
that does not matters much..
We try to clean all unused disks too, even if one deletion fails
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
When calling qmrestore a config file is created and locked with a lock
property. The following destroy_vm has been impossible as skiplock has not
been set.
Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
This function has been used in one place only into which we inlined its
functionality. Removing it avoids confusion between vm_destroy and vm_destroy.
The whole $importfn is executed in a lock_config_full.
As a consequence, for the inlined code:
1. lock_config is redundant
2. it is not possible that the VM has been started (check_running) in the
meanwhile
Additionally, it is not possible that the "lock" property has been written into
the VM's config file (check_lock) in the meanwhile
Add warning after eval so that it does not go unnoticed if it ever comes into
action.
Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
with qemu 4.0.1, there is now a machine type pc-q35-4.0.1 which does not fit
into our regex
this broke live migration of q35, as we give the machine type (incl version
info) to 'qm start' on the target node, which checks it against the
JSONSchema
to fix this, extend the regex to allow any number of version levels,
for q35, i440fx and virt (to be more future proof)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
As mentioned on the mailing list [0] disks owned by the VM and unused
disks should be removed before the config file is removed.
[0] https://pve.proxmox.com/pipermail/pve-devel/2019-October/039593.html
Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
As mentioned in #2408, live-migrating a VM between storages that use
different scsi backends (scsi-hd, scsi-generic, scsi-block) breaks.
To fix, from QEMU 4.1 machine types onward (to not break current
behaviour any more), only use scsi-hd, as in recent versions, there is
almost no difference between the two anyway.
scsi-block (which potentially also breaks) requires a flag to be
manually set on the disk, so we can assume the user knows what they're
doing.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Suggested-by: Daniel Berteaud <daniel@firewall-services.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Each pending options has a hash value which has the 'force'
information encoded as entry. But, this can be { force => 1 } or
{ force => 0 }, so we actually need to check the value and not just
set force to the hash directly, as else we have force always truthy..
fixes a bug where 'detach' caused disks to be destroyed immediately,
because $force parameter was always true since hash is true.
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
We cannot activate a path, only volume IDs with activate_volumes
(duh)
fixes commit 5c1d42b7f8
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
most of the pending changes related code has been moved into
AbstractConfig, so we have to call them as class methods from QemuConfig instead.
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
brings us more in line with what we do in pve-container, also it's
good to not use file_set_contents directly if we have all those nice
wrapper interface methods to do things in a safe and guaranteed way.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
So, while we could just make this a special case before the
config_to_command call and set the $conf->{vmstate} to the statefile
for the case were it's a valid volumeid, the special case handling
get's much easier when we do this outside of that method.
So it's basically a trade-off, and after looking far to long at all
nice revisions Alwin made for me and Fabians request, and even trying
out different approaches, it was never perfect.
But having slight code duplication over the movement mess I proposed
(as I did not had the full picture then, sorry Alwin) felt like the
slightly nicer trade off, as all worked I just use this one now, it
has very clear semantics, easy to understand and that now three lines
are duplicated is IMO irrelevant.
Co-developed-by: Alwin Antreich <a.antreich@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
the volume id list is only used to activate before real start and
deactivate later, so use it for the vmstate file too.
This not only makes config_to_command have less side effects, it also
ensures that the vmstate is deactivated again
Co-developed-by: Alwin Antreich <a.antreich@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
and use it also for efidisk creation and importdisk
this way we correctly handle zfs-over-iscsi options for those cases
also write tests for it
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
In general it matters where a command line options is positioned
inside a QEMU command, so we want to actually also check the order in
the cfg2cmd test, to do so we need to avoid false positives like this
added.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thanks to Gilberto Nunes for finding a bug where the VM would not start
with foldersharing enabled and the qemu agent option disabled [0].
The cause was that the device org.spice-space.webdav.0 would not find a
virtio-serial-bus in this situation.
Since we always create a virtio-serial-bus for the spice vdagent it
seems sensible to use that also for the foldersharing device by moving
it in front of the other spice devices.
[0]: https://pve.proxmox.com/pipermail/pve-devel/2019-October/039441.html
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
