Modern noVNC does not needs this anymore, actually things may get
worse if it's used. E.g., when one sets 'de' and the VM locale is
'de' you may get a 'ĸ' (unicode kra) if you want to send an ampersand
character through pressing SHIFT + 6.
Qemus manual pages confirms that this is most times not needed
anymore:
> -k language
> Use keyboard layout language (for example "fr" for
> French). This option is only needed where it is not
> easy to get raw PC keycodes (e.g. on Macs, with some
> X11 servers or with a VNC or curses display). You don't
> normally need to use it on PC/Linux or PC/Windows
> hosts.
-- man kvm
An user can always set it per VM, wew simply remove the implict
default derived from the cluster wide datacenter.cfg
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
The git history of this is not immediately obvious due to
the date of the cloud init patches, but the removal of this
line was basically reverted by them later at merge-time.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
We introduced our QMP socket with commit
c971c4f221 (29.05.2012)
Already tried to remove this with commit
7b7c6d1b5d (13.07.2012)
But reverted that to allow migration of VMs still using the old
montior to ones which already switched over to the new QMP one,
in commit dab36e1ee9 (17.08.2012)
see bug #242 for reference
This was all done and released in PVE 2.2, as no migration through
nodes differing more than one major version is possible we can
finally remove this code for good.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Checking for the cgroup directory is a kind of time-of-check
time-of-use race condition stop-mode backups seem to
occasionally run into on some systems.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
With configdrives we end up with the /etc/network/interfaces
file containing the interface names we use on the disk, ie.
eth0/eth1/..., which doesn't work on systems which do not
use this name.
With the 'nocloud' image type we can provide a
network-config in yaml which matches mac addresses. Ideally
we'd use version 2, but debian stretch ships with a too old
cloud-init for this, so for now we're writing version 1.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
move: don't error out with "you can't move a cdrom"
clone: always full-clone cloud-init images
They get completely replaced anyway at the next start, so
there's no point in keeping them.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
*) always replace old cloudinit images
*) apply pending cloudinit changes when generating a new
image
For cloudinit we now always use vdisk_free before
vdisk_alloc in order to always replace old images, this
allows us to hotplug a new drive by setting it to
`none,media=cdrom` first (to eject the disk), then setting
it back to 'storage:cloudinit' to have a new image generated
after applying the currently pending changes.
socat tunnel for nbd mirror was introduce here
https://pve.proxmox.com/pipermail/pve-devel/2017-January/024777.html
to workaround when nbd client was hanging on non responding nbd server.
We have added a 30s timeout on socat tunnel, but when we migrate
multiple disks, it can break migration if for example first disk
is already finished and don't send any new datas in the tunnel.
The connect timeout bug has been fixed in qemu 2.9,
so we can remove the socat tunnel now.
With shared=1, (live) migration ignores the disk and assumes it is
present on all target nodes. This works similar to shared=1 on LXC
mountpoints.
Signed-off-by: Chris Hofstaedtler <chris.hofstaedtler@deduktiva.com>
Reviewed-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Tested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
if the value was '0', we did not append the option to the drive,
resulting in wrong command line if the qemu default of an option is not
'0'
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Skylake-Server is the Xeon variant of Skylake
max is "all features supported by the accelerator in the current host"
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
this have the 'spec-ctrl' flag by default to allow IBRS based Spectre
mitigation by the guest kernel.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Hugepages can take some time to be allocated by qemu at start (60s for 120G of 1G hugepages).
This patch increase start timeout to 5min when hugepages are enabled.
Currently this only allows specifying '+pcid' or '-pcid'
but might be extended in the future.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Fixes: 2bfbee039b ("include format for efidisk")
Reviewed-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Tested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
This also fixes a bug where VMs with no memory defined in the config
where reported as using 0MB instead of 512.
Signed-off-by: Emmanuel Kasper <e.kasper@proxmox.com>
when having an unused disk on a storage for which there are multiple
definitions, we added it again on another storage when that storage
was alphabetically before the already existing one
this happens for example when using our automatically generated
ceph storages: 'pool_ct' and 'pool_vm' and having a vm with
an unused disk
with this patch, we also leave the unused disks in the hash
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
this means that we do not include the '-k' parameter anymore by default
(which is deprecated by qemu)
with this, noVNC and spice always respect the guest keyboard
configuration and altgr keys work without problems
tested:
ubuntu with english intl and german with novnc and spice
windows 10 with english intl and german with novnc and spice
live migration
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
since the guest-fsfreeze-freeze command has a timeout of 1 hour,
we want to check if the guest-agent even runs before executing that,
or else we wait 1 hour and then continue
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Reviewed-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
if the efidisk is in 'raw' format, qemu will prevent writes
on block zero if the format is not explicitely given
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
'These options take an integer value and control the "cpu.shares"
control group attribute. The allowed range is 2 to 262144. Defaults to
1024.' – man 5 systemd.resource-control
we only checked if a vm had in use base disks when deleting them,
at which point we do not stop to delete the vm even when a
disk deletion fails, which means we could successfully delete the config
and all not used (base) disks of a template, resulting in left over vm disks
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
perls 'local' must be either used in front of each $SIG{...}
assignments or they must be put in a list, else it affects only the
first variable and the rest are *not* in local context.
In all cases the global signal handlers we overwrote were in cli programs or
forked workers, not in daemons.
this was only kept for PVE 4.X where the switch to the newer OVMF
image with actual working persisten EFIVARS was made.
We do not ship this old image in PVE 5.0 anymore so remove this
legacy code as it can never trigger anyhow.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
factor out code in a new create_efidisk submethod, as else this code
is hardly readable as the efidisk0 case is a special case. Refer from
putting all this specialised handling directly to the much shorter
code for all other cases.
Also the disk was created with a specific format and then a format
detection on the newly created disk was done, which is pretty
useless, clear that up.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
perls 'local' must be either used in front of each $SIG{...}
assignments or they must be put in a list, else it affects only the
first variable and the rest are *not* in local context.
This may cause weird behaviour where daemons seemingly do not get
terminating signals delivered correctly and thus may not shutdown
gracefully anymore.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Moved the check to the beginning of the function.
VMs configured to use KVM won't start if KVM is not available.
VMs not configured to use KVM will start regardless.
Fixes a typo in the function name and removes the $nokvm parameter, as it's only
used to immideately exit the function. Instead calling the function
conditionally.
If we get an VM machine older than 2.9 we use the old selection
expression for the VGA type. This allows to live migrate VMs to PVE
5.0 from beta 1 and PVE 4.4 again.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
the syntax was wrong, it was (e.g. for iops-write):
throttling.iops-write=-max100
instead of
throttling.iops-write-max=100
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
QemuServer::lspci() iterates over /sys/bus/pci/devices which
doesn't guarantee any order which means functions sometimes
ended up in the wrong order and it was never clear which
one would get the additional options such as x-vga passed
to them.
bps_max_length & friends were wrongly named and were only
passed to qemu when hot-applying changes. They can only
be passed via the command line with their new names. For
consistency let's rename them all, that way they're all in
one place.
Fixes#1195 (for real this time).
We cannot look for ports on "any" wildcard address while
letting qemu bind to "localhost", this may lead to a qemu
process occupying ::1 while the next search successfully
finds the same port available for IPv4's '*' address.
Instead, we now lookup the IP of the desired family for
'localhost'. Note that while we could simply be hardcoding
::1 or 127.0.0.1, with this code we are protocol agnostic.
For calculation of "current_core" the input variable id is decremented.
For calculation of "current_socket" this decrement was missing resulting
in a wrong value when "cores" is set to 1.
Signed-off-by: Tobias Böhm <tb@robhost.de>
This patch will include all necessary properties for the replication.
Also will it enable and disable a replication job
when appointed flags are set or deleted.
this (correctly!) errored out with Qemu 2.9 when live-migrating
local disks, because the NBD server blocks the VM from being
resumed. was probably missed when migrating via unix domains
was originally introduced..
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
This was checking for scsihw being set in both branches
whereas lsi is also the default. Added the missing 'not'.
Fixes a bug where a VM with a disk with a scsi index >= 7
refused to start due to an invalid scsi id.
Reported-by: Friedrich Ramberger <f.ramberger@proxmox.com>
since it can cause I/O errors and data corruption in low
memory or highly fragmented memory situations since Qemu 2.7
use scsi-hd by default instead
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
if qga is enabled, we try to freeze the fs before cancelling block job.
if not , we pause the vm.
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
This will create a new drive for each local drive found,
and start the vm with this new drives.
if targetstorage == 1, we use same sid than original vm disk
a nbd server is started in qemu and expose local volumes to network port
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
we can use multiple drive_mirror in parralel.
block-job-complete can be skipped, if we want to add more mirror job later.
also add support for nbd uri to qemu_drive_mirror
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
otherwise we end up with undeletable VM configs in case
vdisk_free fails (which could happen because of cluster-wide
lock contention, storage problems, ..).
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
When trying to migrate a VM from a node with qemu server <= 4.0-92 to
a node with qemu server >= 4.0-93 we failed as the remote qemu-server
got no explicit migration_type' from the older qemu server on the
source.
Check if migration_type is defined on a incoming migration start, if
not set it.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
All special flags for Windows 8 and Windows 2012 (win8 type)
are kept the same , since we set flags based on checking if
/^win(\d+)$/ is greater than 6 or 7
like for snapshot, we need to check if krbd is enabled, to known
if we need to use qmp delete-drive-snapshot or storage command directly
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
This cleanup windows guest os version handling,
with normalizing ostype with numbers in a new windows_version sub.
if($ostype eq 'wxp' || $ostype eq 'w2k3' || $ostype eq 'w2k') {
$winversion = 5;
} elsif($ostype eq 'w2k8' || $ostype eq 'wvista') {
$winversion = 6;
} elsif ($ostype =~ m/^win(\d+)$/) {
$winversion = $1;
}
so we can simply do test on windows version with lower or upper version
Hyperv enlightments configuration is centralized
in a new add_hyperv_enlighments sub.
Also disable hyperv with win < 8 + ovmf.
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
Without this patch we use the network were the cluster traffic runs
for sending migration traffic. This is not ideal as it may hinder
cluster traffic. Further some users have a powerful network which
would be perfect for migrations, with this patch they can run the
migration traffic over such a network without having the corosync
traffic on the same network.
The network is configurable through /etc/pve/datacenter.cfg which
got a new property, namely migration. migration has two
subproperties: type (replaces the old migration_unsecure property)
and network.
For the case of a network failure or that a VM has to be moved over
another network for arbitrary other reasons I added the
migration_type and migration_network parameters to qm migrate (and
respectively vm_start as this gets used on migration).
They allow overwriting the datacenter.cfg settings.
Fixes bug #1177
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Let 'cdrom' use the pve-qm-ide format, as it's supposed to
be an alias to ide2.
We're not using the 'alias' schema property since the qemu
configs still use a custom parser (due to the
pending-changes system and the filename-to-volume-id
conversion for legacy support) which does not deal with
schema aliases.
when restoring into an existing VM, we don't want to die
half-way through because we can't delete one of the existing
volumes. instead, warn about the deletion failure, but
continue anyway. the not deleted disk is then added as
unused automatically.
this adds a bootsplash image in /usr/share/qemu-server
and if this file exists, use it for seabios
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
if efidisk0 is defined, use it as a efivars disk,
to permanently store efivars (such as boot options)
we check if the files exist, and act accordingly
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
just a simple disk (only size, format and volid) for
efivars disk
also do not add it to command line in foreach_drive
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
drive-mirror is not working with qemu 2.6 when iothread is enabled.
with virtio-blk : mirror is working, but block-job-completed crash the vm
with virtio-scsi : mirror hang at start.
This should be fixed in qemu 2.7
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
we have a few problems with hotplug at the moment:
qemu may add usb hubs when adding usb devices but fails to remove them
when removing the usb device (this is a qemu bug)
also when starting a guest with a usb device we add ehci and uchi
controllers, which we cannot hot unplug
with those devices, it is impossible to live migrate the guest
to another host, meaning even if you remove all usb devices,
the migrate fails
so we deactivate usb hotplugging for now
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
this patch introduces working usb hotplugging
you can now add a usb device while a vm is running
this does not work with spice at the moment, only
with usb passthrough
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
since usb devices do not have their own
"query" command in qmp, we have to use
qom-list /machines/peripheral
which essentially gets a list of peripheral devices of
the vm
there we only get the usb devices
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
vm configuration
----------------
hugepages: (any|2|1024)
any: we'll try to allocate 1GB hugepage if possible, if not we use 2MB hugepage
2: we want to use 2MB hugepage
1024: we want to use 1GB hugepage. (memory need to be multiple of 1GB in this case)
optionnal host configuration for 1GB hugepages
----------------------------------------------
1GB hugepages can be allocated at boot if user want it.
hugepages need to be contiguous, so sometime it's not possible to reserve them on the fly
/etc/default/grub : GRUB_CMDLINE_LINUX_DEFAULT="quiet hugepagesz=1G hugepages=x"
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
Acked-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
add them by default for qemu 2.6
(support is already present in qemu 2.5, but we don't want to break live migration for current running vm)
vpindex && runtime need host kernel 4.4
Theses 3 enlightements are needed by windows to use vmbus
http://searchwindowsserver.techtarget.com/definition/Microsoft-Virtual-Machine-Bus-VMBus
details :
- When Hyper-V "vpindex" is on, guest can use MSR HV_X64_MSR_VP_INDEX
to get virtual processor ID.
- Hyper-V "runtime" enlightement feature allows to use MSR
HV_X64_MSR_VP_RUNTIME to get the time the virtual processor consumes
running guest code, as well as the time the hypervisor spends running
code on behalf of that guest.
- Hyper-V "reset" allows guest to reset VM.
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
We cannot guarantee when the SSH forward Tunnel really becomes
ready. The check with the mtunnel API call did not help for this
prolem as it only checked that the SSH connection itself works and
that the destination node has quorum but the forwarded tunnel itself
was not checked.
The Forward tunnel is a different channel in the SSH connection,
independent of the SSH `qm mtunnel` channel, so only if that works
it does not guarantees that our migration tunnel is up and ready.
When the node(s) where under load, or when we did parallel
migrations (migrateall), the migrate command was often started
before a tunnel was open and ready to receive data. This led to
a direct abortion of the migration and is the main cause in why
parallel migrations often leave two thirds or more VMs on the
source node.
The issue was tracked down to SSH after debugging the QEMU
process and enabling debug logging showed that the tunnel became
often to late available and ready, or not at all.
Fixing the TCP forward tunnel is quirky and not straight ahead, the
only way SSH gives as a possibility is to use -N (no command)
-f (background) and -o "ExitOnForwardFailure=yes", then it would
wait in the foreground until the tunnel is ready and only then
background itself. This is not quite the nicest way for our special
use case and our code base.
Waiting for the local port to become open and ready (through
/proc/net/tcp[6]] as a proof of concept is not enough, even if the
port is in the listening state and should theoretically accept
connections this still failed often as the tunnel was not yet fully
ready.
Further another problem would still be open if we tried to patch the
SSH Forward method we currently use - which we solve for free with
the approach of this patch - namely the problem that the method
to get an available port (next_migration_port) has a serious race
condition which could lead to multiple use of the same port on a
parallel migration (I observed this on my many test, seldom but if
it happens its really bad).
So lets now use UNIX sockets, which ssh supports since version 5.7.
The end points are UNIX socket bound to the VMID - thus no port so
no race and also no limitation of available ports (we reserved 50 for
migration).
The endpoints get created in /run/qemu-server/VMID.migrate and as
KVM/QEMU in current versions is able to use UNIX socket just as well
as TCP we have not to change much on the interaction with QEMU.
QEMU is started with the migrate_incoming url at the local
destination endpoint and creates the socket file, we then create
a listening socket on the source side and connect over SSH to the
destination.
Now the migration can be started by issuing the migrate qmp command
with an updated uri.
This breaks live migration from new to old, but *not* from old to
new, so there is a upgrade path.
If a live migration from new to old must be made (for whatever
reason), use the unsecure_migration setting (man datacenter.conf)
to allow this, although that should only be done in trusted network.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
With systemd-run qemu's --daemonize forks often happen
before systemd finishes setting up the scopes, which means
the limits we apply often don't work.
We now use enter_systemd_scope() to create the scope before
running qemu directly without systemd-run.
Note that vm_start() runs in a forked-worker or qm cli
command, so entering the scope in such a process should not
affect the rest of the pve daemon.
if we got an option which was not valid, we still
wrote it to the config, and subsequently returned
it on every api call
instead, now we die instead of warn and do not accept
invalid options
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
