Commit Graph

167 Commits

Author SHA1 Message Date
Thomas Lamprecht
2c29655663 Revert "config_aware_timeout: add 5s if TPM is configured"
This reverts commit d4e1e1f862.

It's bogus, the VM start timeout is only starting to tick after we
started the TPM already...
2021-10-18 09:47:42 +02:00
Thomas Lamprecht
d4e1e1f862 config_aware_timeout: add 5s if TPM is configured
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-18 09:43:46 +02:00
Thomas Lamprecht
c077cc166e cloudinit: opennebula: refactor to reduce code bloat
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-15 19:58:16 +02:00
Thomas Lamprecht
2eee6748a0 cloudinit: better use of string variable interpolation
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-15 19:58:16 +02:00
Thomas Lamprecht
d01de38cb6 pci: prepare: improve no-IOMMU error message
give some context

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-15 19:58:16 +02:00
Thomas Lamprecht
a01593676c pci reservation: rework helpers style and readability wise
both style and readability are naturally subjective to a certain
degree...

Also, this patch mixes a bit much into one thing, but splitting that
up would mean lots of work I just wanted to avoid, sorry about that.

Among other things:

- avoid a level of indentation in the reserve loop
- rename pciids to reservation_list where it was a better fit
- make reserve set either pid or time to avoid suggesting that we
  save both
- rename parameters to requested/dropped IDs for easier understanding
  what's going on in the code
- avoid old_pid/pid, use running_pid and reserver_pid instead to
  clarify what they actually mean
- drop useless returns to avoid suggesting the return value has any
  use and save some lnes
- use a hash slice to delete all dropped IDs at once, shorter and
  faster
- use 5 second timeout for reservation, this does nothing intensive
  nor does it wait for anything, so the critical section should be
  really short, 5s is really long enough for a wait..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-15 19:58:16 +02:00
Thomas Lamprecht
bda0ebff2d pci reservation: move lock/reservation file into /run/qemu-server
lck needs to die, the days of any 8.3 file naming schemes are long
gone (in the server space that is ;)

/var/run is /run so use the shorter, and while /var/lock is a OK
place for the locks we try to keep lock and lock-object together
nowadays. The qemu-server sub-directory avoids overly cluttering the
already crowded top-level /run dir

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-15 18:17:34 +02:00
Thomas Lamprecht
cda95d5223 pci reservation: encode locklessness of parsers in name
to avoid that they're misused

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-15 14:44:50 +02:00
Dominik Csapak
3bfee796f4 pci: add helpers to (un)reserve pciids for a vm
saves a list of pciid <-> vmid mappings in /var/run
that we can check when we start a vm

if we're not given a pid but a timeout, we save the time when the
reservation will run out (current time + timeout + 5s) since each
vm start (until we can save the pid) varies from config to config

reserve_pci_usage and remove_pci_reservation always expect a list of ids
so that we can update the reservation for a vm all at once

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-10-11 09:07:52 +02:00
Thomas Lamprecht
71cb8e0f87 pci related code cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-11 08:39:28 +02:00
Thomas Lamprecht
e2b42bee6d pci: use local helper to generated generate_mdev_uuid
avoid (API) leaking qemu-server specific stuff into pve-common

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-11 08:38:28 +02:00
Thomas Lamprecht
82712fcd3c pci: prepare_pci_device: fixup parameter name
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-11 08:37:35 +02:00
Dominik Csapak
acd4b77745 pci: refactor pci device preparation
makes the vm start a bit less crowded

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-10-08 06:27:19 +02:00
Thomas Lamprecht
a064e5117f efi: use vendor-agonstic "pre-enrolled-keys" + description fix
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-05 18:35:25 +02:00
Stefan Reiter
b5099b4f6c ovmf: support secure boot with 4m and 4m-ms efidisk types
Provide support for secure boot by using the new "4m" and "4m-ms"
variants of the OVMF code/vars templates. This is specified on the
efidisk via the 'efitype' and 'ms-keys' parameters.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-10-05 18:04:03 +02:00
Dominik Csapak
a4d5b84c9c pci: to not capture first group in PCIRE
we do not need this group, but want to use the regex where we have
multiple groups, so make it a non-capture group

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-10-05 16:14:42 +02:00
Thomas Lamprecht
2b9ee9441a trivial: indentation/formatting fixup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-05 07:05:07 +02:00
Stefan Reiter
f9dde219f2 fix #3075: add TPM v1.2 and v2.0 support via swtpm
Starts an instance of swtpm per VM in it's systemd scope, it will
terminate by itself if the VM exits, or be terminated manually if
startup fails.

Before first use, a TPM state is created via swtpm_setup. State is
stored in a 'tpmstate0' volume, treated much the same way as an efidisk.

It is migrated 'offline', the important part here is the creation of the
target volume, the actual data transfer happens via the QEMU device
state migration process.

Move-disk can only work offline, as the disk is not registered with
QEMU, so 'drive-mirror' wouldn't work. swtpm itself has no method of
moving a backing storage at runtime.

For backups, a bit of a workaround is necessary (this may later be
replaced by NBD support in swtpm): During the backup, we attach the
backing file of the TPM as a read-only drive to QEMU, so our backup
code can detect it as a block device and back it up as such, while
ensuring consistency with the rest of disk state ("snapshot" semantic).

The name for the ephemeral drive is specifically chosen as
'drive-tpmstate0-backup', diverging from our usual naming scheme with
the '-backup' suffix, to avoid it ever being treated as a regular drive
from the rest of the stack in case it gets left over after a backup for
some reason (shouldn't happen).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-10-05 06:51:02 +02:00
Constantin Herold
ae776a6288 fix #3581: pass size via argument for memory-backend-ram qmp call
Signed-off-by: Constantin Herold <proxmox8914@herold.me>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-08-18 10:41:04 +02:00
Dominic Jäger
8717d89d92 Fix #3371: parse ovf: Allow dots in VM name
Dots are allow in PVE VM names, so they should not be dropped during import.

Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
2021-07-29 17:17:39 +02:00
Fabian Grünbichler
75748d4492 drive: factor out read-only helper
we also need it for efidisks.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-06-23 12:27:54 +02:00
Stefan Reiter
59e5934270 enable io-uring support
Note that the value in this enum directly represents the value passed to
QEMU, so we need to use the underscore.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-23 12:01:59 +02:00
Dominik Csapak
30664f14ff fix bootdisk_size for new bootorder config scheme
Previously, we ever only had a single boot *disk*, while possibly
having multiple cdroms/nics in the boot order

e.g. the config:

 boot: dnc
 bootdisk: scsi0
 ide0: media=cdrom,none
 scsi0: xxx
 net0: ...

would return the size of scsi0 even though it would first boot
from cdrom/network.

When editing the bootorder with such a legacy config, we
remove the 'bootdisk' property and replace the legacy notation
with an explicit order, but we only search the first disk
for the size now.

Restore that behaviour by iterating over all disks in the boot
order property string until we get one that is not a cdrom
and has a size.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-29 16:15:33 +02:00
Fabian Ebner
ab5b97d8a8 drive: volume in-use check: remove unused closure parameter
and simplify the calling iteration.

Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-04-19 22:09:17 +02:00
Stefan Reiter
d4be7f31b5 cfg2cmd: fix +pveN machine types with pxe
Pinned machine versions like "pc-i440fx-4.2+pve2.pxe" would otherwise
get a second "+pve0" suffix, which is incorrect.

Also deal with non-pve pinned versions correctly, i.e.
"pc-i440fx-5.2.pxe" becomes "pc-i440fx-5.2+pve0.pxe".

Handle .pxe suffixes in Machine.pm as well, and add two test cases.

Co-developed-by: Luca Berneking <luca@berneking.net>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-18 17:58:56 +02:00
Fabian Ebner
a0d8d8592a drive: volume in-use check: fix fallback path comparison
When checking whether a volume is still referenced by a snapshot, the volid
itself is first checked. When the volid is different, we fall back to comparing
the path.

As the first value to be compared is a volume's path, the second value better be
a volume's path too, and not a snapshot's path.

See also 77019edfe0 for historical context.

The error that led me here:
* had a VM with ZFS over iSCSI storage with an exsiting snapshot
* add new unused drive
* try to remove the unsued drive
* fails, because ZFS (not Pool!) Plugin does not support snapshot paths.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-04-18 17:26:05 +02:00
Fabian Ebner
98c3d99e64 schema: mention special syntax for allocating a new volume
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-03-30 18:37:16 +02:00
Mira Limbeck
988be8d052 fix #2670: cloudinit enable SLAAC
cloud-init's SLAAC option was disabled in 2018 because there was no
support for it. Now that cloud-init 19.4 or newer versions are more
widespread, we can finally reenable it.

Also include minimum required cloud-init version for SLAAC support in
format description.

Tested on Ubuntu 20.04 (ci 20.4), CentOS 8 (ci 19.4), Debian 10 (ci
20.2).

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2021-03-30 18:25:06 +02:00
Mira Limbeck
617a864ac2 fix #3314: IPv6 requires type 'static6'
A fix was also provided in bugzilla by user wsapplegate:
https://bugzilla.proxmox.com/show_bug.cgi?id=3314

Tested on Ubuntu 20.04, CentOS 8 and Debian 10.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2021-03-30 18:25:06 +02:00
Thomas Lamprecht
9edb618257 can_run_pve_machine_version: PVE version can really be optional
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-05 18:49:06 +01:00
Fabian Ebner
ea71be24d6 machine: split out helper for handling query-machines qmp command result
to be re-used in the vmstatus() call.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Reviewed-by: Stefan Reiter <s.reiter@proxmox.com>
2021-03-04 13:57:11 +01:00
Alexandre Derumier
545eec65cd cloudinit: add opennebula config format
This is an alternative format for cloudinit use by opennebula,
https://cloudinit.readthedocs.io/en/latest/topics/datasources/opennebula.html

but it can be also used by opennebula context scripts

https://github.com/OpenNebula/addon-context-linux
https://github.com/OpenNebula/addon-context-windows

This context scripts are simple udev trigger/bash scripts
and allow live configuration changes.

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2021-02-06 14:44:38 +01:00
Stefan Reiter
4893f9b970 anchor CPU flag regex to avoid arbitrary flag suffixes
Previously one could specify a CPU flag like 'pcidfoobar' and it would
be accepted, even though we attempt to filter VM-only flags for
security. AFAICT none of the flags we allow can be turned into any
others just by appending text, but better safe than sorry.

Reported-by: Oguz Bektas <o.bektas@proxmox.com>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-01-26 19:27:05 +01:00
Wolfgang Bumiller
114d2e765a add PVE::QemuServer::Cgroup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-24 12:00:33 +01:00
Thomas Lamprecht
6ef6d68f6c cloudinit: use normal grep
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-19 15:32:03 +02:00
Thomas Lamprecht
41af2dfc25 PCI: use warnings/strict and fix setting $vga from config2command
fixes commit 74c17b7a23 which moved
this code here, but forgot to pass $vga ref, as the module was not
using warning nor strict mode this was not caught..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-16 18:03:32 +02:00
Thomas Lamprecht
f7d1505b0c tree wide cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-16 18:03:32 +02:00
Thomas Lamprecht
d1c1af4b02 tree wide cleanup of s/return undef/return/
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-16 16:20:05 +02:00
Stefan Reiter
2141a802b8 fix #3010: add 'bootorder' parameter for better control of boot devices
(also fixes #3011)

Deprecates the old-style 'boot' and 'bootdisk' options by adding a new
'order=' subproperty to 'boot'.

This allows a user to specify more than one disk in the boot order,
helping with newer versions of SeaBIOS/OVMF where disks without a
bootindex won't be initialized at all (breaks soft-raid and some LVM
setups).

This also allows specifying a bootindex for USB and hostpci devices,
which was not possible before. Floppy boot support is not supported in
the new model, but I doubt that will be a problem (AFAICT we can't even
attach floppy disks to a VM?).

Default behaviour is intended to stay the same, i.e. while new VMs will
receive the new 'order' property, it will be set so the VM starts the
same as before (using get_default_bootorder).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-10-14 12:30:50 +02:00
Stefan Reiter
5cfa9f5f73 add new 'boot' property format and introduce legacy conversion helpers
The format is unused in this commit, but will replace the current
string-based format of the 'boot' property. It is included since the
parameter of bootorder_from_legacy follows it.

Two helper methods are introduced:
* bootorder_from_legacy: Parses the legacy format into a hash closer to
    what the new format represents
* get_default_bootdevices: Encapsulates the legacy default behaviour if
    nothing is specified in the boot order

resolve_first_disk is simplified and gets a new $cdrom parameter to
control the behaviour of excluding CD-ROMs or instead searching for only
them.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-10-14 12:30:50 +02:00
Fabian Ebner
b2e813a6d5 validate_cpu_conf: return config
as parse_property_string expects it to.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Reviewed-by: Stefan Reiter <s.reiter@proxmox.com>
2020-08-12 13:54:38 +02:00
Fabian Ebner
c15b597137 Add dot character to CPU commandline regular expression
to be able to parse CPU features like "+sse4.2" correctly.

Reported here: https://forum.proxmox.com/threads/zsf-snapshot-unable-to-parse-value-of-runningcpu-got-undefined-value.74094/

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-08-11 12:58:43 +02:00
Dominik Csapak
7de7f675c2 fix mdev cmdline generation
during refactoring, the vmid got lost, but is necessary to get
the correct mdev id

Fixes commit 74c17b7a23
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
[ reference fixed commit ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-13 10:29:25 +02:00
Stefan Reiter
7b8c4de3b8 fix #2671: include CPU format in man page again
Use the new register_format(3) call to use a validator (instead of a
parser) for 'pve-(vm-)?cpu-conf'. This way the $cpu_fmt hash can be used for
generating the documentation, while still applying the same verification
rules as before.

Since the function no longer parses but only verifies, the parsing in
print_cpu_device/get_cpu_options has to go via JSONSchema directly.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-07-08 10:28:20 +02:00
Fabian Ebner
9b29cbd0ed update_disksize: make interface leaner
Pass new size directly, so the function doesn't need to know about
how some hash is organized. And return a message directly, instead
of both size-strings. Also dropped the wantarray, because both
existing callers use the message anyways.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-07-01 09:18:13 +02:00
Thomas Lamprecht
1fac3a0b31 pci: whitespace, indentation and formating fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-25 13:33:26 +02:00
Stefan Reiter
13d689792e fix #2794: allow legacy IGD passthrough
Legacy IGD passthrough requires address 00:1f.0 to not be assigned to
anything on QEMU startup (currently it's assigned to bridge pci.2).
Changing this in general would break live-migration, so introduce a new
hostpci parameter "legacy-igd", which if set to 1 will move that bridge
to be nested under bridge 1.

This is safe because:
* Bridge 1 is unconditionally created on i440fx, so nesting is ok
* Defaults are not changed, i.e. PCI layout only changes when the new
parameter is specified manually
* hostpci forbids migration anyway

Additionally, the PT device has to be assigned address 00:02.0 in the
guest as well, which is usually used for VGA assignment. Luckily, IGD PT
requires vga=none, so that is not an issue either.

See https://git.qemu.org/?p=qemu.git;a=blob;f=docs/igd-assign.txt

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-25 13:25:35 +02:00
Stefan Reiter
74c17b7a23 cfg2cmd: hostpci: move code to PCI.pm
To avoid further cluttering config_to_command with subsequent changes.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-25 13:25:35 +02:00
Stefan Reiter
1b7824d349 CPUConfig: add add_cpu_json_properties()
Useful for APIs and docs.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-18 17:11:47 +02:00
Mira Limbeck
d4fa9981b8 fix #2748: make order of interfaces consistent
As perl hashes have random order, sort them before iterating through.
This makes the output of 'qm cloudinit dump <vmid> network' consistent
between calls if the config has not changed.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2020-06-02 11:45:35 +02:00