Commit Graph

253 Commits

Author SHA1 Message Date
Thomas Lamprecht
e07c055df4 revert "fix #4333: redirect API handler output to STDERR"
Redirecting stdout is not a feasible approach, as that also affects
all run_commands and other command executions/forks done by the API
handler, and thus breaks parsing outputs of such command executions
in the API handlers.

We plan to add a `--result-fd` option instead, allowing users to pass
their own file, open FD or named pipe to the pvesh, so that they can
process the output in streaming or in full afterward afterwards.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-03-14 11:21:29 +01:00
Thomas Lamprecht
5ea98762aa pvesh: don't bloat the code on comments
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-03-11 18:10:09 +01:00
Fabian Grünbichler
89fba14715 fix #4333: redirect API handler output to STDERR
otherwise, print/warn statements by the API endpoint are included in the
output, which breaks JSON parsing in case of output-format == json*.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-03-11 18:08:00 +01:00
Moayad Almalat
7d8a9a99fc fix #3037: include the split_list to shell_qoute
Based to the suggestion of Wolfgang, in regard to `split_list()`,
 I converted the `split_list()` to `split(/\0/, $param->{$key});`
 this will split the `$param->{$key}` null characters and push each
 element to the `$args` array along with the key value.

 changes since v1:
 * get rid of the `use PVE::Tools qw(split_list);` since not need it anymore.
 * replace the split_list to split(/\0/).

Signed-off-by: Moayad Almalat <m.almalat@proxmox.com>
2023-01-25 09:30:13 +01:00
Fabian Grünbichler
774c5031b4 pvesubscription: remove leftover $key
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-09 15:06:26 +02:00
Fabian Grünbichler
d4df1b145c pvesubscription: add 'set-offline-key' command
and disallow updating signed offline keys via regular check/update code
path.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-06 11:02:59 +02:00
Fiona Ebner
55b1af44fd pvesr: prepare local job: remove stale replicated volumes immediately
Commit d8cd8e8cf9 introduced a
regression where only stale replicated volumes with an older
timestamp would be cleaned up. This meant that after removing a volume
from the guest config, it would only be cleaned up the second time the
replication ran afterwards. And the volume could become completely
orphaned in case the relevant storage wasn't used by the job anymore.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-08-02 11:05:06 +02:00
Fiona Ebner
e3e0e09637 pvesr: rename last_snapshots to local_snapshots
while dropping the instance where the local variable was unused.
prepare() was changed a while ago to return all local snapshots.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-08-02 11:04:55 +02:00
Thomas Lamprecht
b23305d5ad pveceph install: clarify comment for conditional ceph-volume inclusion
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-07-05 08:38:51 +02:00
Stefan Sterz
3409fbeaff pveceph install: exclude 'ceph-volume' from older than quincy releases
when installing non-quincy versions, 'ceph-volume' is not contained in
the respective repositories and, thus, the install process would fail.

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
 [ T: reworded commit subject ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-07-05 08:19:28 +02:00
Thomas Lamprecht
2812907f23 pveceph: install split-out ceph-volume package
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-07-04 13:30:51 +02:00
Thomas Lamprecht
7271e6f67d pveceph: reuse supported ceph versions for non-default warning
albeit this is a bit odd safety "check" anyway

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-21 17:04:49 +02:00
Thomas Lamprecht
983921b9a1 pveceph: factor out supported/default ceph version to single source
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-21 17:04:49 +02:00
Thomas Lamprecht
ef25743f04 pveceph: warn on (soon) EOL octopus
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-21 16:52:54 +02:00
Thomas Lamprecht
9e81f36459 pveceph: add quincy installation support
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-21 16:52:25 +02:00
Oguz Bektas
61a687ac13 fix #3961: pveam: error on 'list' if storage doesn't exist
storage_check_enabled() already dies with an appropriate error message
so we don't have to handle it here

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2022-06-14 10:12:35 +02:00
Fabian Ebner
0782c03469 pveam: remove: add content type check
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-01 09:24:35 +02:00
Thomas Lamprecht
0b1d64764f pve6to7: update expected running kernel version to 5.13 or 5.15
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-14 08:19:55 +01:00
Dominik Csapak
02c1e98e83 pveceph: add 'fs destroy' command
with 'remove-storages' and 'remove-pools' as optional parameters

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 17:52:08 +01:00
Thomas Lamprecht
d380d00008 ceph: code/whitespace cleanups
avoid line bloat, use same capitalization style in warnings as (most)
of the rest of code, some style nits

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-05 08:01:33 +02:00
Thomas Lamprecht
9f6dc0750a ceph: use method to have a single source for install flag file
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-05 08:00:46 +02:00
Aaron Lauterer
4dd27d5034 ceph install: improve ceph install checks during installation
Adding a flag file during the Ceph installation helps to cover the time
span in which the binary is already present but the installation not yet
done.

The most noticeable effect is that the 'Next' button in the GUI will
only become active once the installation is actually finished and not
earlier.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2021-10-05 07:42:57 +02:00
Thomas Lamprecht
81dbe056fc drop pve5to6 checker tool, not relevant in 7.x release
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-04 12:22:02 +02:00
Rhonda D'Vine
b8436f0c6b Only check deb sources.list entries
The check isn't specific enough, it also catches deb-src entries and
would give a false impression of security in certain circumstances, or
lead to false negatives in case you have a deb-src entry for
buster/updates even though you have bullseye-security in just the next
line -- something that isn't that uncommon for developers.

Signed-off-by: Rhonda D'Vine <rhonda@deb.at>
Reviewed-by: Fabian Ebner <f.ebner@proxmox.com>
2021-07-13 06:33:45 +02:00
Fabian Ebner
d8aa2f664d pve6to7: storage content: ignore misconfigured unreferenced volumes
If the same local storage is configured twice with content type
separation, migration in PVE 6 would lead to the volumes being
duplicated. As that would happen for every migration, such an issue
would likely be noticed already, and in PVE 7 such configuration is
not problematic for migration anymore. Also, misconfigured
unreferenced volumes are not an issue with respect to the upgrade
itself, just drop the check.

It's not necessary to scan storages with either 'images' or 'rootdir'
anymore, as only the log_info() remains.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-07 18:44:54 +02:00
Fabian Ebner
fcae887b85 pve6to7: storage content: skip scanning storage if shared
Shared storages are not scanned for migration either, so they cannot
be problematic in this context. This could lead to false positives
where it actually is completely unproblematic:

https://forum.proxmox.com/threads/proxmox-ve-7-0-released.92007/post-401165

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-07-07 18:43:42 +02:00
Fabian Ebner
759d1a6ba8 pve6to7: add check for Debian security repository
since the pattern for the suite changed.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-07-07 18:10:50 +02:00
Thomas Lamprecht
71202973d3 pve6to7: avoid parsing configs to often, merge lxc.cgroup check and note length one
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-07 12:45:39 +02:00
Thomas Lamprecht
06f915e63f pve6to7: remember if cgroup is forced to legacy
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-07 12:40:43 +02:00
Stoiko Ivanov
2445be40e3 pve6to7: add check for 'lxc.cgroup.' keys in container config
The check is rather straight forward - and might help users who
passthrough devices to their containers.

Reported in our community forum:
https://forum.proxmox.com/threads/pve-7-0-lxc-intel-quick-sync-passtrough-not-working-anymore.92025/

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2021-07-07 12:20:08 +02:00
Fabian Grünbichler
6d451a4bc3 pve6to7: dont guard noout check on Ceph version
we don't have a mandatory Ceph major version upgrade this time around,
so this check does not make sense. instead, we want noout until the full
cluster is upgraded. let's use the simple approach and just flip the
switch to "turn off noout if all of Ceph is a single version" in the PVE
7.x branch.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-07-06 14:20:46 +02:00
Fabian Grünbichler
a2e5da3fdf pve6to7: check for >= Octopus
and drop the Nautilus OSD upgrade check while we are at it..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-07-06 14:20:46 +02:00
Fabian Grünbichler
1ba568d67e pve6to7: remove PASS noise for ceph
these were mostly relevant for the Luminous -> Nautilus upgrade, and we
don't need to list all the default passing states that our tooling sets
up anyway.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-07-06 14:20:46 +02:00
Fabian Grünbichler
3bc032ad0e pve6to7: use new flags API
the old one is not available post-upgrade, let's use a single codepath
for this.

the new API only allows querying user-settable flags, but the only flags
we check besides 'noout' are not relevant for an upgrade of PVE 6.x to
7.x (PVE 6.x only supports Nautilus+ which requires these flags to be
set in order to work) so we can just drop those outdated checks instead
of extending/refactoring the API.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-07-06 14:20:46 +02:00
Thomas Lamprecht
222db0f513 pve6to7: rework full-check skip-message slightly
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-05 17:38:32 +02:00
Thomas Lamprecht
1d5529612f pve6to7: rework cgroup message slightly
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-05 17:38:16 +02:00
Thomas Lamprecht
2331534996 pve6to7: try to iterate in a more stable fashion
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-05 17:37:58 +02:00
Thomas Lamprecht
ca0c50a1e6 pve6to7: handle undefined ostype
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-05 17:37:40 +02:00
Stoiko Ivanov
3d779b1274 pve6to7: add 'full' parameter for expensive checks
and place the container cgroupv2 support checks behind it.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2021-07-05 17:06:43 +02:00
Stoiko Ivanov
9f5f981914 pve6to7: check for containers not supporting pure cgroupv2
Helpers copied from pve-container to avoid versioned bumps.

Early returns when no containers are running, or the containers don't
use systemd, as well as returning after finding the first affected
container to minimize impact and resource usage.

Checking running containers first since following /proc/<pid>/root is
cheaper than mounting all volumes for a container

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2021-07-05 17:06:43 +02:00
Fabian Ebner
08e22c1eac pveceph: also install nvme-cli
The nvme-cli package is recommended by (our) Ceph packages, but here
--no-install-recommends is used to avoid pulling in too much.

The issue with not installing nvme-cli is that a "security
information" mail notification is triggered by sudo each time Ceph
tries to get the device health metrics. While there is a sudoers
rule for /usr/sbin/nvme, Ceph uses 'sudo nvme ...', so it does not
apply when the package is not installed.

This didn't seem to happen with sudo in buster.

It's about 1 MiB of additional packages (nvme-cli + uuid-runtime).

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-07-05 12:05:13 +02:00
Fabian Ebner
d2aae44882 cli: tasks: don't treat warning status as an error status
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-07-02 12:52:08 +02:00
Thomas Lamprecht
37208950a7 pve6to7: fix description-length warning for guests
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-02 11:49:48 +02:00
Fabian Grünbichler
cea2918a85 pve6to7: drop PASS for Luminous -> Nautilus check
this should pass on pretty much every system running hyper converged
Ceph on PVE 6.x ..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-07-02 11:43:44 +02:00
Fabian Grünbichler
23db33ca4c pve6to7: reduce number of cluster PASS statements
these were mostly releveant for upgrading from Corosync 2.x to 3.x - so
keep the warnings/errors, but reduce the noise a bit by skipping lots of
PASS output.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-07-02 11:43:44 +02:00
Dominik Csapak
97723d760c pve6to7: skip user.cfg if it does not exist
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-30 16:25:50 +02:00
Fabian Ebner
1126b8cd7d pve6to7: more fine-grained detection of misconfigured guest volumes
If neither 'rootdir' nor 'images' are configured on a storage, but
there are guest images, just log the number of volumes found. If they
are relevant for migration, the check for unreferenced volumes will
catch them later.

Also detect content type mismatch for all volumes of existing guests,
which also covers the case of a VM image on a storage with only
'rootdir' and vice versa. To catch all such unreferenced volumes too,
it is necessary to scan all storages that do not have both content
types configured.

Change the message from 'will not work' to 'might not work'. If a
volume only referenced by a snapshot is misconfigured, it doesn't mean
that the guest doesn't work at all. Or it might be an ISO on a
misconfigured storage.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-06-30 14:21:11 +02:00
Fabian Ebner
f68689bbf8 pve6to7: content check: fix detecting pass
If there is a log_fail, because of misconfigured 'none' content type, the final
log_pass should not be printed.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-06-30 14:21:11 +02:00
Fabian Ebner
4bb46baab5 cli tools: correctly handle warnings status
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-06-24 08:50:38 +02:00
Fabian Ebner
bbc34b4789 cli: pveam: remove unused private sub
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-06-24 08:50:38 +02:00