Commit Graph

1613 Commits

Author SHA1 Message Date
Fabian Grünbichler
58f763e1aa fix #3430: handle hook script paths better
if passing the hook script command as string, it might get interpreted
as shell command with side-effects. this is pretty harmless, since only
root is allowed to set the script parameter anyway, but making it more
robust and future-proof does not hurt.

tested with a reproducer of "/bin/echo $(touch $(whoami))" as script
parameter, with a file with that name existing, being executable and
having the following contents:

----8<----
echo "hello from hook script"
---->8----

without this change, the hookscript itself is not executed, but
'/bin/sh -c "/bin/echo $(touch $(whoami)) job"' and similar calls are,
which cause the file 'root' to be touched in the current working
directory of the vzdump process (or task worker).

with this change, the file is executed as is without any side-effects of
shell commands in the file name, and the 'hello from hook script' lines
are printed whenever the hook script is called by vzdump.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 09:49:34 +02:00
Thomas Lamprecht
cdb638259d report: top: also print command line arguments
at least the first 512 characters, which is the biggest width one can
instruct `top` too.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-11 16:27:30 +02:00
Thomas Lamprecht
bc415cf7cf report: top: list more processes and order them by CPU time spent
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-11 16:27:30 +02:00
Thomas Lamprecht
8b94e6bbd4 report: re-organize: add systemd-load, unify bios/pci in HW one
Also rename disks to blockdevices, as especially with the iSCSI stuff
the listed devices do not necessarily need to be real disks.

Grouping current(ish) load info in it's own system-load section seems
to be a better fit in general too.

bios/pci stuff is all hardware, so group it there to avoid adding to
much sections.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-11 16:27:30 +02:00
Thomas Lamprecht
6b72e9dd74 report: dir2text: print what we process on STDERR
to avoid those "lonely" "OK" prints for those

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-11 15:12:18 +02:00
Thomas Lamprecht
f784b92573 report: avoid extra oder array, define it inline
Makes all report-def entries hashes which allows to drop some
handling for the ARRAY ref case.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-11 15:02:38 +02:00
Thomas Lamprecht
3a0b2c40d2 report: avoid adding to report by side-effects, be more explicit
And move the helper methods up and scope them to module-local only

Uses the fact that perl methods return the last statement, so the
dir2text sub closures in the command list do not need to be changed

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-11 14:28:47 +02:00
Thomas Lamprecht
73cdcc98be report: set timeout and PATH env only locally
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-11 14:27:40 +02:00
Thomas Lamprecht
fb925d656a report: whitespace cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-11 14:26:15 +02:00
Oguz Bektas
4befa0abe0 pvereport: get pressure stall information
PSI can be queried in /proc/pressure/{cpu,io,memory} for the
corresponding resources. this helps us track down disruptions caused
by resource overcommitment.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2021-05-11 14:07:25 +02:00
Lorenz Stechauner
230c4416f2 api: pools: fix typo/syntax error
Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
2021-05-10 17:12:17 +02:00
Thomas Lamprecht
274a0e2394 api: pools: whitespace, indentation and code cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-09 18:58:42 +02:00
Thomas Lamprecht
926ce2b8d9 api: pools: whitespace, indentation and code cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-09 18:52:28 +02:00
Stoiko Ivanov
36ad2b3c0e proxy: fix wildcard address use
This patch fixes a regression for hosts disabling ipv6 via kernel
commandline ('ipv6.disable=1')introduced in commit
e224b7d2e6
(disabling IPv6 via sysctl did not exhibit these problems)

by hardcoding the address to '::', pveproxy and spiceproxy failed to
start with:
'unable to create socket - Address family not supported by protocol'

This patch depends on the commit in pve-common, which tries first
binding to '::' and then falling back to '0.0.0.0', and needs a
versioned dependency bump on libpve-common-perl.

With this patch the listening addresses are (`ss -tlnp |grep 8006` output)
* ipv6 disabled via kernel cmdline: '0.0.0.0:8006'
* sysctl net.ipv6.conf.all.disable_ipv6=1: '*:8006'
* sysctl net.ipv6.bindv6only=1: '[::]:8006'
* else: '*:8006'

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2021-05-07 18:15:43 +02:00
Thomas Lamprecht
96d4c3b4a6 api: acme: fix API index entry for challenge-schema
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-05 15:52:49 +02:00
Thomas Lamprecht
6110ed0348 api: nodes: whitespace cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-04 11:58:55 +02:00
Thomas Lamprecht
37bf860e3e pveceph: fixup syntax issue
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-03 14:38:16 +02:00
Fabian Ebner
5b9a4030dd api: vzdump: add call to get currently configured vzdump defaults
on a given node (and storage).

There is no datacenter/storage fallback for the bandwidth limit, so the default
can just be returned as is. While the bandwidth limit is a root-only option when
executing the backup, it still makes sense to return it for all users, so they
can see what's going to be used.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-05-03 14:36:59 +02:00
Thomas Lamprecht
ece49b3c90 ceph: reload API daemons after installation
To make them load the updated librados2, as else they may potentially
not be able to communicate with the potentially newer ceph monitors,
as Debian 10 ships Jewel (12.2) by default...

While we could do some more fancy signaling to the workers to reload
the lib, that is rather a PITA and complex solution for something
that happens once in a blue moon.

We may want to add a trigger in ceph for this on updates though, that
would effectively fix this too - but needs to be thought out better.
So for now lets go with the simplest solution.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-03 14:32:50 +02:00
Thomas Lamprecht
51498a2664 ceph: code/indentation cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-03 14:03:32 +02:00
Thomas Lamprecht
0dd48804e1 api: ceph/monitor: automatically disable insecure global ID reclaim after creating first monitor
nautilus 14.2.20 and octopus 15.2.11 fixed a security issue with
reclaiming the global ID auth (CVE-2021-20288). As fixing this issue
means that older client won't be able to connect anymore, the fix was
done behind a switch, with a HEALTH warning if it was not active
(i.e., disallowed connection from older clients).

New installations have this switch also at the insecure level, for
compat reasons, so lets deactivate it ourself after monitor creation
to avoid the health warning and slightly insecure setup (in default
PVE ceph the whole issue was of rather low impact/risk). But, only do
so when creating the first monitor of a ceph cluster, to avoid
breaking existing setups by accident.

An admin can always switch it back again, e.g., if they're recovering
from some failure and need to setup fresh monitors but have still old
clients.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-27 12:35:34 +02:00
Thomas Lamprecht
d26556c07b pveceph: octopus is OK and wanted to be installed/used
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-27 12:27:32 +02:00
Oguz Bektas
e224b7d2e6 fix #417: proxy: allow settnig LISTEN_IP for pveproxy and spiceproxy
The $host variable is set to "::0" by default to listen on wildcard
(with 'Domain' => PF_INET6).

If 'LISTEN_IP' is defined in /etc/default/pveproxy, that IP will be used
instead.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2021-04-23 17:13:45 +02:00
Stoiko Ivanov
6cd854d8c7 fix #3385: api: network: check for used ports before writing
Currently the check for used ports for bonds and bridges happens
while rendering '/etc/network/interfaces.new' in PVE::Inotify
(pve-common).
However at that stage the new/updated interface is already merged
with the old settings, making it impossible to indicate where a NIC
is currently used.

The code is adapted from the renderer in
PVE::Inotify::__write_etc_network_interfaces.

Tested on a virtual PVE instance.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-22 21:32:26 +02:00
Thomas Lamprecht
736b919d04 http server: comment and refactor CSRF skip-check logic
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-22 11:19:40 +02:00
Lorenz Stechauner
7d11059e2d http-server: fix for api token csrf token check
Do not check any csrf tokens for auth with api tokens.

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
2021-04-22 10:54:26 +02:00
Thomas Lamprecht
e45f98ef8b http-server: minor code/indentation/comment cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-22 09:10:24 +02:00
Lorenz Stechauner
d9fc56efab fix #3389: Skip CSRF token check also for API tokens on file upload
On file upload, the check for CSRF tokens was already skipped when
performing user authentication. This now happens for API tokens also.

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-22 09:01:11 +02:00
Thomas Lamprecht
a91bd3c370 api: ceph pool create: replace left-over complex error handling
this was from the time where we had a loop here to add two storages,
one for KRDB-only and one for KRBD-never. Nowadays we can handle the
mixed case just fine, but the patch dropping that forget to cleanup
the error handling..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-21 17:34:23 +02:00
Thomas Lamprecht
68f94af88a ceph: pool set: fix printing problematic settings
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-21 16:06:29 +02:00
Thomas Lamprecht
cb83113ddc ceph: pool set: print settings as they are applied
a bit of a weird place to print code-wise, but it shows up in the
task log just fine.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-21 16:05:42 +02:00
Thomas Lamprecht
69c0ff3e60 ceph: pool set: improve handling "size" parameter
instead of fully duplicating the code just order it first if it
exists...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-21 15:49:43 +02:00
Thomas Lamprecht
84b08e8aec api: ceph/pool: fix formatting of API parameters
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-21 15:45:35 +02:00
Dominik Csapak
08db34257a API2/Ceph/Pools: remove unnecessary boolean conversion
we do nothing with that field, so leave it like it is

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-20 18:20:35 +02:00
Alwin Antreich
11d7427400 fix: ceph: always set pool size first
Since Ceph Nautilus 14.2.10 and Octopus 15.2.2 the min_size of a pool is
calculated by the size (round(size / 2)). When size is applied after
min_size to the pool, the manual specified min_size will be overwritten.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-20 18:20:35 +02:00
Alwin Antreich
6b36f36842 ceph: set allowed minimal pg_num down to 1
In Ceph Octopus the device_health_metrics pool is auto-created with 1
PG. Since Ceph has the ability to split/merge PGs, hitting the wrong PG
count is now less of an issue anyhow.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-20 18:20:35 +02:00
Alwin Antreich
5a3d794242 ceph: add autoscale_status to api calls
the properties target_size_ratio, target_size_bytes and pg_num_min are
used to fine-tune the pg_autoscaler and are set on a pool. The updated
pool list shows now autoscale settings & status. Including the new
(optimal) target PGs. To make it easier for new users to get/set the
correct amount of PGs.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-20 18:20:35 +02:00
Thomas Lamprecht
d7a63207a3 ceph: osd_belongs_to_node: only check tree-entries of type host, refactor
We want to check explicitly for type host, so filter for that first
and create a hash map for easier usage afterwards.

Drop the error when there's no tree, as either RADOS error'd on bad
command already, or there really is no tree (but RADOS worked OK), in
which case we simply return that the OSD did not belong to this node.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-20 18:06:07 +02:00
Dominic Jäger
220173e9c6 Fix #2053: OSD destroy only on specified node
Allow destroying only OSDs that belong to the node that has been specified in
the API path.

So if
 - OSD 1 belongs to node A and
 - OSD 2 belongs to node B
then
 - pvesh delete nodes/A/ceph/osd/1 is allowed but
 - pvesh delete nodes/A/ceph/osd/2 is not

Destroying an OSD via GUI automatically inserts the correct node
into the API path.

pveceph automatically insert the local node into the API call, too.
Consequently, it can now only destroy local OSDs (fix #2053).
 - pveceph osd destroy 1 is allowed on node A but
 - pveceph osd destroy 2 is not

Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
2021-04-20 16:42:12 +02:00
Mira Limbeck
1226cd97f9 report: add multipath.conf and wwids file
These 2 files can be helpful for issues with multipath. The multipath -v3
output is too large most of the time and not required for analyzing and
solving the issues.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2021-04-16 15:20:40 +02:00
Thomas Lamprecht
424c94fee8 api: resources: only get guest lock after real access check
The guest iteration is slightly confusing as we also handle the
accumulated pool settings there, so we only check the VM.Audit privs.
for a specific VM and skip to the next if the permissions is not
there after those pool handling.

So, move operations which are only required when VM privs. are there
below this check.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-15 19:45:01 +02:00
Thomas Lamprecht
ab8ed6f4b0 api: resources: push guest, pool and sdn info sorted
makes it easier to compare in API responses, and those list are not
huge, seldom over a few thousands, which is peanut crumbs compared to
all the other thing in this perl stack.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-15 19:35:31 +02:00
Fabian Ebner
875c2e5aae vzdump: getlock: return lock file handle and let the caller close it
so it doesn't get out of scope too early.

Regression introduced by 5620e5761e as pointed
out by Fabian Grünbichler.

Reported in the community forum:
https://forum.proxmox.com/threads/limit-simultaneous-backup-jobs.87489

Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-04-12 14:36:08 +02:00
Fabian Ebner
164651dab6 vzdump: storage info: move out activate storage call
Otherwise storage_info() cannot be used for (at least) PBS storages from an API
call without 'protected => 1', because the password cannot be read from
'/etc/pve/priv'. Note that the function itself does not need the storage to be
active, because it only uses storage_config() and get_backup_dir().

AFAICT new() is the only existing user of this function and can be responsible
for activating the storage itself.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-03-31 15:40:13 +02:00
Dominik Csapak
bc33c73963 metrix: influx: fix default api_prefix
we set the api prefix by default to '/' so we always triggered
the the replacement and added '///' which is wrong and does not
work for the 'health' api path
(influxdb returns 404 for 'https://ip:port///health')

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-15 15:28:11 +01:00
Dominik Csapak
c7777408ea metrics: influx: special case 'health' api path i _get_v2url
the forwards compatible api of 1.8 only contains this path
(not api/v2/health) and it it also contained in the v2 api

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-15 15:26:46 +01:00
Thomas Lamprecht
23c9eaf63d metrics: influx: allow one to add an API URL-path prefix
I normally use a reverse proxy in front of my influxdb instances,
proxying all from the /influx/ path to the only locally listening
influxdb. So here I'd need to set "influx" as api-path-prefix.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-15 14:45:05 +01:00
Thomas Lamprecht
6e5405fb21 metrics: influx: do not error out when credendtials could not be loaded
Not a hard error, some network box (proxy) down the line could add it
for us, or it could be just not required, so ...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-15 14:45:05 +01:00
Thomas Lamprecht
bb35a833d1 metrics: influx: send along auth token on connection test too
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-15 14:45:05 +01:00
Thomas Lamprecht
9f8d8f2b05 metrics: influx: include unrecognized protocol value in error
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-15 14:45:05 +01:00