Commit Graph

825 Commits

Author SHA1 Message Date
Thomas Lamprecht
a91bd3c370 api: ceph pool create: replace left-over complex error handling
this was from the time where we had a loop here to add two storages,
one for KRDB-only and one for KRBD-never. Nowadays we can handle the
mixed case just fine, but the patch dropping that forget to cleanup
the error handling..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-21 17:34:23 +02:00
Thomas Lamprecht
84b08e8aec api: ceph/pool: fix formatting of API parameters
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-21 15:45:35 +02:00
Dominik Csapak
08db34257a API2/Ceph/Pools: remove unnecessary boolean conversion
we do nothing with that field, so leave it like it is

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-20 18:20:35 +02:00
Alwin Antreich
6b36f36842 ceph: set allowed minimal pg_num down to 1
In Ceph Octopus the device_health_metrics pool is auto-created with 1
PG. Since Ceph has the ability to split/merge PGs, hitting the wrong PG
count is now less of an issue anyhow.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-20 18:20:35 +02:00
Alwin Antreich
5a3d794242 ceph: add autoscale_status to api calls
the properties target_size_ratio, target_size_bytes and pg_num_min are
used to fine-tune the pg_autoscaler and are set on a pool. The updated
pool list shows now autoscale settings & status. Including the new
(optimal) target PGs. To make it easier for new users to get/set the
correct amount of PGs.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-20 18:20:35 +02:00
Thomas Lamprecht
d7a63207a3 ceph: osd_belongs_to_node: only check tree-entries of type host, refactor
We want to check explicitly for type host, so filter for that first
and create a hash map for easier usage afterwards.

Drop the error when there's no tree, as either RADOS error'd on bad
command already, or there really is no tree (but RADOS worked OK), in
which case we simply return that the OSD did not belong to this node.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-20 18:06:07 +02:00
Dominic Jäger
220173e9c6 Fix #2053: OSD destroy only on specified node
Allow destroying only OSDs that belong to the node that has been specified in
the API path.

So if
 - OSD 1 belongs to node A and
 - OSD 2 belongs to node B
then
 - pvesh delete nodes/A/ceph/osd/1 is allowed but
 - pvesh delete nodes/A/ceph/osd/2 is not

Destroying an OSD via GUI automatically inserts the correct node
into the API path.

pveceph automatically insert the local node into the API call, too.
Consequently, it can now only destroy local OSDs (fix #2053).
 - pveceph osd destroy 1 is allowed on node A but
 - pveceph osd destroy 2 is not

Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
2021-04-20 16:42:12 +02:00
Thomas Lamprecht
424c94fee8 api: resources: only get guest lock after real access check
The guest iteration is slightly confusing as we also handle the
accumulated pool settings there, so we only check the VM.Audit privs.
for a specific VM and skip to the next if the permissions is not
there after those pool handling.

So, move operations which are only required when VM privs. are there
below this check.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-15 19:45:01 +02:00
Thomas Lamprecht
ab8ed6f4b0 api: resources: push guest, pool and sdn info sorted
makes it easier to compare in API responses, and those list are not
huge, seldom over a few thousands, which is peanut crumbs compared to
all the other thing in this perl stack.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-15 19:35:31 +02:00
Fabian Ebner
875c2e5aae vzdump: getlock: return lock file handle and let the caller close it
so it doesn't get out of scope too early.

Regression introduced by 5620e5761e as pointed
out by Fabian Grünbichler.

Reported in the community forum:
https://forum.proxmox.com/threads/limit-simultaneous-backup-jobs.87489

Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-04-12 14:36:08 +02:00
Thomas Lamprecht
c6c4b2788d api: node: add capabilities path and move qemu machines over
avoid further crowding the top-level node API path with such
"what can some part of the node currently do" stuff, rather move it
down.

The QEMU cpu stuff should move also down there.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-05 21:31:32 +01:00
Thomas Lamprecht
5a378425b8 api: nodes: sort use statements
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-05 21:31:32 +01:00
Stefan Reiter
32ad9758a2 api: register Qemu::Machine call
as 'machine-types', so it is clear this refers to QEMU machines, not the
local machine (as one might think, this being a 'node' API call).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-05 21:31:32 +01:00
Thomas Lamprecht
cc5e0685be buildsys: api: sort sources
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-05 21:31:32 +01:00
Thomas Lamprecht
784d6ee9b4 api: nodes: sort index
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-05 21:31:32 +01:00
Fabian Ebner
f8ed6af80d vzdump: refactor parsing mailto so it can be mocked
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Reviewed-By: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-19 16:33:33 +01:00
Alwin Antreich
54ba7dd991 ceph: add get api call for single pool
Information of a single pool can be queried.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2021-02-06 14:28:39 +01:00
Alwin Antreich
461e214155 ceph: add titles to ceph_pool_common_options
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2021-02-06 14:25:53 +01:00
Alwin Antreich
51d6db5815 ceph: setpool, use parameter extraction instead
of the unneeded ref copy for params.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2021-02-06 14:24:30 +01:00
Alwin Antreich
56d02a863b api: ceph: subclass pools
for better handling and since the pool endpoints got more entries.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2021-02-06 14:17:53 +01:00
Fabian Ebner
d55b2be9a3 api: Ceph: add reminder to remove 'disks' call
This API call is the predecessor of /nodes/{node}/disks/list, which has seen a
few more improvements. The latter API call should be used instead, and the web
UI already does so.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-06 14:04:40 +01:00
Dominik Csapak
27bc5e8e02 status/plugin: extend with add/update/delete hooks
like we do in it for the storage section configs

we will need this to store the token for influxdbs http api

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 17:32:23 +01:00
Dominik Csapak
161d904f13 api: cluster/metricserver: prevent simultaneosly setting and deleting of property
like we do in other apis of section configs (e.g. storage)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 17:32:23 +01:00
Thomas Lamprecht
f5b6ccb1d5 api: fixup: commit change to storage module
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-02 11:45:33 +01:00
Thomas Lamprecht
1c182fc59f api: add usb list in hardware endpoint
As envisioned in[0][1], better late than never.

[0]: commit 523d5f486b
[1]: https://lists.proxmox.com/pipermail/pve-devel/2018-November/034694.html

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-02 11:27:02 +01:00
Thomas Lamprecht
d6441a5d87 api: use storage scan module from pve-storage
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:30:13 +01:00
Dominik Csapak
dadba141a8 api: metrics/server: test connection on add/update
just a basic check, but better than not checking at all

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-25 14:55:25 +01:00
Stoiko Ivanov
c92fc8a1e8 api2: osd destroy: untaint device before pvremove
We get the device list from ceph-volume lvm list, and decode the json
output, which at that point is tainted (perlsec (1)).
Untaint it here before calling, because it is currently the only
call-site using the information in a problematic way (run_command).
(the only other call-site being in pve5to6)

Alternatively we could untaint while reading the information, but then
should only return a small subset of the ceph-volume output.

The issue is most likely due to
cb9db10c1a9855cf40ff13e81f9dd97d6a9b2698 in pve-common ('run_command:
improve performance for logging and long lines'),

Tested on a virtual testsetup by creating OSDs with second DB disk,
and destroying it via GUI (did not manage to get the error without the
DB disk)

Reported via our community forum:
https://forum.proxmox.com/threads/insecure-dependency-in-exec-during-osd-destroy.79574/

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2020-11-24 23:37:33 +01:00
Stoiko Ivanov
259b557cf4 api2: osd destroy: fix error function
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2020-11-24 23:37:33 +01:00
Thomas Lamprecht
77a9ce3248 api: resources: return plugin type
The storage one for now.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 23:01:54 +01:00
Dominik Csapak
063b6d5e0e API2/Ceph: fix createpools 'add_storages' parameter
by deleteing it from $ceph_param we deleted it also from
$param since it was only a reference

fix it by extracting it beforehand

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-24 15:45:24 +01:00
Fabian Ebner
74c8984dfc cluster resources: include content type for storages
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-22 18:39:16 +01:00
Thomas Lamprecht
f24be7e72d api: nodes: allow root to pass arbitrary arguments to commands
If the command itself allows it, which normally means it has good
verification of passed arguments.

We may want to re-evaluate security here if we allow execution for a
group of non-root users.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-22 17:59:36 +01:00
Thomas Lamprecht
22ad422ac9 api: cluster/ceph: allow version metadata call on non-ceph nodes
the information comes only from the key value store in the pmxcfs, so
we do not actually require ceph to be installed.

So only check if ceph is locally initialized and create the rados
connection after the early return when only versions scope is set.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-22 17:59:36 +01:00
Thomas Lamprecht
d57f8d9482 ui/api: metrics: move server to subpath, post to path with ID
We actually wanted to use that scheme for more new API paths, lets
see if it is really fitting starting with this.

Use the new widget-toolkit submitUrl helper to add the ID on create.

And unify the edit/create window creation, which may fit better in a
separate commit, it's quite small and was to cumbersome to untangle,
so just go against my one rules here... :(

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-21 20:53:05 +01:00
Thomas Lamprecht
9dedf1e2d5 api: drop Config from MetricServer module name
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-21 20:39:33 +01:00
Dominik Csapak
19e3a7f2a4 api2/cluster: add 'metricserver' api endpoints
modeled after our typical api endpoints for sectionschema configs

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-21 20:39:33 +01:00
Dominik Csapak
33b4554849 fix iteration over hash without keys
without 'keys' we iterate also over the values and use them as keys

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-20 15:34:27 +01:00
Stefan Reiter
7828eef6f2 apt: add libproxmox-backup-qemu0 to displayed packages
Can provide important information for debugging issues in forum etc...

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-11-05 10:30:08 +01:00
Thomas Lamprecht
6d6cf3f802 followup: simplify dropping defaults from common ceph options
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-22 18:18:35 +02:00
Alwin Antreich
c423489a41 ceph: add pg_autoscale_mode to pool create
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-10-22 18:18:35 +02:00
Alwin Antreich
7e03ec643b ceph: use pool common options pool create
to keep the pool create & set in sync.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-10-22 18:18:35 +02:00
Alwin Antreich
ba24f026e0 ceph: allow to alter pool settings
after creation, so that users don't need to go the ceph tooling route.
Separate common pool options to reuse them in other places.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-10-22 18:18:35 +02:00
Alwin Antreich
50adb13155 ceph: split out pool set into own method
to reduce code duplication and make it easier to add more options for
pool commands.

Use a new rados object for each 'osd pool set', as each command can set
an option independent of the previous commands success/failure. On
failure a new rados object would need to be created and that will
confuse task tracking of the REST environment.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-10-22 18:18:35 +02:00
Fabian Ebner
d4d0177fa6 simplify get_included_vmids function
by collecting all the guest IDs first.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-10-22 16:29:26 +02:00
Fabian Ebner
7f87414894 backup: include IDs for non-existent guests
Like this, there will be a backup task (within the big worker task)
for such IDs, which will then visibly (i.e. also visible in the
notification mail) fail with, e.g.:
unable to find VM '123'

In get_included_guests, the key '' was chosen for the orphaned IDs,
because it cannot possibly denote a nodename.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-10-22 16:29:26 +02:00
Fabian Ebner
e6946086e3 Allow prune-backups as an alternative to maxfiles
and make the two options mutally exclusive as long
as they are specified on the same level (e.g. both
from the storage configuration). Otherwise prefer
option > storage config > default (only maxfiles has a default currently).

Defines the backup limit for prune-backups as the sum of all
keep-values.

There is no perfect way to determine whether a
new backup would trigger a removal with prune later:
1. we would need a way to include the not yet existing backup
   in a 'prune --dry-run' check.
2. even if we had that check, if it's executed right before
   a full hour, and the actual backup happens after the full
   hour, the information from the check is not correct.

So in some cases, we allow backup jobs with remove=0, that
will lead to a removal when the next prune is executed.
Still, the job with remove=0 does not execute a prune, so:
1. There is a well-defined limit.
2. A job with remove=0 never removes an old backup.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-09-30 10:56:30 +02:00
Fabian Grünbichler
8f8073cb2e tasks: allow API tokens to query their own UPIDs
and regular users to read all their own tasks as well as those of their
associated tokens.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-09-23 15:12:31 +02:00
Stoiko Ivanov
d49b703723 use PVE::Tools::sendmail for update notifications
instead of calling /usr/sbin/sendmail directly.

Tested with:
```
echo "[]" > /var/lib/pve-manager/pkgupdates
pvesh create /nodes/rosa/apt/update --notify 1

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2020-09-07 09:42:24 +02:00
Thomas Lamprecht
6f5c3b98ac pveversion: track proxmox-backup-client
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-21 13:31:24 +02:00