Commit Graph

783 Commits

Author SHA1 Message Date
Alwin Antreich
ba24f026e0 ceph: allow to alter pool settings
after creation, so that users don't need to go the ceph tooling route.
Separate common pool options to reuse them in other places.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-10-22 18:18:35 +02:00
Alwin Antreich
50adb13155 ceph: split out pool set into own method
to reduce code duplication and make it easier to add more options for
pool commands.

Use a new rados object for each 'osd pool set', as each command can set
an option independent of the previous commands success/failure. On
failure a new rados object would need to be created and that will
confuse task tracking of the REST environment.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-10-22 18:18:35 +02:00
Fabian Ebner
d4d0177fa6 simplify get_included_vmids function
by collecting all the guest IDs first.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-10-22 16:29:26 +02:00
Fabian Ebner
7f87414894 backup: include IDs for non-existent guests
Like this, there will be a backup task (within the big worker task)
for such IDs, which will then visibly (i.e. also visible in the
notification mail) fail with, e.g.:
unable to find VM '123'

In get_included_guests, the key '' was chosen for the orphaned IDs,
because it cannot possibly denote a nodename.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-10-22 16:29:26 +02:00
Fabian Ebner
e6946086e3 Allow prune-backups as an alternative to maxfiles
and make the two options mutally exclusive as long
as they are specified on the same level (e.g. both
from the storage configuration). Otherwise prefer
option > storage config > default (only maxfiles has a default currently).

Defines the backup limit for prune-backups as the sum of all
keep-values.

There is no perfect way to determine whether a
new backup would trigger a removal with prune later:
1. we would need a way to include the not yet existing backup
   in a 'prune --dry-run' check.
2. even if we had that check, if it's executed right before
   a full hour, and the actual backup happens after the full
   hour, the information from the check is not correct.

So in some cases, we allow backup jobs with remove=0, that
will lead to a removal when the next prune is executed.
Still, the job with remove=0 does not execute a prune, so:
1. There is a well-defined limit.
2. A job with remove=0 never removes an old backup.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-09-30 10:56:30 +02:00
Fabian Grünbichler
8f8073cb2e tasks: allow API tokens to query their own UPIDs
and regular users to read all their own tasks as well as those of their
associated tokens.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-09-23 15:12:31 +02:00
Stoiko Ivanov
d49b703723 use PVE::Tools::sendmail for update notifications
instead of calling /usr/sbin/sendmail directly.

Tested with:
```
echo "[]" > /var/lib/pve-manager/pkgupdates
pvesh create /nodes/rosa/apt/update --notify 1

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2020-09-07 09:42:24 +02:00
Thomas Lamprecht
6f5c3b98ac pveversion: track proxmox-backup-client
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-21 13:31:24 +02:00
Thomas Lamprecht
09575ae8f6 replication: followup: include guest node in error message
tell the user what check actually failed..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-20 13:48:53 +02:00
Fabian Grünbichler
b88b01008b replication: check for source == target on job creation
and die.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-20 13:47:21 +02:00
Fabian Grünbichler
10257794f2 replication: target is a required parameter
no need to check for definedness

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-20 13:47:21 +02:00
Fabian Grünbichler
ef3fde16e2 replication: refactor source checks on job creation
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-20 13:47:21 +02:00
Thomas Lamprecht
1e67b44463 api: ACME revoke: do not abort if CA is just expired
Else, a user would need to renew it first before being able to revoke
it, which does not make much sense..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-19 18:52:48 +02:00
Thomas Lamprecht
25beb9038c api: ACME renew: handle error from revoking old cert gracefully
this normally just means that the old cert is already expired, we do
not care for that - after all: we got a new (renewed) valid cert
successfully.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-19 18:49:20 +02:00
Fabian Ebner
9671488c5d Set source when creating a new replication job
If source is missing, pvesr will set it via job_status
on the next run. But the info is already present here,
so it doesn't hurt to use it.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-08-11 13:54:21 +02:00
Thomas Lamprecht
3cd0759af0 api/services: fallback to restart in reload
And document that it is preferred

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-24 15:24:02 +02:00
Thomas Lamprecht
d438bb3e67 api/services: improve essential service check, allow reload and start
reload is actually preferred, and even if most of the time this even
won't reach the API, allowing to start them is still definitively
fine!

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-24 15:24:02 +02:00
Thomas Lamprecht
05d5632db2 api/services: whitespace cleanup, modern closure call
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-24 15:24:02 +02:00
Alwin Antreich
2184098ed3 Allow setting device class on osd create
In some situations Ceph's auto-detection doesn't recognize the device
class correctly. The option allows to set it directly on osd create,
instead of altering it afterwards. This way the cluster doesn't need to
shift data back and forth unnecessarily.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-07-24 10:26:11 +02:00
Aaron Lauterer
f26c7b54e0 fix #2609 api: backupinfo: add non job specific endpoint
Adds a new api endpoint at cluster/backupinfo for cluster wide backup
stuff. This is necessary because cluster/backup expects a backup job ID
at the next level and thus other endpoints are hard to impossible to
implement under that hierarchy.

The only api endpoint available for now is the `not_backed_up` which
returns a list of all guests which are not covered by any backup job.

The top level index endpoint is left unsused for now to be available for
a more generic summary endpoint in the future.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-07-09 14:12:59 +02:00
Aaron Lauterer
ac0fe8b66c api: backup: add endpoint to list included guests and volumes
This patch adds a new API endpoint that returns a list of included
guests, their volumes and whether they are included in a backup.

The output is formatted to be used with the extJS tree panel.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-07-09 14:12:59 +02:00
Thomas Lamprecht
e6d963ca3e backup: update job: add missing permission description
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-08 13:53:57 +02:00
Thomas Lamprecht
2617768f2c backup: factor out param permission check
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-08 13:48:21 +02:00
Oguz Bektas
d5b9f2e1cb backup: add missing user check in update_job
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2020-07-08 13:41:38 +02:00
Thomas Lamprecht
ecddd2e231 Revert "Close #1623: replace apt-get to apt"
This reverts commit 19137ed620.
2020-07-08 10:55:56 +02:00
Moayad Almalat
19137ed620 Close #1623: replace apt-get to apt
Signed-off-by: Moayad Almalat <m.almalat@proxmox.com>
2020-07-07 18:52:28 +02:00
Thomas Lamprecht
d6e7fa0404 api nodes/netstat: cleanup indentation and refactor slightly
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-03 08:06:44 +02:00
Thomas Lamprecht
677bee7c57 api/nodes: factor out getting VNC connection info
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-01 09:02:49 +02:00
Stefan Reiter
01ac98c702 api: register /nodes/X/cpu call for CPU models
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-17 15:51:11 +02:00
Aaron Lauterer
df5875b41e vzdump: move remaining guest include logic to single method
The `guest include` logic handling `all` and `exclude` parameters was in
the `PVE::VZDump->exec_backup()` method. Moving this logic into the
`get_included_guests` method allows us to simplify and generalize it.

This helps to make the overall logic easier to test and develop other
features around vzdump backup jobs.

The method now returns a hash with node names as keys mapped to arrays
of VMIDs on these nodes that are included in the vzdump job.

The VZDump API call to create a new backup is adapted to use the new
method to create the list of local VMIDs and the skiplist.

Permission checks are kept where they are to be able to handle missing
permissions according to the current context. The old behavior to die
on a backup job when the user is missing the permission to a guest and
the job is not an 'all' or 'exclude' job is kept.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-06-17 15:16:06 +02:00
Fabian Grünbichler
4aa89cc0e8 fix #2784: always compare ACME domains in lower case
otherwise the ACME endpoint might return the ordered domain in lower
case and we fail to find our plugin config.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-06-17 14:00:43 +02:00
Fabian Grünbichler
63ad21645d fix #2771: relax cert API endpoints permissions
allow users with Sys.Modify to modify custom or ACME certificates. those
users can already hose the system in plenty of ways, no reason to
restrict this in particular to being root@pam only.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-06-17 14:00:43 +02:00
Alexandre Derumier
9afcbd26b2 api : cluster resources: add sdn audit permission
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2020-06-15 15:35:30 +02:00
Thomas Lamprecht
47399e41ad small nit follouwp: use $var->[0] to acccess array ref element
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-08 15:57:34 +02:00
Aaron Lauterer
5c4da4c3e8 vzdump: make guest include logic testable
As a first step to make the whole guest include logic more testable the
part from the API endpoint has been moved to its own method with as
little changes as possible.

Everything concerning `all` and `exclude` logic is still in the
PVE::VZDump->exec_backup() method.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-06-08 15:54:42 +02:00
Alwin Antreich
d4dba076a6 ceph: extend pveceph pool ls
to present more data on pools and a nicer formated output on the command
line.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-06-03 16:08:27 +02:00
Alwin Antreich
06bd1c5215 ceph: extend the pool view
to add the pg_autoscale_mode since its activated in Ceph Octopus by
default and emmits a waring (ceph status) if a pool has too many PGs.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-06-03 16:08:27 +02:00
Alwin Antreich
e25dda254c Make PVE6 compatible with supported ceph versions
Luminous, Nautilus and Octopus. In Octopus the mon_status was dropped.
Also the ceph status was cleaned up and doesn't provide the mgrmap and
monmap.

The rados queries used in the ceph status API endpoints (cluster / node)
were factored out and merged to one place.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-06-03 14:23:38 +02:00
Thomas Lamprecht
9ad4656d57 sdn: use new helper methods to update config
and do a breaks on older network package as we do not depend on it
due to it being an optional/experimental feature, so reverse the
depends with the breaks.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-03 14:22:21 +02:00
Fabian Grünbichler
1a664b0f9d api: improve node index with missing/broken cert
since this API endpoint is used for the node selector in the GUI, which
causes quite widespread breakage.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-06-02 10:28:56 +02:00
Thomas Lamprecht
bc12fba5fc ui: fix system report window indentation + cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-27 14:11:16 +02:00
Thomas Lamprecht
e82ed1675b subscription delete: followup fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-27 13:39:13 +02:00
Moayad Almalat
85222f8237 api, ui: allow to remove subscription
Signed-off-by: Moayad Almalat <m.almalat@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-27 13:28:27 +02:00
Thomas Lamprecht
4047ea2461 api: apt/versions: track libpve-network-perl as optional package
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-10 17:14:04 +02:00
Thomas Lamprecht
e01302b764 api acme: reduce after validation-request delay, print status on err
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-07 21:47:53 +02:00
Thomas Lamprecht
a72428b8c6 ACME: add comment for implicit standalone plugin fallback
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-07 13:53:05 +02:00
Fabian Grünbichler
25fd63631d ACME: fix fallback to implicit standalone plugin
we need to parse the config even if it does not exist - it will return
the 'standalone' entry that's needed to be backwards compatible with
existing setups.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-05-07 12:59:37 +02:00
Dominik Csapak
75a2be66bb ACME: add challengeschema api call
which returns a list of challenge api types with the schema of their
required data (if it exists)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
[ Thomas: adapt to my changes from proxmox-acme schema def and change
  path from challengeschema to challenge-schema ]
2020-05-05 19:23:04 +02:00
Dominik Csapak
463ea0b1d8 ACMEPlugin: check digest on update
and extract the param, otherwise the check dies because of an unknown
field 'digest'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-05 18:34:46 +02:00
Thomas Lamprecht
c794bb5dc0 api acme: reduce verification delays
As proxmox-acme has now a default delay for DNS challenge plugins,
which is the important one. Those are just for not overloading the
acme servers with a lot of requests, but once the challenge was
propagate they have it verified pretty quickly, so reduce delay for
checking validation after first requesting it down to 10 seconds

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-03 18:46:12 +02:00