Commit Graph

1062 Commits

Author SHA1 Message Date
Thomas Lamprecht
f7b7e942a7 api: ceph osd: drop unused variable and useless intermediate code
$raw isn't used anywhere here and probably just a left over from copy
pasting, and the "int cast ternary" can be avoided by just directly
casting to int when assigning the variable in the first place.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-09-04 14:22:31 +02:00
Stefan Hanreich
808eb12f8c api: ceph: improve reporting of ceph OSD memory usage
Currently we are using the MemoryCurrent property of the OSD service
to determine the used memory of a Ceph OSD. This includes, among other
things, the memory used by buffers [1]. Since BlueFS uses buffered
I/O, this can lead to extremely high values shown in the UI.

Instead we are now reading the PSS value from the proc filesystem,
which should more accurately reflect the amount of memory currently
used by the Ceph OSD.

Aaron and I decided on PSS over RSS, since this should give a better
idea of used memory - particularly when using a large amount of OSDs
on one host, since the OSDs share some of the pages.

[1] https://www.kernel.org/doc/Documentation/cgroup-v1/memory.txt

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
Tested-by: Aaron Lauterer <a.lauterer@proxmox.com>
2023-09-04 13:53:35 +02:00
Christian Ebner
347f88fecd website: update external links to www.proxmox.com
During the redesign of www.proxmox.com the menu structure and therefore
some url changed. Update the external link in order to avoid an
unneccessary redirect

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2023-08-23 10:09:25 +02:00
Lukas Wagner
4c40d7cbed api: notification: make the 'mail-to-root' target visible to any user
Since the target does not require Mapping.Use, it should also be
visible and testable by all users.

Short explanation why the 'mail-to-root' is exempt from priv checks:

To ensure backwards compatibility, the 'mail-to-root' target does not
require the `Mapping.Use` privs. This is needed due to the fact that
this target is used as a fallback in case no other target is
configured for an event. For instance, the /node/<name>/apt/update API
call only requires Sys.Modify for the node, but it can also send a
notification. If we were to require Mapping.Use, we could break the
apt/update API compat in the case that a notification shall be sent,
but without any configured notification target (which will then
default to 'mail-to-root').

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-08-16 11:11:24 +02:00
Lukas Wagner
1ba1988dcf api: notification: disallow removing targets if they are used
Check notification targets configured in datacenter.cfg and jobs.cfg,
failing if the group/endpoint to be removed is still in use there.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-08-16 11:11:08 +02:00
Lukas Wagner
7e6efd3905 api: notification: allow to test targets
This API call allows the user to test a notification target.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-08-16 11:11:06 +02:00
Lukas Wagner
e678a5dbfa api: notification: allow fetching notification targets
The API call returns all entities that can be used as notification
targets (endpoints, groups). Only targets for which the user has
appropriate permissions are returned.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-08-16 11:10:53 +02:00
Lukas Wagner
56977d48a9 api: notification: add api routes for filters
The Perl part of the API methods primarily defines the API schema,
checks for any needed priviledges and then calls the actual Rust
implementation exposed via perlmod. Any errors returned by the Rust
code are translated into PVE::Exception, so that the API call fails
with the correct HTTP error code.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-08-16 11:10:51 +02:00
Lukas Wagner
aed4eff9cf api: notification: add api routes for gotify endpoints
The Perl part of the API methods primarily defines the API schema,
checks for any needed priviledges and then calls the actual Rust
implementation exposed via perlmod. Any errors returned by the Rust
code are translated into PVE::Exception, so that the API call fails
with the correct HTTP error code.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-08-16 11:10:50 +02:00
Lukas Wagner
c2c3125126 api: notification: add api routes for sendmail endpoints
The Perl part of the API methods primarily defines the API schema,
checks for any needed priviledges and then calls the actual Rust
implementation exposed via perlmod. Any errors returned by the Rust
code are translated into PVE::Exception, so that the API call fails
with the correct HTTP error code.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-08-16 11:10:49 +02:00
Lukas Wagner
95c2dc1bc9 api: notification: add api routes for groups
The Perl part of the API methods primarily defines the API schema,
checks for any needed priviledges and then calls the actual Rust
implementation exposed via perlmod. Any errors returned by the Rust
code are translated into PVE::Exception, so that the API call fails
with the correct HTTP error code.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-08-16 11:10:47 +02:00
Lukas Wagner
b6fa29f3f5 api: prepare api handler module for notification config
This commit adds a new Perl module, PVE::API2::Cluster::Notification.
The module will contain all API handlers for the new notification
subsystem.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-08-16 11:10:15 +02:00
Lukas Wagner
2ab19e8351 api: replication: send notifications via new notification module
If the new 'target-replication' option in datacenter.cfg is set to a
notification target, we send notifications that way. If it is not set,
we continue send a notification to the default target (mail to
root@pam).

There is also a new 'replication' option. It controls whether to send
a notification at all.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-08-16 11:10:14 +02:00
Lukas Wagner
05855908c4 api: apt: send notification via new notification module
... instead of using sendmail directly

If the new 'target-package-updates' is set, we send a notification to
this target. If not, we continue to send a mail to root@pam (if the
mail address is configured)

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-08-16 11:10:12 +02:00
Lukas Wagner
c4afde55f2 vzdump: send notifications via new notification module
... instead of using sendmail directly.

If the new 'notification-target' parameter is set,
we send the notification to this endpoint or group.
If 'mailto' is set, we add a temporary endpoint and a
temporary group containg both targets.

This commit also refactors the old 'sendmail' sub heavily:
  - Use template-based notification text instead of endless
    string concatenations
  - Removing the old plaintext/HTML table rendering in favor of
    the new template/property-based approach offered by the
    `proxmox-notify` crate.
  - Rename `sendmail` sub to `send_notification`
  - Breaking out some of the code into helper subs, hopefully
    reducing the spaghetti factor a bit

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-08-16 11:10:10 +02:00
Philipp Hufnagl
3a9764ad80 fix #474: api: allow transfer from container/vms
When the newly introduced optional parameter "transfer" is set, the user
add a vm/container to a pool even if it is already in one. If so it will
be removed from the old pool

Signed-off-by: Philipp Hufnagl <p.hufnagl@proxmox.com>
2023-08-14 12:26:28 +02:00
Philipp Hufnagl
58ab77d189 fix whitespaces
Signed-off-by: Philipp Hufnagl <p.hufnagl@proxmox.com>
2023-08-04 13:54:34 +02:00
Wolfgang Bumiller
d4b490009c api: use standard vmid type for /cluster/resources
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-08-02 14:40:18 +02:00
Fabian Grünbichler
3fc687f57b handle pve-kernel -> proxmox-kernel rename
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-08-01 10:33:12 +02:00
Alexander Zeidler
c62f096e2b api: backup: refactor backup permission check
Alter style to make the parameter check more concise

Signed-off-by: Alexander Zeidler <a.zeidler@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2023-07-27 10:40:17 +02:00
Dominik Csapak
d958b1306f api: resource usb mapping: add missing proxyto_callback
i have added it to the pci api call, but forgot to add it for usb
otherwise adding a mapped usb device only works on the node where the
gui is connected to

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2023-06-21 12:51:47 +02:00
Thomas Lamprecht
f507ec30bb api ceph: fix directory endpoint index
actually drop the deprecated ones from the API routes index and
ensure the replacement /pool is returned (/cfg already was)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-21 09:32:49 +02:00
Aaron Lauterer
cf2c8b2f52 api: ceph: remove deprecrated config and configdb endpoints
Both are superseeded by ceph/cfg/raw and ceph/cfg/db

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2023-06-21 09:21:04 +02:00
Aaron Lauterer
27f6d19848 api: ceph: remove deprecrated Pools path
The replacement is Pool (singular).

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2023-06-21 09:21:04 +02:00
Thomas Lamprecht
dd6433ff1c api: mappings: cleanup perl imports
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-19 08:30:53 +02:00
Thomas Lamprecht
b74e71f2c2 api: cluster jobs: fix perl module not ending with a true value
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-19 08:30:53 +02:00
Thomas Lamprecht
ebed76a242 api: PCI mappings: code/style cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-19 08:30:53 +02:00
Thomas Lamprecht
3a8bf3b613 api: PCI mappings: rename errors to checks in response
as it contains warnings too, so having it named errors might be
confusing.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-19 08:30:53 +02:00
Thomas Lamprecht
d4830b941f api: PCI mappings: complete return schema
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-19 08:30:53 +02:00
Dominik Csapak
797bcf9aa2 api: add resource map api endpoints for PCI and USB
this adds the typical section config crud API calls for
USB and PCI resource mapping to /cluster/mapping/{TYPE}

the only special thing that this series does is the list call
for both has a special 'check-node' parameter that uses the
'proxyto_callback' to reroute the api call to the given node
so that it can check the validity of the mapping for that node

in the future when we e.g. broadcast the lspci output via pmxcfs
we drop the proxyto_callback and directly use the info from
pmxcfs (or we drop the parameter and always check all nodes)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2023-06-16 16:25:42 +02:00
Fabian Grünbichler
9ed5d4f5af cluster resources: correctly filter 'localnetwork'
it should only be displayed if sdn entries are requested, or all resource
types.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-06-12 13:18:12 +02:00
Alexandre Derumier
cdc140f0a3 api2: cluster: ressources: add "localnetwork" zone
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2023-06-12 13:00:16 +02:00
Thomas Lamprecht
eb85935498 api: mark batch-execute api calls root-only
This is weird and buggy and breaches the unpriv./priv. separation of
our api daemons, so root-only for now and possibly removal soon.

note that this had several limitations already anyway, like running
in sync context and thus failing after 30s.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-09 10:47:44 +02:00
Fiona Ebner
e81a10a4ab api: replication job status: allow querying disabled jobs too
Rather than failing with an error claiming that the job doesn't exist.
The disabled status will be visible in the result of the call.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-07 18:43:32 +02:00
Fabian Grünbichler
8961f9f780 api2: network: improve code readability
nested conditionals stretching over multiple lines are always a bit hard to
untangle, so let's make it explicit:

1. is the interface a bridge
2. if it is, are we looking for one?
3. is it something else that we are looking for?

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-06-07 17:04:29 +02:00
Fabian Grünbichler
9df839bead api2: network: re-use existing variable tfilter
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-06-07 17:04:29 +02:00
Alexandre Derumier
d2894179f4 api2: network: check permissions for local bridges
always check permissions, also when not filtered

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2023-06-07 17:04:29 +02:00
Thomas Lamprecht
e36bc44112 api: backup: check param permission before pool for consistency
Like it did here before 9f65a584 ("api: backup: update: check
permissions of delete params too") and like it does in the create
case.

This should not have a practical effect, it's mostly for consistency
and to avoid anybody reading anything into the different orders of
checks between update and create.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-07 16:52:40 +02:00
Fiona Ebner
43f83ad9ce api: backup/vzdump: add get_storage_param helper
to capture the logic in a single place.

Suggested-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-07 16:47:08 +02:00
Fiona Ebner
b6e561304a api: backup: require Datastore.Allocate on storage
In particular this ensures that the user is allowed to remove data on
the storage, because configuring low retention results in removed
older backups. Of course setting the storage itself also needs to
require the same privilege then.

This is a breaking API change, but it seems sensible to require
permissions on the affected storage too.

Jobs with a dumpdir setting can be configured by root only.

Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-07 16:47:08 +02:00
Fiona Ebner
9f65a584b7 api: backup: update: check permissions of delete params too
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-07 16:47:08 +02:00
Fiona Ebner
659032f48e api: backup: update: allow only deleting
Previously, it was required to set something at the same time.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-07 16:47:08 +02:00
Fiona Ebner
bda3f2aab7 api: backup: update: turn delete into a hash
makes it easier to check for keys in the following patches.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-07 16:47:08 +02:00
Dominik Csapak
9802297517 api: add realm-sync crud api to /cluster/jobs
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 [ T: fix-up realm sync module namespace, moved to PVE::API2::Jobs ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-07 11:38:34 +02:00
Fiona Ebner
8b3c353ed7 api: nodes: add 'migrateall' to index
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-06 17:05:06 +02:00
Thomas Lamprecht
93542d7748 pveceph: support new enterprise repository
With Proxmox VE 8, we'll have support for a enterprise ceph repo,
accessed through Proxmox VE subscriptions, to provide more broadly
tested ceph updates for production setups.

Replace the test-repository parameter with an actual enum of
selectable repo types for:

- test (same as previously selected through setting test-repository)
- no-subscription (the previous default, then named "main")
- enterprise (new and the default now, recommended for production)

Note that writing the auth-part is a bit hacky and might/should be
improved.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-05 19:04:12 +02:00
Thomas Lamprecht
fc3e061bd0 api: subscription: factor out check for cache being valid
multi-line post-if's are against our style guide.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-05 19:04:12 +02:00
Thomas Lamprecht
76a165a1f2 api: subscription: style cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-05 19:04:12 +02:00
Thomas Lamprecht
147d67c495 makefile: convert to use simple parenthesis
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-29 18:24:00 +02:00
Fiona Ebner
16f3482b34 api: ceph: mon create: remove superfluous verification call
The pve_verify_cidr{,v4,v6} functions were originally intended for
the /etc/network/interfaces API endpoints and thus are a bit
restrictive. For example, as reported in the community forum[0],
pve_verify_cidr() does not consider '0::/0' and '0::/1' to be valid.

The error message in this scenario being
> value does not look like a valid CIDR network
is also confusing, as the first thought of users will be that it comes
from the passed-in monitor address.

The public networks are not written here and read from the Ceph config
and via a RADOS mon command, so no need to try and verify them. If
something really would go wrong during parsing, the
get_local_ip_from_cidr() call would complain afterwards.

[0]: https://forum.proxmox.com/threads/125226/

Suggested-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-04-12 13:26:49 +02:00