Commit Graph

297 Commits

Author SHA1 Message Date
Christian Ebner
00254d60e3 api types: version: implement traits to allow for version comparison
Derive and implement the traits to allow comparison of two
`ApiVersion` instances for more direct and easy api version
comparisons. Further, add some basic test cases to reduce risk of
regressions.

This is useful for e.g. feature compatibility checks by comparing api
versions of remote instances.

Example comparison:
```
api_version >= ApiVersion::new(3, 3, 0)
```

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-12-02 15:27:37 +01:00
Christian Ebner
d11393c70e api types: version: drop unused repoid field
The `ApiVersion` type was introduced in commit a926803b
("api/api-types: refactor api endpoint version, add api types")
including the `repoid`, added for completeness when converting from
a pre-existing `ApiVersionInfo` instance, as returned by the
`version` api endpoint.

Drop the additional `repoid` field, since this is currently not used,
can be obtained fro the `ApiVersionInfo` as well and only hinders the
implementation for easy api version comparison.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-12-02 15:27:37 +01:00
Fabian Grünbichler
4d5e14c07e datastore: extract nesting check into helper
and improve the variable namign while we are at it. this allows the check to be
re-used in other code paths, like when starting a garbage collection.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-11-27 15:25:37 +01:00
Dominik Csapak
af4d5607f1 sync jobs: remove superfluous direction property
since the SyncJobConfig struct now contains a 'sync-direction' property, we can
omit the 'direction' property of the SyncJobStatus struct. This makes a
few adaptions in the ui necessary:

* use the correct field
* handle 'pull' as default (since we don't necessarily get a
  'sync-direction' in that case)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-11-26 16:02:22 +01:00
Christian Ebner
e9dfb83131 api types: drop unused config type helpers for sync direction
Jobs for both sync directions are now stored using the same `sync`
config section type, so drop the outdated helpers.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>
2024-11-26 16:00:50 +01:00
Christian Ebner
4b76b731cb api: admin/config: introduce sync direction as job config parameter
Add the sync direction for the sync job as optional config parameter
and refrain from using the config section type for conditional
direction check, as they are now the same (see previous commit).

Use the configured sync job parameter instead of passing it to the
various methods as function parameter and only filter based on sync
direction if an optional api parameter to distingush/filter based on
direction is given.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>
2024-11-26 16:00:40 +01:00
Dominik Csapak
23185135bb api: admin: sync: add direction to sync job status
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-11-26 14:54:33 +01:00
Christian Ebner
0db4d9031b api types: add missing conf to blob archive name mapping
Commit addfae26 ("api types: introduce `BackupArchiveName` type")
introduced a dedicated archive name api type to add rust type
checking and bundle helpers to the api type. Since this, the backup
archive name to server archive name mapping is handled by its parser.

This however did not cover the `.conf` extension used for VM config
files. Add the missing `.conf` to `.conf.blob` to the match statement
and the test cases.

Fixes: addfae26 ("api types: introduce `BackupArchiveName` type")
Reported-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-11-26 13:28:17 +01:00
Hannes Laimer
8c7492b99b api: types: add 'mount_status' to schema
... and deserialize with default if field is missing in data.

Reported-by: Aaron Lauterer <a.lauterer@proxmox.com>
Fixes: 76609915d6 ("pbs-api-types: add mount_status field to DataStoreListItem")
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2024-11-26 13:08:04 +01:00
Hannes Laimer
e90baeaaa8 api: maintenance: allow setting of maintenance mode if 'unmounting'
So it is possible to reset it after a failed unmount, or abort an
unmount task by resetting it through the API.

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2024-11-26 13:07:53 +01:00
Hannes Laimer
76609915d6 pbs-api-types: add mount_status field to DataStoreListItem
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2024-11-25 21:34:22 +01:00
Dietmar Maurer
66389b2fd9 maintenance: add 'Unmount' maintenance type
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2024-11-25 21:34:22 +01:00
Hannes Laimer
652b774eb0 maintenance: make is_offline more generic
... and add MaintenanceType::Delete to it. We also want to clear any
cach entries if we are deleting the datastore, not just if it is marked
as offline.

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2024-11-25 21:34:22 +01:00
Hannes Laimer
89c650b83e pbs-api-types: add backing-device to DataStoreConfig
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2024-11-25 21:34:22 +01:00
Shannon Sterz
fb5b6f3eab api: enforce minimum character limit of 8 on new passwords
we already have two different password schemas, `PBS_PASSWORD_SCHEMA`
being the stricter one, which ensures a minimum length of new
passwords. however, this wasn't used on the change password endpoint
before, so add it there too. this is also in-line with NIST's latest
recommendations [1].

[1]: https://pages.nist.gov/800-63-4/sp800-63b.html#passwordver

Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
2024-11-25 15:51:47 +01:00
Fabian Grünbichler
2009d8de41 api types: replace PathPatterns with Vec<PathPattern>
PathPatterns is hard to distinguish from PathPattern, so would need to be
renamed anyway.. but there isn't really a reason to define a separate API type
just for this.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-11-25 12:28:40 +01:00
Christian Ebner
45b5556765 api-types: implement dedicated api type for match patterns
Introduces a dedicated api type `PathPattern` and the corresponding
format and input validation schema. Further, add a `PathPatterns`
type for collections of path patterns and implement required traits
to be able to replace currently defined api parameters.

In preparation for using this common api type for all api endpoints
exposing a match pattern parameter.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-11-25 11:57:07 +01:00
Gabriel Goller
0974ddfa17 fix #3786: api: add resync-corrupt option to sync jobs
This option allows us to "fix" corrupt snapshots (and/or their chunks)
by pulling them from another remote. When traversing the remote
snapshots, we check if it exists locally, and if it is, we check if the
last verification of it failed. If the local snapshot is broken and the
`resync-corrupt` option is turned on, we pull in the remote snapshot,
overwriting the local one.

This is very useful and has been requested a lot, as there is currently
no way to "fix" corrupt chunks/snapshots even if the user has a healthy
version of it on their offsite instance.

Originally-by: Shannon Sterz <s.sterz@proxmox.com>
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-11-25 10:53:26 +01:00
Fabian Grünbichler
83810759ee api types: extend backup archive name parsing tests
and also test the error triggered by a directory path being passed in.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-11-22 13:47:25 +01:00
Christian Ebner
db5bf33cfe api types: add unit tests for backup archive name parsing
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-11-22 13:47:25 +01:00
Christian Ebner
addfae26cf api types: introduce BackupArchiveName type
Introduces a dedicated wrapper type to be used for backup archive
names instead of plain strings and associated helper methods for
archive type checks and archive name mappings.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>

FG: use LazyLock for constant archive names reduces churn, and saves some
allocations

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-11-22 13:46:35 +01:00
Christian Ebner
e932ec101e datastore: move ArchiveType to api types
Moving the `ArchiveType` to avoid crate dependencies on
`pbs-datastore`.

In preparation for introducing a dedicated `BackupArchiveName` api
type, allowing to set the corresponding archive type variant when
parsing the archive name based on it's filename.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-11-22 11:45:43 +01:00
Fabian Grünbichler
a5350595fc version: remove named features
and use version comparison for the push code that previously used it.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-11-21 11:53:49 +01:00
Christian Ebner
bcd80bf976 api types/config: add sync-push config type for push sync jobs
In order for sync jobs to be either pull or push jobs, allow to
configure the direction of the job.

Adds an additional config type `sync-push` to the sync job config, to
clearly distinguish sync jobs configured in pull and in push
direction and defines and implements the required `SyncDirection` api
type.

This approach was chosen in order to limit possible misconfiguration,
as unintentionally switching the sync direction could potentially
delete still required snapshots.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-11-21 10:14:53 +01:00
Christian Ebner
a926803b92 api/api-types: refactor api endpoint version, add api types
Add a dedicated api type for the `version` api endpoint and helper
methods for supported feature comparison.
This will be used to detect api incompatibility of older hosts, not
supporting some features.

Use the new api type to refactor the version endpoint and set it as
return type.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-11-21 10:14:53 +01:00
Christian Ebner
f982c915f5 api types: implement api type for BackupGroupDeleteStats
Make the `BackupGroupDeleteStats` exposable via the API by implementing
the ApiTypes trait via the api macro invocation and add an additional
field to account for the number of deleted groups.
Further, add a method to add up the statistics.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-11-21 10:14:53 +01:00
Christian Ebner
d1e5e4533c datastore: move BackupGroupDeleteStats to api types
In preparation for the delete stats to be exposed as return type to
the backup group delete api endpoint.

Also, rename the private field `unremoved_protected` to a better
fitting `protected_snapshots` to be in line with the method names.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-11-21 10:14:53 +01:00
Christian Ebner
db4f1f64b6 api types: define remote permissions and roles for push sync
Adding the privileges to allow backup, namespace creation and prune
on remote targets, to be used for sync jobs in push direction.

Also adds dedicated roles setting the required privileges.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-11-21 10:14:53 +01:00
Christian Ebner
da0fd4267a api types: implement remote acl path method for sync job
Add `remote_acl_path` method which generates the acl path from the sync
job configuration. This helper allows to easily generate the acl path
from a given sync job config for privilege checks.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-11-21 10:14:53 +01:00
Christian Ebner
ae56a50b9d api types: add remote acl path method for BackupNamespace
Add a `remote_acl_path` helper method for creating acl paths for
remote namespaces, to be used by the priv checks on remote datastore
namespaces for e.g. the sync job in push direction.

Factor out the common path extension into a dedicated method.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-11-21 10:14:53 +01:00
Dietmar Maurer
01bbaef7fa config: factor out method to get the absolute datastore path
removable datastores will have a PBS-managed mountpoint as path, direct
access to the field needs to be replaced with a helper that can account
for this.

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2024-11-17 19:57:33 +01:00
Thomas Lamprecht
c606fdaa88 api-types: add missing doc-comment description for api enums
this is used as description in the api schema

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-10-22 15:25:52 +02:00
Lukas Wagner
8ecbb5f152 pbs-api-types: add types for the new metrics endpoint
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2024-10-15 14:09:41 +02:00
Dominik Csapak
ff485aa320 fix #5622: backup client: properly handle rate/burst parameters
The rate and burst parameters are integers, so the mapping from value
with `.as_str()` will always return `None` effectively never
applying any rate limit at all.

Fix it by turning them into a HumanByte instead of an integer.

To not crowd the parameter section so much, create a
ClientRateLimitConfig struct that gets flattened into the parameter list
of the backup client.

To adapt the description of the parameters, add new schemas that copy
the `HumanByte` schema but change the description.

With this, the rate limit actually works, and there is no lower limit
any more.

The old TRAFFIC_CONTROL_RATE/BURST_SCHEMAs can be deleted since the
client was the only user of them.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-08-30 13:21:29 +02:00
Maximiliano Sandoval
a47b71a9ce api-types: remove unused lazy_static dependency
Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-08-14 12:08:01 +02:00
Lukas Wagner
f9843eec16 api-types: rrd: use api-types from proxmox-rrd
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2024-08-09 13:04:57 +02:00
Maximiliano Sandoval
19dfc86198 fix typos in strings
Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-08-07 16:49:31 +02:00
Maximiliano Sandoval
1198253b20 fix typos in rust documentation blocks
Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-08-07 16:49:31 +02:00
Dietmar Maurer
55e7bef4d2 use new apt/apt-api-types crate 2024-07-08 15:28:59 +02:00
Dominik Csapak
b5af9333f8 tape: include drive activity in status
Since we don't query each drives status seperately, but rely on a single
call to the drives listing parameter for that, we now add the option
to query the activity there too. This makes that data avaiable for us
to show in a seperate (by default hidden) column.

Also we show the activity in the 'State' column when the drive is idle
from our perspective. This is useful when e.g. an LTO-9 tape is loaded
the first time and is calibrating, since that happens automatically.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-14 10:31:33 +02:00
Dominik Csapak
1d6b1e0258 tape: add drive activity to drive status api
and show it in the gui for single drives. Adds the known values for the
activity to the UI.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-14 10:25:42 +02:00
Dominik Csapak
3f1a084b90 tape: add functions to parse drive device activity
we use the VHF part from the DT Device Activity page for that.
This is intended to query the drive for it's current state and activity.

Currently only the activity is parsed and used.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-14 10:11:06 +02:00
Dominik Csapak
4b21a00744 tape: save 'bytes used' in tape inventory
and show them on the ui. This can help uses with seeing how much a tape
is used.

The value is updated on 'commit' and when the tape is changed during a
backup.

For drives not supporting the volume statistics, this is simply skipped.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-14 10:07:57 +02:00
Gabriel Goller
3e69aba2d8 api-types: remove influxdb bucket name restrictions
Remove the regex for influxdb organizations and buckets. Influxdb does
not place any constraints on these names and allows all characters. This
allows influxdb organization names with slashes.

Also remove a duplicate comment and add some missing ones.

This also aligns the behavior to PVE as there are no restrictions there
either.

The motivation for this patch is this forum post:
https://forum.proxmox.com/threads/influx-db-organization-doesnt-allow-slash.145402/

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2024-04-26 17:54:51 +02:00
Stefan Lendl
5f14a0a2ab config: write vlan network interface
* Add vlan_id and vlan_raw_device fields to the Interface api type
* Write to the network config the vlan specific properties for vlan
  interface type
* Add several tests to verify the functionally

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:48:50 +02:00
Christoph Heiss
a8636bbb66 realm sync: add sync job for AD realms
Basically just a thin wrapper over the existing LDAP-based realm sync
job, which retrieves the appropriate config and sets the correct user
attributes.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-24 21:06:14 +02:00
Christoph Heiss
c7051f3342 api: access: add routes for managing AD realms
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-24 21:06:14 +02:00
Lukas Wagner
89ef8b5035 api-types: api: tape: add notification-mode parameter
Same as with datastores, this option determines whether we send
notifications the old way (send email via sendmail to a user's email
address) or the new way (emit matchable notification events to the
notification stack).

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
7b668dbc2b api-types: api: datatore: add notification-mode parameter
This one lets the user choose between the old notification behavior
(selecting an email address/user and always/error/never behavior per
datastore) and the new one (emit notification events to the
notification system)

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Fabian Grünbichler
3ae21d87c1 GC: flatten existing status into job status
to avoid drifting definitions and reduce duplication. with the next major
release, the 'upid' field could then be renamed and aliased to be in line with
the other jobs, which all use 'last-run-upid'. doing it now would break
existing callers of the GC status endpoint (or consumers of the on-disk status
file).

the main difference is that the GC status fields are now not optional (except
for the UPID) in the job status, since flattening an optional value is not
possible. this only affects datastores that were never GCed at all, and only
direct API consumers, since the UI handles those fields correctly.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-04-22 13:58:08 +02:00