Jobs for both sync directions are now stored using the same `sync`
config section type, so drop the outdated helpers.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>
Add the sync direction for the sync job as optional config parameter
and refrain from using the config section type for conditional
direction check, as they are now the same (see previous commit).
Use the configured sync job parameter instead of passing it to the
various methods as function parameter and only filter based on sync
direction if an optional api parameter to distingush/filter based on
direction is given.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>
By default, the sync client from proxmox-http (powered by ureq) does not
have any request timeout. To avoid blocking the current thread for a
prolonged period of time, we now make use of the previously added
`Client::new_with_timeout` function to create a new HTTP client with a
default timeout of 10 seconds.
In the long run it would be nicer to have a higher timeout here, say
60s, to cope with flaky and high-latency networks and potentially
overloaded targets. But for that we need to change the architecture of
how notifications are send out to ensure that now thread accepting
connections can be blocked.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
[ TL: Change timeout from 5s to 10s as trade-off and expand commit
message slightly with some reasoning for that still relatively
short time value ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
This commits adds the possibility to set a HTTP request timeout for the
sync client.
For now, I've opted to add this as a separate option than can be set via
a separate new_with_timeout method as compared to adding it to the HttpOptions
struct. While it of course would make a lot of sense to add it to the
latter, this would require adding support for request timeouts to the
async client as well. Some users of the async client handle request
timeouts externally via tokio::time::timeout, so these would need to
modified as well. I don't want to touch this at the moment,
so I've opted to introduce the timeout to the sync client only for now.
We can always revisit this at a later time and move the option to the
HttpOptions struct.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Commit 0d66acd3 ("api types: introduce `BackupArchiveName` type")
introduced a dedicated archive name api type to add rust type
checking and bundle helpers to the api type. Since this, the backup
archive name to server archive name mapping is handled by its parser.
This however did not cover the `.conf` extension used for VM config
files. Add the missing `.conf` to `.conf.blob` to the match statement
and the test cases.
Fixes: 0d66acd3 ("api types: introduce `BackupArchiveName` type")
Reported-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
... and deserialize with default if field is missing in data.
Reported-by: Aaron Lauterer <a.lauterer@proxmox.com>
Fixes: 35fb5d4f7f ("pbs-api-types: add mount_status field to DataStoreListItem")
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
So it is possible to reset it after a failed unmount, or abort an
unmount task by resetting it through the API.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
The proxmox-notify crate can render notification text based on two
different templates, plaintext and html. The html template is at the
moment only used for email-based notifications. If we try to render
a html-formatted message but there is no html template, we try to
fall back to the plaintext template and wrap the rendered message
in <pre> tags.
As a preparation for user-supplied/overridden templates, I added a log
message "html template not found, falling back to plaintext ..." to
educate the user about this behavior.
In Proxmox Backup Server, we only ship plaintext templates at the
moment, meaning that this log message will be shown for every single
(email) notification that is sent out. This might be a bit confusing,
because the log message can be interpreted as an error, which it isn't.
This commit removes the log message completely for now. Once we add
support for user-overridable notification templates we could consider
adding it back it, but maybe phrased a bit differently, to avoid it
being interpreted as an error.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Even if we don't have an HTML template available, we always
send an HTML part (the plain text part wrapped in <pre>) to
improve rendering in certain mail clients. This means
we can simply message formatting, since we do not have to
distinguish between single-part and multi-part messages.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
... and add MaintenanceType::Delete to it. We also want to clear any
cach entries if we are deleting the datastore, not just if it is marked
as offline.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Selective because there are quite a few more such old-style format
strings, but I had those already adapted and currently do not have
time do clean-up tree-wide, but it's fine to change this
incrementally.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
That's rather excessive and has not much value for users. So degrade
two of the messages to debug-level.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Add a custom handlebars escape function. It's nearly identical to the
default `html_escape` fn [0], but it does not escape the '='. This is
needed to support base64 encoded values.
[0]: https://docs.rs/handlebars/latest/handlebars/fn.html_escape.html
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
[ TL: use full width for comment ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
we already have two different password schemas, `PBS_PASSWORD_SCHEMA`
being the stricter one, which ensures a minimum length of new
passwords. however, this wasn't used on the change password endpoint
before, so add it there too. this is also in-line with NIST's latest
recommendations [1].
[1]: https://pages.nist.gov/800-63-4/sp800-63b.html#passwordver
Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
PathPatterns is hard to distinguish from PathPattern, so would need to be
renamed anyway.. but there isn't really a reason to define a separate API type
just for this.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Introduces a dedicated api type `PathPattern` and the corresponding
format and input validation schema. Further, add a `PathPatterns`
type for collections of path patterns and implement required traits
to be able to replace currently defined api parameters.
In preparation for using this common api type for all api endpoints
exposing a match pattern parameter.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
This option allows us to "fix" corrupt snapshots (and/or their chunks)
by pulling them from another remote. When traversing the remote
snapshots, we check if it exists locally, and if it is, we check if the
last verification of it failed. If the local snapshot is broken and the
`resync-corrupt` option is turned on, we pull in the remote snapshot,
overwriting the local one.
This is very useful and has been requested a lot, as there is currently
no way to "fix" corrupt chunks/snapshots even if the user has a healthy
version of it on their offsite instance.
Originally-by: Shannon Sterz <s.sterz@proxmox.com>
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Introduces a dedicated wrapper type to be used for backup archive
names instead of plain strings and associated helper methods for
archive type checks and archive name mappings.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
FG: use LazyLock for constant archive names reduces churn, and saves some
allocations
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Moving the `ArchiveType` to avoid crate dependencies on
`pbs-datastore`.
In preparation for introducing a dedicated `BackupArchiveName` api
type, allowing to set the corresponding archive type variant when
parsing the archive name based on it's filename.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
In order for sync jobs to be either pull or push jobs, allow to
configure the direction of the job.
Adds an additional config type `sync-push` to the sync job config, to
clearly distinguish sync jobs configured in pull and in push
direction and defines and implements the required `SyncDirection` api
type.
This approach was chosen in order to limit possible misconfiguration,
as unintentionally switching the sync direction could potentially
delete still required snapshots.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Add a dedicated api type for the `version` api endpoint and helper
methods for supported feature comparison.
This will be used to detect api incompatibility of older hosts, not
supporting some features.
Use the new api type to refactor the version endpoint and set it as
return type.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Make the `BackupGroupDeleteStats` exposable via the API by implementing
the ApiTypes trait via the api macro invocation and add an additional
field to account for the number of deleted groups.
Further, add a method to add up the statistics.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
In preparation for the delete stats to be exposed as return type to
the backup group delete api endpoint.
Also, rename the private field `unremoved_protected` to a better
fitting `protected_snapshots` to be in line with the method names.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Adding the privileges to allow backup, namespace creation and prune
on remote targets, to be used for sync jobs in push direction.
Also adds dedicated roles setting the required privileges.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Add `remote_acl_path` method which generates the acl path from the sync
job configuration. This helper allows to easily generate the acl path
from a given sync job config for privilege checks.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Add a `remote_acl_path` helper method for creating acl paths for
remote namespaces, to be used by the priv checks on remote datastore
namespaces for e.g. the sync job in push direction.
Factor out the common path extension into a dedicated method.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
For how to work on the crates in this workspace while actually working
on a separate project without having to constantly reinstall `.deb`
files.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
The 'Anyhow' error is not useful and meant for throw-away errors which
cannot be dealt with anyway, and we'd like to be able to tell apart
network problems from actual HTTP responses, so that we can
potentially try a different node in a cluster connection.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
removable datastores will have a PBS-managed mountpoint as path, direct
access to the field needs to be replaced with a helper that can account
for this.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
I considered keeping it as log::trace level, but IMO that's just not
worth it, as just the peek_len is not giving one much more and can
also be basically also gathered through strace.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
