Commit 0d66acd3 ("api types: introduce `BackupArchiveName` type")
introduced a dedicated archive name api type to add rust type
checking and bundle helpers to the api type. Since this, the backup
archive name to server archive name mapping is handled by its parser.
This however did not cover the `.conf` extension used for VM config
files. Add the missing `.conf` to `.conf.blob` to the match statement
and the test cases.
Fixes: 0d66acd3 ("api types: introduce `BackupArchiveName` type")
Reported-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
... and deserialize with default if field is missing in data.
Reported-by: Aaron Lauterer <a.lauterer@proxmox.com>
Fixes: 35fb5d4f7f ("pbs-api-types: add mount_status field to DataStoreListItem")
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
So it is possible to reset it after a failed unmount, or abort an
unmount task by resetting it through the API.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
The proxmox-notify crate can render notification text based on two
different templates, plaintext and html. The html template is at the
moment only used for email-based notifications. If we try to render
a html-formatted message but there is no html template, we try to
fall back to the plaintext template and wrap the rendered message
in <pre> tags.
As a preparation for user-supplied/overridden templates, I added a log
message "html template not found, falling back to plaintext ..." to
educate the user about this behavior.
In Proxmox Backup Server, we only ship plaintext templates at the
moment, meaning that this log message will be shown for every single
(email) notification that is sent out. This might be a bit confusing,
because the log message can be interpreted as an error, which it isn't.
This commit removes the log message completely for now. Once we add
support for user-overridable notification templates we could consider
adding it back it, but maybe phrased a bit differently, to avoid it
being interpreted as an error.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Even if we don't have an HTML template available, we always
send an HTML part (the plain text part wrapped in <pre>) to
improve rendering in certain mail clients. This means
we can simply message formatting, since we do not have to
distinguish between single-part and multi-part messages.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
... and add MaintenanceType::Delete to it. We also want to clear any
cach entries if we are deleting the datastore, not just if it is marked
as offline.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Selective because there are quite a few more such old-style format
strings, but I had those already adapted and currently do not have
time do clean-up tree-wide, but it's fine to change this
incrementally.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
That's rather excessive and has not much value for users. So degrade
two of the messages to debug-level.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Add a custom handlebars escape function. It's nearly identical to the
default `html_escape` fn [0], but it does not escape the '='. This is
needed to support base64 encoded values.
[0]: https://docs.rs/handlebars/latest/handlebars/fn.html_escape.html
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
[ TL: use full width for comment ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
we already have two different password schemas, `PBS_PASSWORD_SCHEMA`
being the stricter one, which ensures a minimum length of new
passwords. however, this wasn't used on the change password endpoint
before, so add it there too. this is also in-line with NIST's latest
recommendations [1].
[1]: https://pages.nist.gov/800-63-4/sp800-63b.html#passwordver
Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
PathPatterns is hard to distinguish from PathPattern, so would need to be
renamed anyway.. but there isn't really a reason to define a separate API type
just for this.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Introduces a dedicated api type `PathPattern` and the corresponding
format and input validation schema. Further, add a `PathPatterns`
type for collections of path patterns and implement required traits
to be able to replace currently defined api parameters.
In preparation for using this common api type for all api endpoints
exposing a match pattern parameter.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
This option allows us to "fix" corrupt snapshots (and/or their chunks)
by pulling them from another remote. When traversing the remote
snapshots, we check if it exists locally, and if it is, we check if the
last verification of it failed. If the local snapshot is broken and the
`resync-corrupt` option is turned on, we pull in the remote snapshot,
overwriting the local one.
This is very useful and has been requested a lot, as there is currently
no way to "fix" corrupt chunks/snapshots even if the user has a healthy
version of it on their offsite instance.
Originally-by: Shannon Sterz <s.sterz@proxmox.com>
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Introduces a dedicated wrapper type to be used for backup archive
names instead of plain strings and associated helper methods for
archive type checks and archive name mappings.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
FG: use LazyLock for constant archive names reduces churn, and saves some
allocations
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Moving the `ArchiveType` to avoid crate dependencies on
`pbs-datastore`.
In preparation for introducing a dedicated `BackupArchiveName` api
type, allowing to set the corresponding archive type variant when
parsing the archive name based on it's filename.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
In order for sync jobs to be either pull or push jobs, allow to
configure the direction of the job.
Adds an additional config type `sync-push` to the sync job config, to
clearly distinguish sync jobs configured in pull and in push
direction and defines and implements the required `SyncDirection` api
type.
This approach was chosen in order to limit possible misconfiguration,
as unintentionally switching the sync direction could potentially
delete still required snapshots.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Add a dedicated api type for the `version` api endpoint and helper
methods for supported feature comparison.
This will be used to detect api incompatibility of older hosts, not
supporting some features.
Use the new api type to refactor the version endpoint and set it as
return type.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Make the `BackupGroupDeleteStats` exposable via the API by implementing
the ApiTypes trait via the api macro invocation and add an additional
field to account for the number of deleted groups.
Further, add a method to add up the statistics.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
In preparation for the delete stats to be exposed as return type to
the backup group delete api endpoint.
Also, rename the private field `unremoved_protected` to a better
fitting `protected_snapshots` to be in line with the method names.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Adding the privileges to allow backup, namespace creation and prune
on remote targets, to be used for sync jobs in push direction.
Also adds dedicated roles setting the required privileges.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Add `remote_acl_path` method which generates the acl path from the sync
job configuration. This helper allows to easily generate the acl path
from a given sync job config for privilege checks.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Add a `remote_acl_path` helper method for creating acl paths for
remote namespaces, to be used by the priv checks on remote datastore
namespaces for e.g. the sync job in push direction.
Factor out the common path extension into a dedicated method.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
For how to work on the crates in this workspace while actually working
on a separate project without having to constantly reinstall `.deb`
files.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
The 'Anyhow' error is not useful and meant for throw-away errors which
cannot be dealt with anyway, and we'd like to be able to tell apart
network problems from actual HTTP responses, so that we can
potentially try a different node in a cluster connection.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
removable datastores will have a PBS-managed mountpoint as path, direct
access to the field needs to be replaced with a helper that can account
for this.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
I considered keeping it as log::trace level, but IMO that's just not
worth it, as just the peek_len is not giving one much more and can
also be basically also gathered through strace.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
When a connection is closed by the client before we have enough data
to determine if it contains a TLS Handshake or not, the socket stays
in a readable state.
While we setup a tokio backed timeout of 10s for the connection
build-up here, this timeout does not trigger on said early connection
abort from the client side, causing then the async_io loop to
endlessly loop around peeking into the client, which always returns
the last available bytes before the connection was closed. This in
turn causes 100% CPU usage for one of the PBS threads.
The timeout not triggering is rather odd, and does indicate some
potential for further improvement in tokio itself, but our
questionable use of the WouldBlock error does violate the API
contract, so this is not a clear cut.
Such an early connection abort is often triggered by monitoring
solutions, which use it to relatively cheaply check if TCP on a port
still works as "is service up" heuristic.
To fix this, save the amount of bytes peek returned and if they did
not change between invocations of the callback, we can assume that the
connection was closed and thus exit the connection attempt with an
error.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
[ TL: reword commit message and change error to ConnectionAborted ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
All in all pretty similar to other endpoint APIs.
One thing worth noting is how secrets are handled. We never ever
return the values of previously stored secrets in get_endpoint(s)
calls, but only a list of the names of all secrets. This is needed
to build the UI, where we display all secrets that were set before in
a table.
For update calls, one is supposed to send all secrets that should be
kept and updated. If the value should be updated, the name and value
is expected, and if the current value should preseved, only the name
is sent. If a secret's name is not present in the updater, it will be
dropped. If 'secret' is present in the 'delete' array, all secrets
will be dropped, apart from those which are also set/preserved in the
same update call.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-By: Stefan Hanreich <s.hanreich@proxmox.com>
This target type allows users to perform HTTP requests to arbitrary
third party (notification) services, for instance
ntfy.sh/Discord/Slack.
The configuration for these endpoints allows one to freely configure
the URL, HTTP Method, headers and body. The URL, header values and
body support handlebars templating to inject notification text,
metadata and secrets. Secrets are stored in the protected
configuration file (e.g. /etc/pve/priv/notification.cfg) as key value
pairs, allowing users to protect sensitive tokens/passwords.
Secrets are accessible in handlebar templating via the secrets.*
namespace, e.g. if there is a secret named 'token', a body
could contain '{{ secrets.token }}' to inject the token into the
payload.
A couple of handlebars helpers are also provided:
- url-encoding (useful for templating in URLs)
- escape (escape any control characters in strings)
- json (print a property as json)
In the configuration, the body, header values and secret values
are stored in base64 encoding so that we can store any string we want.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-By: Stefan Hanreich <s.hanreich@proxmox.com>
A recent commit [1] changed the `Display` implementation of `TimeSpan` such
that minutes are now displayed as `20m` instead of `20min`.
This commit adapts the tests for the notification template renderer
accordingly.
[1] 19129960 ("time: display minute/month such that it can be parsed again")
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
