Commit Graph

3408 Commits

Author SHA1 Message Date
Lukas Wagner
b2000d1f75 notify: pbs-context: exclude successful prunes in default matcher
PBS sends notifications for all events but successful prune jobs.
There we only care about errors.

This commit adapts the 'default-matcher' to reflect that behavior
as well.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-04-23 23:06:56 +02:00
Lukas Wagner
7035d57312 notify: use std::sync::OnceCell instead of lazy_static!
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-04-23 23:06:56 +02:00
Lukas Wagner
d0b1502803 notify: expose config module
This is needed because we want to access CONFIG and PRIVATE_CONFIG
from the docgen helper in PBS.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-04-23 23:06:56 +02:00
Lukas Wagner
c028a32c1e notify: renderer: add relative-percentage helper from PBS
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
803bf7cdc7 notify: pbs context: include nodename in default sendmail author
The old notification stack in proxmox-backup includes the nodename, so
we include it here as well.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
c55f37b8c4 notify: derive Deserialize/Serialize for Notification struct
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
1a40d34083 notify: derive api for Deleteable*Property
The API endpoints in Proxmox Backup Server require ApiType to be
implemented for any deserialized parameter.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
87f7dfa111 notify: api: add get_targets
This method allows us to get a list of all notification targets.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
e83269be1d notify: give each notification a unique ID
We need this for queuing notifications on PBS from the unprivileged
proxy process.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
efb576385b notify: cargo.toml: add spaces before curly braces
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
2f40b79f8f notify: make the mail-forwarder feature depend on proxmox-sys
It uses proxmox_sys::nodename - the dep is needed, otherwise the code
does not compile in some feature flag permutations.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
50d80328e5 notify: don't make tests require pve-context
Tests now have their own context, so requiring pve-context is not
necessary any more.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
d61e3fc759 notify: convert Option<Vec<T>> -> Vec<T> in config structs
Suggested-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
a4d5594721 notify: make api methods take config struct ownership
This saves us from some of the awkward cloning steps when updating.

Suggested-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
1516cc26d2 notify: switch to file-based templating system
Instead of passing the template strings for subject and body when
constructing a notification, we pass only the name of a template.
When rendering the template, the name of the template is used to find
corresponding template files. For PVE, they are located at
/usr/share/proxmox-ve/templates/default. The `default` part is
the 'template namespace', which is a preparation for user-customizable
and/or translatable notifications.

Previously, the same template string was used to render HTML and
plaintext notifications. This was achieved by providing some template
helpers that 'abstract away' HTML/plaintext formatting. However,
in hindsight this turned out to be pretty finicky. Since the
current changes lay the foundations for user-customizable notification
templates, I ripped these abstractions out. Now there are simply two
templates, one for plaintext, one for HTML.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Fabian Grünbichler
730f4e58ff GC: flatten existing status into job status
to avoid drifting definitions and reduce duplication. with the next major
release, the 'upid' field could then be renamed and aliased to be in line with
the other jobs, which all use 'last-run-upid'. doing it now would break
existing callers of the GC status endpoint (or consumers of the on-disk status
file).

the main difference is that the GC status fields are now not optional (except
for the UPID) in the job status, since flattening an optional value is not
possible. this only affects datastores that were never GCed at all, and only
direct API consumers, since the UI handles those fields correctly.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-04-22 13:58:08 +02:00
Stefan Lendl
163732177d api: garbage collect job status
Adds an api endpoint on the datastore that reports the gc job status
such as:
 - Schedule
 - State (of last run)
 - Duration (of last run)
 - Last Run
 - Next Run (if scheduled)
 - Pending Chunks (of last run)
 - Pending Bytes (of last run)
 - Removed Chunks (of last run)
 - Removed Bytes (of last run)

Adds a dedicated endpoint admin/gc that reports gc job status for all
datastores including the onces without a gc-schedule.

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Originally-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewd-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-22 13:58:08 +02:00
Dietmar Maurer
bec18b8e60 api: assert that maintenance mode transitions are valid
Maintenance mode Delete locks the datastore. It must not be possible to go
back to normal modes, because the datastore may be in undefined state.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-04-22 12:19:22 +02:00
Dietmar Maurer
15c013f758 maintenance: derive Copy for maintenance type and make maintenance mode fields public
Because it is a public api type.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-04-22 12:19:22 +02:00
Dietmar Maurer
73bf2b1994 pbs-api-types: use SchemaDeserializer for maintenance mode
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2024-04-22 12:19:22 +02:00
Fabian Grünbichler
42fb9ed26b fix #5249: apt: allow parsing Packages without Priority field
it seems there are repositories out there that don't (always) include
it, and while it is required for the .deb packages themselves in Debian,
the repository "spec" doesn't make it mandatory.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-04-19 08:47:34 +02:00
Fabian Grünbichler
f03f16d643 fix #5249: apt: allow parsing Packages without Priority field
it seems there are repositories out there that don't (always) include
it, and while it is required for the .deb packages themselves in Debian,
the repository "spec" doesn't make it mandatory.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-04-19 08:43:54 +02:00
Lukas Wagner
6b393ac0ce notify: fix #5274: also set 'X-Gotify-Key' header for authentication
Versions of Gotify < 2.2.0 only supported the 'X-Gotify-Key' header
for passing the API token. This comment sets this header in addition
to the regular 'Authorization' header in order to be compatible with
older Gotify servers.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-04 16:51:55 +02:00
Wolfgang Bumiller
6858672642 bump proxmox-http to 0.9.1-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-28 13:18:21 +01:00
Wolfgang Bumiller
9be9d4b6ab http: support ALPN negotiated http2
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-28 11:19:56 +01:00
Thomas Lamprecht
d73eb3dcf1 tree-wide: run cargo fmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-25 17:04:17 +01:00
Christoph Heiss
0475421498 auth-api: implement Display for Realm{, Ref}
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
2024-03-25 17:03:27 +01:00
Christoph Heiss
72afba8b5b ldap: add method for retrieving root DSE attributes
The root DSE holds common attributes about the LDAP server itself.
Needed to e.g. support Active Directory-based LDAP servers to retrieve
the base DN from the server itself, based on an valid bind.

See also RFC 4512, Section 5.1 [0] for more information about this
special object.

[0] https://www.rfc-editor.org/rfc/rfc4512#section-5.1

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
2024-03-25 17:03:27 +01:00
Christoph Heiss
84fbfb22ec ldap: avoid superfluous allocation when calling .search()
The `attrs` parameter of `Ldap::search()` is an `impl AsRef<[impl
AsRef<str>]>` anyway, so replace `vec![..]` with `&[..]`.

Suggested-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
2024-03-25 17:03:27 +01:00
Gabriel Goller
78bf05a458 fix: use fragmented block size for space calculation
We currently calculate the size of a datastore using `statfs64`, which
returns the number of blocks in the fs and the two block sizes:
fragemented block size(f_frsize) and block size (f_bsize). To calculate
eg the total space in a datastore we use total_blocks * f_bsize, which
is not always correct.

`f_frsize` is the minimum unit of allocation on the filesystem (in
bytes) and in 99% of the cases equal to `f_bsize`, but in some cases
it differs. For example some filesystems allow smaller blocks for small
files, in case f_frsize < f_bsize. In that case, f_frsize * total_blocks
returns (mostly) the correct result (ceph also did some weird stuff, which is
now being fixed though [0][1]). `statvfs` also documents this as the
recommended way ('fsblkcnt_t f_blocks;   /* Size of fs in f_frsize units */')[2].

This patch aligns the the behavior with the libc utilities (also used by
`df`) [3].

Motivation: [4] (Forum post)

[0]: https://tracker.ceph.com/issues/3793
[1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=92a49fb0f79f3300e6e50ddf56238e70678e4202
[2]: https://www.man7.org/linux/man-pages/man3/statvfs.3.html
[3]: https://git.savannah.gnu.org/cgit/gnulib.git/tree/lib/fsusage.c#n147
[4]: https://forum.proxmox.com/threads/pbs-3-1-2-wrong-datastore-information-sshfs.139875/#post-626959

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2024-03-25 16:59:17 +01:00
Hannes Laimer
71ff7c3344 datastore: remove datastore from internal cache based on maintenance mode
We keep a DataStore cache, so ChunkStore's and lock files are kept by
the proxy process and don't have to be reopened every time. However,
for specific maintenance modes, e.g. 'offline', our process should not
keep file in that datastore open. This clears the cache entry of a
datastore if it is in a specific maintanance mode and the last task
finished, which also drops any files still open by the process.

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
2024-03-25 16:12:41 +01:00
Wolfgang Bumiller
158f98fe72 cargo fmt (import reordering)
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-20 11:13:13 +01:00
Dietmar Maurer
b8f2582bd9 pbs-api-types: use const_format and new api-types from proxmox-schema
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-03-20 11:09:26 +01:00
Thomas Lamprecht
55f4d532c7 sys: d/copyright: update years
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-19 11:23:43 +01:00
Wolfgang Bumiller
e32081ea5f bump proxmox-notify to 0.3.3-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-19 11:11:02 +01:00
Wolfgang Bumiller
2c2783451f bump proxmox-auth-api to 0.3.4
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-19 11:09:49 +01:00
Wolfgang Bumiller
d653ac343b bump proxmox-schema to 3.1.0-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-19 11:08:28 +01:00
Wolfgang Bumiller
6f1d439f09 notify: adapt to proxmox_schema changes, use const_format
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-19 10:29:27 +01:00
Wolfgang Bumiller
686453a28c notify: sort and group dependencies
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-19 10:27:25 +01:00
Dietmar Maurer
1ac8b7f652 proxmox-schema: moved common api types from pbs-api-types
We want to use those types in all of our products.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-03-18 10:19:52 +01:00
Dietmar Maurer
d74fa06253 proxmox-schema: add IP address regex/api-types
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-03-18 10:19:50 +01:00
Dietmar Maurer
a6f1b36fa6 proxmox-auth-api: use const_format to define static strings
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-03-18 10:19:48 +01:00
Dietmar Maurer
0c5e2640d0 proxmox-schema: use const_format to define static strings.
Macro rules are not hygienic, and current rust macro visibility rules
are a nightmare. Using const_format::concatcp!() is a much cleaner
solution.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-03-18 10:19:46 +01:00
Wolfgang Bumiller
c67a13f1d7 bump proxmox-acme to 0.5.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-07 13:27:47 +01:00
Wolfgang Bumiller
fbb3049768 acme: formatting fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-07 13:24:42 +01:00
Wolfgang Bumiller
b5255f1868 acme: drop api-types feature from schema dependency
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-07 13:24:37 +01:00
Dietmar Maurer
0370723261 proxmox-acme: derive PartialEq for API types
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-03-07 13:21:54 +01:00
Dietmar Maurer
619414d4f1 proxmox-acme: add api-types feature
Because AccountData is exposed via our API (currently as type Object).

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-03-07 13:21:52 +01:00
Wolfgang Bumiller
724c3dda6f rrd: fixup examples with the renamed types
Some types were recently renamed but the examples not updated
accordingly.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Fixes: 2f94283367 "rrd: spell out hard to understand abbreviations in public types"
2024-02-21 12:28:10 +01:00
Wolfgang Bumiller
05ff6b545a bump proxmox-schema to 3.0.1-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-02-21 12:28:02 +01:00