Commit Graph

3472 Commits

Author SHA1 Message Date
Gabriel Goller
bfa73aadf8 metrics: encode influxdb org and bucket parameters
In order to remove the current limitations on the bucket and
organization names, we need to make sure that they are transmitted
correctly. In order to do this, we encode them using the url crate.

This way we support organization/bucket names that include slashes,
whitespaces, etc.

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2024-04-26 17:25:52 +02:00
Thomas Lamprecht
5987eb0c3f sys: bump version to 0.5.4-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-26 17:24:23 +02:00
Thomas Lamprecht
c68b69e949 sys: bump version to 0.5.4-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-26 16:52:46 +02:00
Stefan Lendl
f2633b462f config: write vlan network interface
* Add vlan_id and vlan_raw_device fields to the Interface api type
* Write to the network config the vlan specific properties for vlan
  interface type
* Add several tests to verify the functionally

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:48:50 +02:00
Thomas Lamprecht
863d760340 auth-api: bump version to 0.3.5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:22:33 +02:00
Thomas Lamprecht
bd944b06f9 ldap: bump version to 0.2.2-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:13:45 +02:00
Christoph Heiss
7db5cd8c48 realm sync: add sync job for AD realms
Basically just a thin wrapper over the existing LDAP-based realm sync
job, which retrieves the appropriate config and sets the correct user
attributes.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-24 21:06:14 +02:00
Christoph Heiss
8bdf9ac45c api: access: add routes for managing AD realms
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-24 21:06:14 +02:00
Fabian Grünbichler
f4a9afd17c notify: fix TemplateType::Subject doc comment
wrongly copied

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-04-24 08:05:16 +02:00
Lukas Wagner
1372617876 api-types: api: tape: add notification-mode parameter
Same as with datastores, this option determines whether we send
notifications the old way (send email via sendmail to a user's email
address) or the new way (emit matchable notification events to the
notification stack).

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
90603f6e25 api-types: api: datatore: add notification-mode parameter
This one lets the user choose between the old notification behavior
(selecting an email address/user and always/error/never behavior per
datastore) and the new one (emit notification events to the
notification system)

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Thomas Lamprecht
0af0bad742 notify: bump version to 0.4.0-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:19 +02:00
Lukas Wagner
8f408ea4af notify: add getter for notification timestamp
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-04-23 23:06:56 +02:00
Lukas Wagner
08b7c501ac notify: endpoints: matcher: improve descriptions for API types
proxmox-schema will automatically append text (e.g. 'Can be specified
more than once'), so we should end every comment with a '.'.

Also copy over some text from PVE docs, since these doc comments will
now be visible in the PBS documentation.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-04-23 23:06:56 +02:00
Lukas Wagner
b2000d1f75 notify: pbs-context: exclude successful prunes in default matcher
PBS sends notifications for all events but successful prune jobs.
There we only care about errors.

This commit adapts the 'default-matcher' to reflect that behavior
as well.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-04-23 23:06:56 +02:00
Lukas Wagner
7035d57312 notify: use std::sync::OnceCell instead of lazy_static!
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-04-23 23:06:56 +02:00
Lukas Wagner
d0b1502803 notify: expose config module
This is needed because we want to access CONFIG and PRIVATE_CONFIG
from the docgen helper in PBS.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-04-23 23:06:56 +02:00
Lukas Wagner
c028a32c1e notify: renderer: add relative-percentage helper from PBS
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
803bf7cdc7 notify: pbs context: include nodename in default sendmail author
The old notification stack in proxmox-backup includes the nodename, so
we include it here as well.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
c55f37b8c4 notify: derive Deserialize/Serialize for Notification struct
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
1a40d34083 notify: derive api for Deleteable*Property
The API endpoints in Proxmox Backup Server require ApiType to be
implemented for any deserialized parameter.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
87f7dfa111 notify: api: add get_targets
This method allows us to get a list of all notification targets.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
e83269be1d notify: give each notification a unique ID
We need this for queuing notifications on PBS from the unprivileged
proxy process.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
efb576385b notify: cargo.toml: add spaces before curly braces
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
2f40b79f8f notify: make the mail-forwarder feature depend on proxmox-sys
It uses proxmox_sys::nodename - the dep is needed, otherwise the code
does not compile in some feature flag permutations.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
50d80328e5 notify: don't make tests require pve-context
Tests now have their own context, so requiring pve-context is not
necessary any more.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
d61e3fc759 notify: convert Option<Vec<T>> -> Vec<T> in config structs
Suggested-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
a4d5594721 notify: make api methods take config struct ownership
This saves us from some of the awkward cloning steps when updating.

Suggested-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Lukas Wagner
1516cc26d2 notify: switch to file-based templating system
Instead of passing the template strings for subject and body when
constructing a notification, we pass only the name of a template.
When rendering the template, the name of the template is used to find
corresponding template files. For PVE, they are located at
/usr/share/proxmox-ve/templates/default. The `default` part is
the 'template namespace', which is a preparation for user-customizable
and/or translatable notifications.

Previously, the same template string was used to render HTML and
plaintext notifications. This was achieved by providing some template
helpers that 'abstract away' HTML/plaintext formatting. However,
in hindsight this turned out to be pretty finicky. Since the
current changes lay the foundations for user-customizable notification
templates, I ripped these abstractions out. Now there are simply two
templates, one for plaintext, one for HTML.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-04-23 23:06:52 +02:00
Fabian Grünbichler
730f4e58ff GC: flatten existing status into job status
to avoid drifting definitions and reduce duplication. with the next major
release, the 'upid' field could then be renamed and aliased to be in line with
the other jobs, which all use 'last-run-upid'. doing it now would break
existing callers of the GC status endpoint (or consumers of the on-disk status
file).

the main difference is that the GC status fields are now not optional (except
for the UPID) in the job status, since flattening an optional value is not
possible. this only affects datastores that were never GCed at all, and only
direct API consumers, since the UI handles those fields correctly.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-04-22 13:58:08 +02:00
Stefan Lendl
163732177d api: garbage collect job status
Adds an api endpoint on the datastore that reports the gc job status
such as:
 - Schedule
 - State (of last run)
 - Duration (of last run)
 - Last Run
 - Next Run (if scheduled)
 - Pending Chunks (of last run)
 - Pending Bytes (of last run)
 - Removed Chunks (of last run)
 - Removed Bytes (of last run)

Adds a dedicated endpoint admin/gc that reports gc job status for all
datastores including the onces without a gc-schedule.

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Originally-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewd-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-22 13:58:08 +02:00
Dietmar Maurer
bec18b8e60 api: assert that maintenance mode transitions are valid
Maintenance mode Delete locks the datastore. It must not be possible to go
back to normal modes, because the datastore may be in undefined state.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-04-22 12:19:22 +02:00
Dietmar Maurer
15c013f758 maintenance: derive Copy for maintenance type and make maintenance mode fields public
Because it is a public api type.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-04-22 12:19:22 +02:00
Dietmar Maurer
73bf2b1994 pbs-api-types: use SchemaDeserializer for maintenance mode
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2024-04-22 12:19:22 +02:00
Fabian Grünbichler
42fb9ed26b fix #5249: apt: allow parsing Packages without Priority field
it seems there are repositories out there that don't (always) include
it, and while it is required for the .deb packages themselves in Debian,
the repository "spec" doesn't make it mandatory.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-04-19 08:47:34 +02:00
Fabian Grünbichler
f03f16d643 fix #5249: apt: allow parsing Packages without Priority field
it seems there are repositories out there that don't (always) include
it, and while it is required for the .deb packages themselves in Debian,
the repository "spec" doesn't make it mandatory.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-04-19 08:43:54 +02:00
Lukas Wagner
6b393ac0ce notify: fix #5274: also set 'X-Gotify-Key' header for authentication
Versions of Gotify < 2.2.0 only supported the 'X-Gotify-Key' header
for passing the API token. This comment sets this header in addition
to the regular 'Authorization' header in order to be compatible with
older Gotify servers.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-04 16:51:55 +02:00
Wolfgang Bumiller
6858672642 bump proxmox-http to 0.9.1-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-28 13:18:21 +01:00
Wolfgang Bumiller
9be9d4b6ab http: support ALPN negotiated http2
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-28 11:19:56 +01:00
Thomas Lamprecht
d73eb3dcf1 tree-wide: run cargo fmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-25 17:04:17 +01:00
Christoph Heiss
0475421498 auth-api: implement Display for Realm{, Ref}
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
2024-03-25 17:03:27 +01:00
Christoph Heiss
72afba8b5b ldap: add method for retrieving root DSE attributes
The root DSE holds common attributes about the LDAP server itself.
Needed to e.g. support Active Directory-based LDAP servers to retrieve
the base DN from the server itself, based on an valid bind.

See also RFC 4512, Section 5.1 [0] for more information about this
special object.

[0] https://www.rfc-editor.org/rfc/rfc4512#section-5.1

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
2024-03-25 17:03:27 +01:00
Christoph Heiss
84fbfb22ec ldap: avoid superfluous allocation when calling .search()
The `attrs` parameter of `Ldap::search()` is an `impl AsRef<[impl
AsRef<str>]>` anyway, so replace `vec![..]` with `&[..]`.

Suggested-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
2024-03-25 17:03:27 +01:00
Gabriel Goller
78bf05a458 fix: use fragmented block size for space calculation
We currently calculate the size of a datastore using `statfs64`, which
returns the number of blocks in the fs and the two block sizes:
fragemented block size(f_frsize) and block size (f_bsize). To calculate
eg the total space in a datastore we use total_blocks * f_bsize, which
is not always correct.

`f_frsize` is the minimum unit of allocation on the filesystem (in
bytes) and in 99% of the cases equal to `f_bsize`, but in some cases
it differs. For example some filesystems allow smaller blocks for small
files, in case f_frsize < f_bsize. In that case, f_frsize * total_blocks
returns (mostly) the correct result (ceph also did some weird stuff, which is
now being fixed though [0][1]). `statvfs` also documents this as the
recommended way ('fsblkcnt_t f_blocks;   /* Size of fs in f_frsize units */')[2].

This patch aligns the the behavior with the libc utilities (also used by
`df`) [3].

Motivation: [4] (Forum post)

[0]: https://tracker.ceph.com/issues/3793
[1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=92a49fb0f79f3300e6e50ddf56238e70678e4202
[2]: https://www.man7.org/linux/man-pages/man3/statvfs.3.html
[3]: https://git.savannah.gnu.org/cgit/gnulib.git/tree/lib/fsusage.c#n147
[4]: https://forum.proxmox.com/threads/pbs-3-1-2-wrong-datastore-information-sshfs.139875/#post-626959

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2024-03-25 16:59:17 +01:00
Hannes Laimer
71ff7c3344 datastore: remove datastore from internal cache based on maintenance mode
We keep a DataStore cache, so ChunkStore's and lock files are kept by
the proxy process and don't have to be reopened every time. However,
for specific maintenance modes, e.g. 'offline', our process should not
keep file in that datastore open. This clears the cache entry of a
datastore if it is in a specific maintanance mode and the last task
finished, which also drops any files still open by the process.

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
2024-03-25 16:12:41 +01:00
Wolfgang Bumiller
158f98fe72 cargo fmt (import reordering)
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-20 11:13:13 +01:00
Dietmar Maurer
b8f2582bd9 pbs-api-types: use const_format and new api-types from proxmox-schema
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-03-20 11:09:26 +01:00
Thomas Lamprecht
55f4d532c7 sys: d/copyright: update years
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-19 11:23:43 +01:00
Wolfgang Bumiller
e32081ea5f bump proxmox-notify to 0.3.3-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-19 11:11:02 +01:00
Wolfgang Bumiller
2c2783451f bump proxmox-auth-api to 0.3.4
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-19 11:09:49 +01:00