Commit Graph

3481 Commits

Author SHA1 Message Date
Christian Ebner
573d4d149b pbs-api-types: add garbage collection cache capacity tuning option
Allows to adjust the capacity for the LRU cache used to keep track of
recently touched chunks during phase 1 of garbage collection.

Values are provided as multiples of 1024 cache entries, the default
value of 1024 * 1024  was chosen as tradeoff between runtime
improvements and memory usage [0]. The maximum of 8192 * 1024 was
chosen based on the linear regression from [1], resulting in about
8 * 80 MiB = 640 MiB of memory requirement, while allowing to keep
chunks which can reference about 32 TiB of data in case of 4 MiB
fixed size chunks.

[0] https://git.proxmox.com/?p=proxmox-backup.git;a=commit;h=03143eee0a59cf319be0052e139f7e20e124d572
[1] https://lore.proxmox.com/pbs-devel/fa3800dd-e812-4c9a-9d3d-2d8673e05355@proxmox.com/

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Link: https://lore.proxmox.com/pbs-devel/20250404130713.376630-1-c.ebner@proxmox.com
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2025-04-05 15:59:11 +02:00
Christian Ebner
88e4f4831a pbs-api-types: sync: add sync encrypted/verified snapshots only flags
Add optional sync job config options to allow to include only
encrypted and/or verified backup snapshots, excluding others from the
sync.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Link: https://lore.proxmox.com/pbs-devel/20250404132106.388829-2-c.ebner@proxmox.com
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2025-04-05 15:59:11 +02:00
Christoph Heiss
969e1ad645 pbs-api-types: introduce proper types for PAM and PBS realms
Introduces two new, simple API types representing the built-in PAM and
PBS authentication realms.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
 [TL: adapt doc-comment for new default field like Shannon pointed out
 but with a slightly different text than she proposed]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2025-04-05 15:59:11 +02:00
Christoph Heiss
b143160c12 fix #5379: pbs-api-types: add default field for all realm types
The field indicates whether the realm should be the default realm to
select in the login dialog.

Per definition, the field should only ever be set to `true` on exactly
one realm - up to the consumer/storage to ensure that.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
 [TL: adapt doc-comment for new default field like Shannon pointed out
  but with a slightly different text than she proposed and change
  regular comment to doc-comment.]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2025-04-05 15:59:08 +02:00
Christian Ebner
8468dd18d2 pbs-api-types: add option to set GC chunk cleanup atime cutoff
Add the `gc-atime-cutoff` option to the datastore tuning parameters.
This allows to specify the time after which the chunks are not
considered in use anymore if their atime has not been updated since
then.

The default is to keep chunks within the 24h 5m timespan (given no
active writers).

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2025-04-05 15:58:45 +02:00
Christian Ebner
0f658cf79b pbs-api-types: add garbage collection atime safety check flag
Add the `gc-atime-safety-check` flag to the datastore tuning
parameters. This flag allows to enable/disable a check to detect if
the atime update is honored by the filesystem backing the chunk store
during phase 1 of garbage collection and during datastore creation.

The default is to perform the check.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2025-04-05 15:58:39 +02:00
Fabian Grünbichler
f1bcce2796 openid: adapt d/control
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2025-04-04 15:51:23 +02:00
Fabian Grünbichler
56763c1184 bump proxmox-openid to 0.10.4-1
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2025-04-04 15:41:08 +02:00
Thomas Skinner
6f5fefecc0 fix #4234: openid: add library functions for optional userinfo endpoint
Signed-off-by: Thomas Skinner <thomas@atskinner.net>
FG: rebased
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2025-04-04 15:40:28 +02:00
Thomas Skinner
1c25b76c5d fix #4411: openid: add library code for generic id token claim support
Signed-off-by: Thomas Skinner <thomas@atskinner.net>
Tested-by: Mira Limbeck <m.limbeck@proxmox.com>
Reviewed-by: Mira Limbeck <m.limbeck@proxmox.com>
2025-04-04 14:52:23 +02:00
Thomas Lamprecht
c057adcfed pbs-api-types: bump to 0.2.1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2025-04-02 15:51:17 +02:00
Hannes Laimer
6dd53a89b5 pbs-api-types: add REGENERATE_TOKEN_SCHEMA
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2025-04-02 15:01:35 +02:00
Dominik Csapak
1fc5630c71 pbs api types: tape backup job: add worker threads option
Sometimes it's useful to give the user control over how much
parallelized the job is, so introduce a worker thread option.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2025-04-02 14:47:57 +02:00
Lukas Wagner
4097d3697d notify: gotify: use constant from http crate for 'Authorization' header
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2025-03-25 19:37:37 +01:00
Lukas Wagner
6d4c115f05 notify: webhook: gotify: set Content-Length header
To quote from RFC 9110 [1]:

  A user agent SHOULD send Content-Length in a request when
  the method defines a meaning for enclosed content and it
  is not sending Transfer-Encoding. For example, a user agent
  normally sends Content-Length in a POST request even when
  the value is 0 (indicating empty content).
  A user agent SHOULD NOT send a Content-Length header field
  when the request message does not contain content and the
  method semantics do not anticipate such data.

It seemed like our HTTP client lib did not set the header
automatically, which is why we should do it manually.

While most services seemed to have worked fine without setting
the header, some Microsoft services seem to require it
to accept the webhook request [2].

[1] https://datatracker.ietf.org/doc/html/rfc9110#name-content-length
[2] https://forum.proxmox.com/threads/158827

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2025-03-25 19:37:37 +01:00
Thomas Lamprecht
7abd2da759 pbs-api-types: acl: fix indentation error in macro
expand tabs to spaces

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2025-03-24 17:56:13 +01:00
Dietmar Maurer
ec8a3de133 sys: procfs: split read_meminfo into read and parse functions
So that we can write tests.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2025-03-20 18:42:05 +01:00
Wolfgang Bumiller
da8fdea632 rest-server: bump to 0.8.8-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2025-03-20 14:45:19 +01:00
Christian Ebner
10cf5ed7b4 rest-server: worker task: include context in state error message
Currently the anyhow error context of a given error is not included
in the error message, as `to_string` does use the default formatting
[0].

Include the error context, formatting it as single line as the
message is also shown to the users in e.g. the Proxmox Backup Severs
task state in the UI.

[0] https://docs.rs/anyhow/latest/anyhow/struct.Error.html#display-representations

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2025-03-20 14:40:58 +01:00
Maximiliano Sandoval
25c08ad247 sys: add variable bindings for temporaries in unsafe blocks
These will produce an error in edition 2024 otherwise. The reason this
is needed is because the `unsafe` block has its own scope.

The bytes were defined inside of the let-mut block to preserve the
lifetime they had before this commit.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2025-03-19 12:32:42 +01:00
Wolfgang Bumiller
e06277ac7a log: bump to 0.2.8-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2025-03-19 12:19:43 +01:00
Wolfgang Bumiller
86a517d087 sys: bump to 0.6.6-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2025-03-19 12:19:43 +01:00
Wolfgang Bumiller
57eb5a36e9 sys, shared-memory: deny unsafe_op_in_unsafe_fn explicitly
can be removed in these and the other crates when switching to edition
2024

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2025-03-19 12:19:43 +01:00
Wolfgang Bumiller
d42810e3c1 log, rest-server: cargo fmt / formatting cleanups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2025-03-19 12:19:43 +01:00
Maximiliano Sandoval
3db442fb8f async: accommodate to edition 2024 changes to RPIT
Prevents the following error:

```
error[E0597]: `inner` does not live long enough
   --> proxmox-async/src/broadcast_future.rs:109:24
    |
107 |         inner: Arc<Mutex<BroadCastFutureBinding<T>>>,
    |         ----- binding `inner` declared here
108 |     ) -> impl Future<Output = Result<T, Error>> {
109 |         let mut data = inner.lock().unwrap();
    |                        ^^^^^ borrowed value does not live long enough
...
121 |         data.broadcast.listen()
    |         ----------------------- argument requires that `inner` is borrowed for `'static`
122 |     }
    |     - `inner` dropped here while still borrowed

error[E0597]: `data` does not live long enough
   --> proxmox-async/src/broadcast_future.rs:121:9
    |
109 |         let mut data = inner.lock().unwrap();
    |             -------- binding `data` declared here
...
121 |         data.broadcast.listen()
    |         ^^^^-------------------
    |         |
    |         borrowed value does not live long enough
    |         argument requires that `data` is borrowed for `'static`
122 |     }
    |     - `data` dropped here while still borrowed
```

The use<...> pattern was introduced in rust 1.82.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2025-03-19 12:19:33 +01:00
Maximiliano Sandoval
51c3a31115 mark blocks inside unsafe fns unsafe
In edition 2024 unsafe code inside unsafe functions has to be explicitly
marked as such.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2025-03-19 12:19:19 +01:00
Maximiliano Sandoval
9be42ea5ad daemon: set_var is now unsafe
In edition 2024 set_var is unsafe.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2025-03-19 12:19:19 +01:00
Maximiliano Sandoval
abd07ffcff mark extern C blocks as unsafe
This is required in edition 2024.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2025-03-19 12:19:19 +01:00
Gabriel Goller
a75b97da76 log, rest-server: worker_task: add log_unfiltered
To write result message manually, bypassing tracing.

The workertasks currently get their status from parsing the log
messages in the task-log file. The problem is that if these messages are
filtered – which is now possible using the PBS_LOG env variable – some
workertasks will end up with a "stopped: unknown" status. This is not
desirable so write the message manually to the workertask file and
bypass tracing.

This way we are guaranteed that, regardless of the max logging level the
user sets, the final message (and status) is written.

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2025-03-19 12:19:02 +01:00
Wolfgang Bumiller
c99308ecfc log: factor out NoWorkerTask filter
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2025-03-19 11:52:08 +01:00
Wolfgang Bumiller
6bdd07075d log: fix doctests
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2025-03-19 11:51:53 +01:00
Gabriel Goller
656fedb0c4 log: add layer for pve workertasks in perlmod crates
Add a layer that outputs messages to stderr in a specific format. In
PVE, stderr is rerouted to the tasklog if the we are within a
workertask. Therefore, ensure the stderr output is formatted
appropriately.

Reported-by: Lukas Wagner <l.wagner@proxmox.com>
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2025-03-19 09:52:35 +01:00
Gabriel Goller
f6269b800d log: introduce logging builder
Add a builder-like struct to compose a tracing logger using different
layers. Instead of having an init function per product/binary or
super-specific init functions that describe the logger, have a dynamic
builder. The builder improves the usability and makes the logging
initialization more self-explaining.

Suggested-by: Lukas Wagner <l.wagner@proxmox.com>
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2025-03-19 09:52:35 +01:00
Shannon Sterz
3e382fd29c auth-api: set content type header for the new HttpOnly ticket endpoint
otherwise some clients might struggle to interpret the body correctly

Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
2025-03-14 09:28:11 +01:00
Fabian Grünbichler
b82e51f15a bump proxmox-router to 3.1.1-1
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2025-03-13 13:58:50 +01:00
Fabian Grünbichler
e4bc435beb env_logger: bump to 0.11
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2025-03-13 13:55:29 +01:00
Wolfgang Bumiller
f32f48b119 router: cli: avoid unnecessary clones/allocation
The `new_args` Vec is directly passed to the other Vec's `.extend()`,
which takes an `IntoIterator` consuming it, so just pass the
intermediate `Iterator`.

The `rest` Vec owns its strings and we don't need it afterwards, so
similarly, we can consume it via `.extend()` instead of a manual
push(s.clone()) loop.

The .truncate(0) can just be .clear() - they are equivalent according
to their documentation.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2025-03-13 11:46:52 +01:00
Shannon Sterz
00c75c734d tree-wide: fix private intra doc links
a previous commit fixed up all intra doc links that were present on
public apis, this also fixes the links for private members.

Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
2025-03-07 11:35:03 +01:00
Shannon Sterz
a9a7bbdabc auth-api: fix intra doc link for Empty
`Display` isn't used directly anymore, so fix up the intra doc link
here again.

Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
2025-03-06 16:35:51 +01:00
Wolfgang Bumiller
5c7b1ab4ab tfa, auth-api: simplify and restyle Display implementation
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2025-03-06 15:25:24 +01:00
Shannon Sterz
4836cb5334 tree-wide: fix intra doc links
this fixes intra document links or rephrases the documentation in a
more appropriate way to remove all `broken_intra_doc_links` warnings.

Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
2025-03-06 15:25:24 +01:00
Shannon Sterz
ccb34b33e2 api-macro: re-order ObjectSchema fields to be sorted
this panics when running `cargo test` otherwise, as the api macro
requires fields in `ObjectSchema`s to be sorted now.

Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
2025-03-06 15:25:24 +01:00
Shannon Sterz
f0b23def30 router: fix nested doc test cases to match inteded output
commit 68b13965 (router: docs: add horizontal line before nested
command docs) broke the nested command group test case. this commit
adapts the expected output accordingly.

Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
2025-03-06 15:25:24 +01:00
Shannon Sterz
4fc074b4ba network-api: ignore clippy lint about upper case acronyms
while the lint is correct about how these enum members should be
capitalized, the enum is marked as `pub` and all users of it would
need to adapt. so ignore the lint for now [1].

[1]:
https://rust-lang.github.io/rust-clippy/master/index.html#upper_case_acronyms

Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
2025-03-06 15:25:24 +01:00
Shannon Sterz
730f908458 apt: ignore clippy lint about new having to return Self
while this is a reasonable convention to follow, in this case the new
function is part of a public trait and changing the signature would
force all users to adapt. so ignore the lint for now [1].

[1]:
https://rust-lang.github.io/rust-clippy/master/index.html#new_ret_no_self

Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
2025-03-06 15:25:24 +01:00
Shannon Sterz
8fc324ee73 apt: ignore clippy lint about using a slice reference instead of &Vec
while the function would be more useful as pointed out by the clippy
lint, it i currently `pub` and users of the function would need to
adapt to the change here. so ignore the lint for now.

[1]: https://rust-lang.github.io/rust-clippy/master/index.html#ptr_arg

Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
2025-03-06 15:25:24 +01:00
Shannon Sterz
59898d0177 rest-server/router: ignore type complexity clippy lint
the `type_complexity` clippy lint [1] is intended to make the code
more legible in most cases. however, the lint triggers on a member of
a private enum, an example minimal rest server and a private static
variable here. so the benefits of declaring a new type that would
encapsulate this complexity is minimal. hence, ignore the warnings for
now.

[1]:
https://rust-lang.github.io/rust-clippy/master/index.html#type_complexity

Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
2025-03-06 15:25:24 +01:00
Shannon Sterz
f9dd576783 router: ignore clippy lint missing_transmute_annotations
the `ApiHandler`'s `PartialEq` implementation makes heavy use of
`transmute`. clippy wants the types to be explicitly stated here and
not inferred, to avoid potential undefined behaviour if an unexpected
type is inferred. however, the types that would be inferred here are
so complex, that the code would become illegible anyway, so ignore
this lint for now.

[1]:
https://rust-lang.github.io/rust-clippy/master/index.html#missing_transmute_annotations

Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
2025-03-06 15:25:24 +01:00
Shannon Sterz
dcc6eb9918 shared-memory: specify generic types for transmute
this annotates a `transmute` call with proper types to avoid possible
undefined behaviour, as suggested by clippy [1].

[1]:
https://rust-lang.github.io/rust-clippy/master/index.html#missing_transmute_annotations

Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
2025-03-06 15:25:24 +01:00
Shannon Sterz
ab2d5c9777 acme/auth-api: add Default for types with un-parameterized new()
this fixes a clippy lint for types that have a `new()` function that
has no parameters [1]. this should allow using these types with
functions such as `unwrap_or_default()`.

[1]:
https://rust-lang.github.io/rust-clippy/master/index.html#new_without_default

Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
2025-03-06 15:25:24 +01:00