Commit Graph

4832 Commits

Author SHA1 Message Date
Thomas Lamprecht
c55884d1a7 system report: factor out getting command output into helper fn
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-26 17:19:48 +02:00
Thomas Lamprecht
3b5cb8fd33 system report: add proxmox-boot-tool status output
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-26 17:15:06 +02:00
Thomas Lamprecht
49d465c72a system report: add basic uptime, usage and process info
Command is copied over from Proxmox VE

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-26 17:14:45 +02:00
Thomas Lamprecht
2f7b3e2ecc system report: add all apt repo files
note, we do not filter by *.list or *.source, so one might get also
files that apt won't read, like .dpkg-dist files, but also those with
typos, and thus possibly helpful when debugging things.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-26 17:14:45 +02:00
Thomas Lamprecht
0b9614d5a4 system report: support outputting all files in a directory
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-26 17:14:45 +02:00
Thomas Lamprecht
eb32373e3c system report: group files by categories
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-26 14:42:41 +02:00
Thomas Lamprecht
20875de2ec system report: switch to markdown-like output
allows one to render this via any of the thousands markdown viewers to
get a better formatting.

We can switch our web ui widget to (optionally) render this as html
when a user is viewing it from the UI too.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-26 14:42:41 +02:00
Thomas Lamprecht
11c4632bcf system report: output ldap and oidc realm list
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-26 12:31:44 +02:00
Thomas Lamprecht
84fb190ee2 system report: add more information to lsblk
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-26 11:32:27 +02:00
Dietmar Maurer
1a1870ba86 move TrafficControlCurrentRate to pbs-api-types
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2023-10-15 11:07:58 +02:00
Dietmar Maurer
4a33ad1ca5 move MetricServerInfo definition to pbs-api-types
And derive Clone, Eq and Ord so that we can sort the list in the GUI.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2023-10-09 08:19:08 +02:00
Dietmar Maurer
6d33fb6fe3 pbs-api-types: move node status types from src/api2/types/mod.rs
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2023-09-28 08:13:30 +02:00
Gabriel Goller
48fbce07df fix #4895: scheduled jobs: ignore task-log not found error
Use the job start-time as end-time when it's stuck in the
`JobState::Starting` state, no active working is running and the task
log of the last run doesn't exists.

A user experienced a power loss, which left a GC job in the `Started`
state, but the task log did not exist. This breaks the schedule and no
following GC runs. Now, the error is simply ignored and a new gc job
is started on the next occurrence.

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2023-09-27 17:31:22 +02:00
Gabriel Goller
2aae8544b7 close #3777: Add source information to backup logs
This will show the ip-address of the client creating
the backup in the logs. For example it will output:
"starting new backup on datastore 'test1' from ::ffff:192.168.1.192:
"host/test/2023-08-21T07:28:10Z"".

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2023-09-18 16:36:31 +02:00
Markus Frank
6d5b559bdb remove unnecessary characters in description
remove ." in description of delete_datastore_disk

Signed-off-by: Markus Frank <m.frank@proxmox.com>
2023-09-07 17:50:56 +02:00
Stoiko Ivanov
03a9f822b3 pbs2to3: fix boot-mode detection
/sys/firmware/efi is a directory and std::path::Path seems to detect
only regular files with is_file [0].

Reported in our Enterprise support portal.

Quickly tested the fix on a VM.

https://doc.rust-lang.org/stable/std/path/struct.Path.html#method.is_file

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2023-08-11 09:20:02 +02:00
Fabian Grünbichler
66215b38f8 blob/chunk parse errors: add context
to make it more obvious that blob is a chunk here and which one is affected.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-08-08 13:57:04 +02:00
Fabian Grünbichler
1f4ae5c777 handle pve-kernel -> proxmox-kernel rename
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-08-01 11:48:26 +02:00
Thomas Lamprecht
7f64a6203a sync over pbs2to3 upgrade check-list script to master
so that it's available after for post-upgrade checks

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-07-08 15:06:40 +02:00
Wolfgang Bumiller
3e35b972cb reset tfa failure count when unlocking
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-07-05 14:58:54 +02:00
Thomas Lamprecht
dd1b684447 api: node services: switch to journald and chrony instead of syslog and timesyncd
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-27 13:35:55 +02:00
Wolfgang Bumiller
9b85fdc71d api: include tfa lock status in user list
Like in PVE.
This means that /access/users is now a 'protected' call to
get access to 'tfa.cfg'.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-06-26 19:47:16 +02:00
Wolfgang Bumiller
c74aa900ce enable tfa lockout
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-06-26 19:47:16 +02:00
Wolfgang Bumiller
cfc3b0ff7d manager: add 'user tfa unlock' command
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-06-26 19:47:16 +02:00
Wolfgang Bumiller
8c7b80e44f api: add /access/users/<userid>/unlock-tfa api call
Same as in PVE: unlocks a user's 2nd factors.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-06-26 19:47:16 +02:00
Stefan Sterz
5210f3b532 access: ldap check connection on creation and change
this commit makes the ldap realm endpoints check whether a new or
updated configuration works correctly. it uses the new
`check_connection` function to make sure that a configuration can be
successfully used to connect to and query an ldap directory.

doing so allows us to remove the ldap domain regex. instead of relying
on a regex to make sure that a given distinguished name (dn) could be
correct, we simply let the ldap directory tell us whether it accepts
it. this should also aid with usability as a dn that looks correct
could still be invalid.

this also implicitly removes unauthenticated binds, since the new
`check_connection` function does not support those. it will simply
bail out of the check if a `bind_dn` but no password is configured.
therefore, this is a breaking change.

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2023-06-26 16:08:24 +02:00
Lukas Wagner
08f8a3e5aa api-types: client: datastore: tools: use proxmox-human-bytes crate
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-06-26 13:56:45 +02:00
Maximiliano Sandoval
12c841b4d9 clippy fixes: deref which would be done by auto-deref
This unnecessarily complicates the code.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2023-06-23 12:01:45 +02:00
Maximiliano Sandoval
0d4b7f0271 clippy fixes: Box::new(_) of default value
From rust-lang:

> Why is this bad?
>
> First, it’s more complex, involving two calls instead of one. Second,
> Box::default() can be faster in certain cases.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2023-06-23 12:01:36 +02:00
Maximiliano Sandoval
f7c8903e22 clippy fixes: casting to the same type is unnecessary
Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2023-06-23 12:01:33 +02:00
Maximiliano Sandoval
c78df869e1 clippy fixes: use of ok_or followed by a function call
The function will always be called. This is only bad if it allocates or does some non-trivial amount of work.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2023-06-23 12:01:31 +02:00
Maximiliano Sandoval
cd0daa8b33 clippy fixes: the borrowed expression implements the required traits
Suggests that the receiver of the expression borrows the expression.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2023-06-23 12:01:29 +02:00
Maximiliano Sandoval
9c85840fe2 fix #4734: manager: add user tfa {list, delete} commands
Adds the commands

    proxmox-backup-manager user tfa list <userid>
    proxmox-backup-manager user tfa delete <userid> <id>

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2023-06-23 11:06:06 +02:00
Maximiliano Sandoval
223842887f api: Fix schema return annotation of tfa_update_auth
In the next commit we expose a command to list the tfa methods of a
user. Without this annotation one would get the following error

    unable to format result: got unexpected data (expected null).

when running the proposed cli command.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2023-06-23 07:52:56 +02:00
Wolfgang Bumiller
177ee20bd9 bump proxmox-auth-api dep to 0.3
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-06-14 09:58:33 +02:00
Thomas Lamprecht
b6e7fc9bec clippy fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-17 17:43:17 +02:00
Wolfgang Bumiller
3b7b52dcf8 disable tfa lockout for now
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-05-10 11:47:19 +02:00
Dominik Csapak
bdce7fa154 fix #4719: wait for tape to be available in changer
instead of aborting. If the tape is currently e.g. offline, in an
import/export slot or in the wrong drive, this gives the user the chance to
manually move it/insert it, so that the backup job can continue.

Send an e-mail like we do on a standalone drive, but adapt the messages
to contain the changer instead of the drive.

This can help when not all tapes are currently available in the changer.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2023-05-09 10:23:51 +02:00
Dominik Csapak
2ebe7bb54a tape restore: show required tape list on single snapshot restore
like we do on a full restore, for that refactor the functionality into
'log_required_tapes' to be reused across the workers

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2023-05-09 10:10:39 +02:00
Fabian Grünbichler
40a57cfab0 pull: impl Display for SkipReason
instead of manually doing it in SkipInfo's Display implementation.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-04-24 15:56:26 +02:00
Stefan Hanreich
71db16151d sync job: pull: improve log output
Adding an opening line for every group makes parsing the log easier.

We can also remove the 're-sync [...] done' line, because the next
line should be a progress line anyway.

The new output for the sync job/pull logs looks as follows:

- skipped already synced (happens in most jobs, except for first run)
- re-sync of last synced snapshot (if it still exists on source)
- skipped because of transfer-last (if set and skips something)
- sync of new snapshots (if they exist)

Suggested-By: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
2023-04-24 15:18:58 +02:00
Stefan Hanreich
9b67352abc partial fix #3701: sync job: pull: add transfer-last parameter
Specifying the transfer-last parameter limits the amount of backups
that get synced via the pull command/sync job. The parameter specifies
how many of the N latest backups should get pulled/synced. All other
backups will get skipped.

This is particularly useful in situations where the sync target has
less disk space than the source. Syncing all backups from the source
is not possible if there is not enough disk space on the target.
Additionally this can be used for limiting the amount of data
transferred, reducing load on the network.

The newest backup will always get re-synced, regardless of the setting
of the transfer-last parameter.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
2023-04-24 15:18:51 +02:00
Thomas Lamprecht
bd8e8991d3 Revert "cli: manager, tape, debug: setup auth context to make local connect work"
This reverts commit 6bac67195e and
commit 783e88c2df as they were both
superseded by 569324cb ("provide separate helpers for pub/priv auth
keyring access").
2023-03-29 17:17:40 +02:00
Wolfgang Bumiller
569324cb95 provide separate helpers for pub/priv auth keyring access
This used to be the case before the switch to the auth api
crate and is required for some helpers where we don't want
to have to setup the complete auth context.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-03-29 14:37:45 +02:00
Thomas Lamprecht
783e88c2df cli: manager, tape, debug: avoid setting up auth context for printdoc
as otherwise build fails in a clean environment, where no API auth
key is available.

This whole printdoc command injection is quite ugly and causes
headache in general though, we'd be better off if we could do away
with that..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-03-28 19:07:08 +02:00
Thomas Lamprecht
6bac67195e cli: manager, tape, debug: setup auth context to make local connect work
The manager, tape and debug admin and debug CLI tools all are meant
to run locally on the PBS directly and often directly by the root
user.

So for convenience they will create a valid API ticket for `root@pam`
to connect via localhost to the API proxy, either transparently if
the UID is 0 (= root) or interactively by asking the password of the
root user.

In d97ff8ae ("use new auth api crate") the API key handling was
reworked, but while the call sites that get the auth keys for signing
a ticket where adapted to use the new auth_keyring helper, the
reworked API keeps the key in shared OnceCell which needs to be
actually setup once per executable to the respective correct key,
i.e., public for the proxy, which only checks signatures, but never
creates them, and private for all those that want to sign tickets;
and that was only added for the prrivileged and public daemon, but
not above mentioned CLI tools.

So all calls into the tools that actually required to create a ticket
for connecting to the local API panicked.

Fix this by adding a call to setup_auth_context in the main entry
point of each of those CLI tools, so that they have the private key
available to be used if the tools is executed by root.

Fixes: d97ff8ae ("use new auth api crate")
Reported-by: Friedrich Weber <f.weber@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-03-28 17:20:23 +02:00
Stefan Sterz
7d2b0d6afd fix #4521: api/tasks: replace upid as filename for task log downloads
previously the upid would just be used without a file extension when
downloading a task log. this lead to rather strange filenames that
appeared unfamiliar to users as the upid is not very prevalent in the
gui. set a proper file name based on the node name, worker type and a
time stamp instead. also add the ".log" file extension to indicate
that these files contain logs.

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2023-03-28 13:54:12 +02:00
Fabian Grünbichler
3df46e018b drop exclusive lock for verify-after-complete
the backup is finished at that point, the only lock clash that is possible when
dropping the exclusive and attempting to obtain a shared lock would be

- the snapshot is pruned/removed
- the backup is in a pre-upgrade process, and the post-upgrade process opens a reader

the first case is OK, if the other invocation wins the race and removes the
snapshot verification is pointless anyway.

the second case means the snapshot is not verified directly after completion
(this fact would be logged in the backup task log), but usable immediately for
pulling/restoring/..

this should decrease the chances of triggering the issues described in #4523

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-03-27 15:05:40 +02:00
Wolfgang Bumiller
20ecaad13b cargo fmt
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-03-27 15:03:05 +02:00
Lukas Wagner
a1a9fdd8b8 report: add date -R to general system info section
Sometimes it can be quite useful to know when exactly a system report
was generated. Adds the following output:

 # date -R
 Thu, 23 Feb 2023 16:21:12 +0100

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-03-27 14:40:41 +02:00
Lukas Wagner
9273dc188f manager: use view_task_result function for realm sync
Previously, the same approach as in `proxmox-backup-debug` was used.
With the changes from this commit, realm syncing uses the same method for
waiting for task output as other parts of `proxmox-backup-manager`.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-03-27 14:23:11 +02:00
Wolfgang Bumiller
5f0965edba realm sync: replace formatted .context() calls
with .map_err/.ok_or_else - since the formatting should not
happen in the non-error case

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-03-27 11:51:34 +02:00
Wolfgang Bumiller
750704400a realm sync: replace and_then chain with '?' in lookup closure
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-03-27 11:47:26 +02:00
Wolfgang Bumiller
0010d56a00 realm sync: take schema per reference
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-03-27 11:47:26 +02:00
Lukas Wagner
cf4ff8a783 realm sync: show warnings if attributes do not meet their constraints
Previously, if the value of a synced property did not validate properly
(e.g only 1 character in length instead of the required 2), the whole
sync job failed without any useful error message.

In this commit, the values are validated manually by their
respective StringSchema. If the validation fails, the value is
ignored and a warning is displayed in the task log.

In addition to that, some error messages have been improved.
Also, user sync is now more fault-tolerant in general, showing
warnings if something goes wrong while creating/updating a
single user, instead of aborting the whole sync job.

Reported-by: Friedrich Weber <f.weber@proxmox.com>
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Friedrich Weber <f.weber@proxmox.com>
2023-03-27 11:47:13 +02:00
Thomas Lamprecht
6b815bc022 proxy: limit theme value in length and disallow '/'
while with rust strings we cannot inject \0, it feels a bit safer to
enforce some basic restrictions, with length and not containing any
slash seems sensible enough.

Admins should not put sensible data as theme-XYZ.css files in
/usr/share (which is normally readable by all system users anyway)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-03-14 17:54:16 +01:00
Stefan Sterz
ee0eaeae63 proxy: switch to "auto" as the default theme
use the "auto" theme per default. it uses a media query to detect the
users preferred theme.

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2023-03-14 17:50:55 +01:00
Stefan Sterz
a443dd5c52 proxy/ui: implement theme switcher
adds a theme switcher to the ui and handles the necessary cookies in
the backend.

this requires a bump of the widget toolkit so the necessary widgets
are present.

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2023-03-09 08:04:18 +01:00
Thomas Lamprecht
dd06b7f1ee server: switch to inline template variables & small cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-03-09 08:03:43 +01:00
Wolfgang Bumiller
15280f936f typo fix
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-03-07 15:50:19 +01:00
Dominik Csapak
98c359241a jobstate: fix wrong error message
we try to create the job stat dir, not the rrdb one

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2023-03-07 15:48:43 +01:00
Wolfgang Bumiller
d97ff8ae2a use new auth api crate
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-03-02 17:01:35 +01:00
Wolfgang Bumiller
5aeeb44a32 adapt to rest-server 0.3 and http 0.8 changes
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-03-02 15:15:36 +01:00
Wolfgang Bumiller
26f03f9e53 use proxmox-sortable-macro directly
instead of via proxmox-sys

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-03-02 15:15:36 +01:00
Dominik Csapak
250a1363e1 fix #4412: tape: initial WORM support
the only thing preventing us from using WORM tapes was that we relied
on being able to rewrite the media set label when first using a tape
that was pre-allocated in a media-pool.

so instead of needing to write a meida set label with a special uuid,
just save the pool in the media label itself. This has currently no
downsides, as we're not able to move tapes from one pool to another
anyway.

this makes some checks a bit trickier, as we now have to get the pool
out of the media set label and as a fallback look into the media label.

such new tapes can still be read and restored by older proxmox-bacukp-server
versions. The only thing missing is when a tape labeled with the new
format that has an assigned pool, that pool won't show up when the tape
is inventoried in an old version (but can still be used otherwise).

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2023-03-01 08:19:59 +01:00
Dominik Csapak
a59ffbbe2c tape: inventory: don't skip unassigned tapes completely
since commit 139acf37 ("tape: inventory: skip unassigned tapes")
we skip unassigned tapes (special all-zero media-set uuid) when we look
for a catalog. We accidentally skipped storing it in the inventory
completely, which means we never inventoried completely empty tapes.

to fix that, simply move the check below the inserting in the inventory

Fixes: 139acf37 ("tape: inventory: skip unassigned tapes")

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2023-03-01 08:17:12 +01:00
Fabian Grünbichler
06e9e20cdb backup: fix indentation
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-02-24 10:15:02 +01:00
Lukas Wagner
06c01f55d0 auth: unify naming for all authenticator implementations
This also removes the need for two clippy-allow directives.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-02-10 12:46:19 +01:00
Lukas Wagner
7c4189522d auth: add dummy OpenIdAuthenticator struct
When manually adding a user, `lookup_authenticator` is called
to verify whether a realm actually exists. Thus is is necessary
to have dummy implementation for the `ProxmoxAuthenticator` trait
for OpenID realms.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-02-10 12:46:19 +01:00
Lukas Wagner
b9d67b014c docs: add configuration file reference for domains.cfg
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-02-10 12:46:19 +01:00
Lukas Wagner
2b75fbaa33 manager: add commands for managing LDAP realms
Adds commands for managing LDAP realms, including user sync, to
`proxmox-backup-manager`.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-02-10 12:46:19 +01:00
Lukas Wagner
73757fe2df server: add LDAP realm sync job
This commit adds sync jobs for LDAP user sync. As of now, they
can only be started manually.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-02-10 12:46:19 +01:00
Lukas Wagner
3eb6a5d957 api-types: add config options for LDAP user sync
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-02-10 12:46:19 +01:00
Lukas Wagner
19dfcfd80f auth: add LDAP realm authenticator
This commits also makes user authentication async, so that e.g. a not
responding LDAP server cannot block other logins.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-02-10 12:46:19 +01:00
Lukas Wagner
3f22f650a9 api: add routes for managing LDAP realms
Note: bind-passwords set via the API  are not stored in `domains.cfg`,
but in a separate `ldap_passwords.json` file located in
`/etc/proxmox-backup/`.
Similar to the already existing `shadow.json`, the file is
stored with 0600 permissions and is owned by root.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-02-10 12:46:19 +01:00
Lukas Wagner
039fb8066c debug cli: use handle_worker in proxmox-rest-server
The function was moved to proxmox-rest-server to make it
usable in the proxmox-backup-manager cli binary.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-02-10 12:46:19 +01:00
Friedrich Weber
86acc77ee9 fix #4341: manager cli: add commands to run prune/sync/verify jobs
Running configured jobs was already possible using the Web UI, but not
using the CLI. To fix that, this commit adds the following commands to
`proxmox-backup-manager`:

* prune-job run <id>
* sync-job run <id>
* verify-job run <id>

Signed-off-by: Friedrich Weber <f.weber@proxmox.com>
2023-02-09 11:38:24 +01:00
Fabian Grünbichler
c78437e320 verify/protect: improve error on disappearing snapshots
or clients passing in a non-existent snapshot.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-02-08 15:19:32 +01:00
Fabian Grünbichler
07b6db8099 backup/snapshot reader: improve error message for ENOENT
instead of

 Error: unable to open snapshot directory "/full/path/to/snapshot" for locking - ENOENT: No such file or directory

this will now print

 Error: Snapshot vm/800/2023-01-16T12:28:11Z does not exist.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-02-08 15:19:29 +01:00
Wolfgang Bumiller
d842189959 move metrics connection from pbs-config to proxy
it's the only user and pbs-config shouldn't depend on the metric client

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-01-20 10:59:55 +01:00
Lukas Wagner
9936c03a52 api: serde-rename deleteable properties to kebab-case
In former commit, the enum members were renamed to be CamelCase, in
accordance with the usual Rust style guide. However, this broke the
GUI in some places due to failing JSON property deserialization.
To fix this, some serde(rename = "kebab-case") directives were added.

Some properties were also serde-renamed to snake_case, otherwise
it would have been necessary to also modify proxmox-widget-toolkit
as well as PVE source code. This can follow in a later commit if so
desired.

Fixes: a2055c38 fix non-camel-case enums
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-01-20 07:35:10 +01:00
Thomas Lamprecht
1f84ac052b api: node tasks: small cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-01-19 11:37:24 +01:00
Fabian Grünbichler
158db8c01f fix #4483: fix task log command interrupt handling
`proxmox-backup-client task log ..` and `proxmox-backup-manager task log ..`
are used to view the logs of tasks that have been started by another client, so
interrupting the task progress view should not forward the interrupt to the
running task. other call sites of the same helper(s) that spawn a task and then
print its progress should keep the old behaviour of interrupting the spawned
task on C^c.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-01-19 10:11:25 +01:00
Fiona Ebner
07151513fa api: apt: versions: add proxmox-mail-forward to package list
Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-01-18 11:26:25 +01:00
Thomas Lamprecht
0b160cfdec tape inventory: code cleanup listing media assigned to a pool
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-01-16 11:05:35 +01:00
Thomas Lamprecht
7792507a12 tape inventory: code cleanup media pool look-up
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-01-16 11:05:35 +01:00
Thomas Lamprecht
6905bd3735 tape inventory: make both media list methods use same appraoch
Was a bit odd that the very similar (same return type) methods used
a for+if/else+push and a iter+filter+collect approach.

Switch both to the latter and use a match instead of if/else for
shorter code

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-01-16 11:05:35 +01:00
Dominik Csapak
025523c059 fix #4466: show snapshots in media content again
while refactoring the the empty media set checks, we accidentally
reversed one check from !is_empty to unassigned, which now never
included the right media sets.

reversing the condition fixes that.

fixes
52517f7b: ("tape: hide internal use of all zero uuid for unassigned tapes")

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2023-01-16 10:48:31 +01:00
Wolfgang Bumiller
a2055c38fe fix non-camel-case enums
This should have never been started to begin with...
2023-01-05 11:13:46 +01:00
Dominik Csapak
79de842ca3 disk: zfs: improve error logging for zfs commands
zfs errors might include a newline in the output (e.g. when trying to
create a mirror on two differently sized disks), which trips up our
task log status parser since that expectes a 'TASK {status}' on the
beginning of the first line.

print the error from zfs into the log and bail out with a short notice
to check the task log

this fixes the 'unknown error' issue in the ui when an error happnes
during the zfs commands

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2023-01-05 10:36:57 +01:00
Christoph Heiss
ff2910666b api2/node/services: Handle optional services and expose unit-state
.. in the same way the PVE api does, esp. regarding the logic to handle
oneshot and missing services.

This then allows re-using the GUI parts from there as well, so that the
services page in PVE and PBS looks the same.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
2023-01-05 10:12:48 +01:00
Hannes Laimer
f6d6b5a317 fix #4256: api2: remove datastore ACL-node on removal of datastore
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-12-20 10:09:09 +01:00
Hannes Laimer
ca1da2cb3c fix #4256: api2: remove prune jobs on datastore delete
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-12-20 10:09:06 +01:00
Wolfgang Bumiller
f72ccdd65d move pbs_tools::ticket to pbs_ticket
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-12-13 13:58:09 +01:00
Wolfgang Bumiller
8ebb984fbe move pbs_config::tape_encryption_keys to pbs crate lib
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-12-12 14:26:02 +01:00
Wolfgang Bumiller
1104d2a268 move pbs_config::key_config to pbs-key-config
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-12-12 14:19:52 +01:00
Wolfgang Bumiller
28917c7127 update for new proxmox-api-macro
A #[default] attribute now conflicts with an explicitly
defined #[api(default: ...)] value for obvious reasons.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-12-12 11:40:13 +01:00
Lukas Wagner
8a8def934e manager: remove accidentally committed ldap.rs
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2022-12-09 14:59:15 +01:00
Lukas Wagner
86e002c71e debug cli: move parameters into the function signature
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2022-12-09 13:38:38 +01:00
Lukas Wagner
f345d32fac debug cli: add colored output for diff archive
This commit adds the `--color` flag to the `diff archive` tool.
Valid values are `always`, `auto` and `never`. `always` and
`never` should be self-explanatory, whereas `auto` will enable
colors unless one of the following is true:
  - STDOUT is not a tty
  - TERM=dumb is set
  - NO_COLOR is set

The tool will highlight changed file attributes in yellow.
Furthermore, (A)dded files are highlighted in green,
(M)odified in yellow and (D)eleted in red.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2022-12-09 13:38:35 +01:00
Lukas Wagner
81d9b121d1 debug cli: add 'compare-content' flag to diff archive command
When --compare-content is set, the command will compare the
file content instead on relying on mtime to detect modified files.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2022-12-09 13:38:33 +01:00
Lukas Wagner
9e5f02f828 debug cli: show more file attributes for diff archive command
This commit enriches the output of the `diff archive` command,
showing pxar entry type, mode, uid, gid, size, mtime and filename.
Attributes that changed between both snapshots are prefixed
with a "*".

For instance:

$ proxmox-backup-debug diff archive ...
A  f   644  10045  10000    0 B  2022-11-28 13:44:51  add.txt
M  f   644  10045  10000    6 B *2022-11-28 13:45:05  content.txt
D  f   644  10045  10000    0 B  2022-11-28 13:17:09  deleted.txt
M  f   644  10045    *29    0 B  2022-11-28 13:16:20  gid.txt
M  f  *777  10045  10000    0 B  2022-11-28 13:42:47  mode.txt
M  f   644  10045  10000    0 B *2022-11-28 13:44:33  mtime.txt
M  f   644  10045  10000   *7 B *2022-11-28 13:44:59 *size.txt
M  f   644 *64045  10000    0 B  2022-11-28 13:16:18  uid.txt
M *f   644  10045  10000   10 B  2022-11-28 13:44:59  type_changed.txt

Also, this commit ensures that we always show the *new* type.
Previously, the command showed the old type if it was changed.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2022-12-09 13:38:29 +01:00
Wolfgang Bumiller
d9f1ca9a46 tree-wide: bump edition to 2021
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-12-06 13:31:01 +01:00
Fabian Grünbichler
16f6766a68 clippy 1.65 fixes
and rustfmt

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-05 11:40:02 +01:00
Dominik Csapak
139acf37a2 tape: inventory: skip unassigned tapes
tapes that are labeled into a pool but are not in a media-set yet, belong
to the special 'all zero' media-set. these will never have a catalog on them,
so skip them

fixes the issue, that an inventory with 'catalog restore' aborted on
such a tape

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-30 17:28:27 +01:00
Dominik Csapak
52517f7b9e tape: hide internal use of all zero uuid for unassigned tapes
a tape assigned to a pool but no media-set, gets the special 'all zero'
media set in it's MediaSetLabel. Instead of having that constant
scattered all over the code, hide this fact by using wrapper functions
to initialize it that way and to check for it

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-30 17:28:27 +01:00
Stefan Hanreich
ab6964799c Fix formatting in proxmox-backup-manager
Small followup to my previous commit, fixing minor formatting issues.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
2022-11-30 17:24:21 +01:00
Wolfgang Bumiller
857f346c22 fix #3335: allow removing datastore contents on delete
Adds an optional 'destroy-data' parameter to the datastore
remove api call.

Based-on: https://lists.proxmox.com/pipermail/pbs-devel/2022-January/004574.html
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-28 14:45:39 +01:00
Wolfgang Bumiller
dd286be58d fix active state when reading task logs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-25 13:39:59 +01:00
Thomas Lamprecht
b1dacb0710 prune jobs: switch default to on-error-only notifications
we can still do that as notifications for prune jobs weren't released
yet.

We may want to evaluate if we adapt (some) other notification types
too on next major release.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-24 17:33:28 +01:00
Dominik Csapak
ec2926c2e4 disk: smartctl: ignore bit 2 of exitcode
under some conditions, the smartctl exitcode sets bit 2, even if the
smartctl call succeeded, but has e.g. some warnings derived from the
attributes

we do the same in pve, but it is only the first step in fixing #4353, since
we probably should parse the smartcl output better to include
such warnings

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-24 14:47:19 +01:00
Wolfgang Bumiller
6d1f8b4b51 cleanup rrd bounds checks in datastore status call
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-24 13:55:22 +01:00
Daniel Tschlatscher
de923258d0 'available' field in rrd data in the API and change usage of 'total'
The API now exposes the field 'available' as well, with which the
unprivileged total is calculated in all corresponsing views in the
frontend.
The rrd charts now also display the total as the unprivileged total
if available, otherwise the absolute total is used.

Signed-off-by: Daniel Tschlatscher <d.tschlatscher@proxmox.com>
2022-11-24 13:26:27 +01:00
Daniel Tschlatscher
f362f8f075 fix #4077: Estimated Full metric on ext4 file systems
The rrd data now includes tracking the available field in disk usage.
The calculation for the estimated_time_full was adapted to use the
total for the unpriviliged user, which is the sum of used + available.

The total for unprivileged users is preferable, because datastores are
always written to by the backup user. Which means that any storage
space reserved for root is unusable for our purposes.

To avoid resetting the estimate when switching to this new version,
the backend will try to use the available value to calculate the
unprivileged total. When that is not an option, it will fall back to
using the absolute total.

Signed-off-by: Daniel Tschlatscher <d.tschlatscher@proxmox.com>
2022-11-24 13:26:24 +01:00
Wolfgang Bumiller
2943d00c84 cleanup for line limit & less indentation
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-24 10:47:09 +01:00
Daniel Tschlatscher
67a5999aa6 make tasklog downloadable in the backup server backend
The read_tasklog API call now stream the whole log file if the query
parameter 'download' is set to true. If the limit parameter is set to
0, all lines in the tasklog will be returned in json format.

To make a file stream and a json response in the same API call work, I
had to use one of the lower level apimethod types from the
proxmox-router. Therefore, the routing declarations and parameter
schemas have been changed accordingly.

Signed-off-by: Daniel Tschlatscher <d.tschlatscher@proxmox.com>
2022-11-24 10:40:05 +01:00
Wolfgang Bumiller
45c7d758e5 distinguish between block/chardevs in diff output
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-23 10:07:23 +01:00
Lukas Wagner
52189f181f fix #3828: proxmox_backup_debug: Introduce diff archive subcommand.
This new subcommand compares a pxar archive in two different
snapshots and prints a list of added/modified/deleted file
entries.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2022-11-23 09:56:16 +01:00
Aaron Lauterer
fcc8e35499 report: add arcstat
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2022-10-21 17:32:02 +02:00
Stefan Hanreich
3c78a1dec2 fix #4301: correctly pass rate limit parameters to API
With the old code the rate limit parameters got passed in their own
dictionary under the limit key, but the API expects the rate-limit
settings as top-level keys. This commit correctly sets the rate-limit
parameters so the API actually uses them.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
2022-10-20 16:06:09 +02:00
Dominik Csapak
647186ddf9 datastore: implement sync-level tuning for datastores
currently, we don't (f)sync on chunk insertion (or at any point after
that), which can lead to broken chunks in case of e.g. an unexpected
powerloss. To fix that, offer a tuning option for datastores that
controls the level of syncs it does:

* None (default): same as current state, no (f)syncs done at any point
* Filesystem: at the end of a backup, the datastore issues
  a syncfs(2) to the filesystem of the datastore
* File: issues an fsync on each chunk as they get inserted
  (using our 'replace_file' helper) and a fsync on the directory handle

a small benchmark showed the following (times in mm:ss):
setup: virtual pbs, 4 cores, 8GiB memory, ext4 on spinner

size                none    filesystem  file
2GiB (fits in ram)   00:13   0:41        01:00
33GiB                05:21   05:31       13:45

so if the backup fits in memory, there is a large difference between all
of the modes (expected), but as soon as it exceeds the memory size,
the difference between not syncing and syncing the fs at the end becomes
much smaller.

i also tested on an nvme, but there the syncs basically made no difference

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-20 14:59:15 +02:00
Dominik Csapak
f963bc0c81 proxmox-tape: inventory: add default to parameters
and convert the 'Option<bool>' to 'bool'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-20 13:22:47 +02:00
Dominik Csapak
4ebc97ad4b proxmox-tape: add 'catalog' option to 'proxmox-tape inventory'
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-20 13:22:43 +02:00
Dominik Csapak
c658ea6185 api/tape/inventory: optionally try to restore catalogs
in a disaster recovery case, it is useful to not only re-invetorize
the labels + media-sets, but also to try to recover the catalogs
from the tape (to know whats on there). This adds an option to
the inventory api call that tries to do a fast catalog restore
from each tape to be inventorized.

also sets the correct default for 'read-all-labels' in the api and
converts to a bool

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-20 13:22:42 +02:00
Dominik Csapak
3921deb29a tape: replace '&Path' with 'AsRef<Path>' in function parameters
this way we can omit the pattern
```
let status_path = Path::new(TAPE_STATUS_DIR);
some_function(status_path);
```
and give the TAPE_STATUS_DIR directly. In some instances we now have to
give TAPE_STATUS_DIR more often, but most often we save a few
intermediary Paths.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-20 13:22:39 +02:00
Fabian Grünbichler
237beaed93 cargo fmt
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-10-11 09:37:16 +02:00
Fabian Grünbichler
f097eaa80f clippy fixes
and one additional API fn "allow many parameters" addition.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-10-11 09:36:12 +02:00
Thomas Lamprecht
c36c901172 prune job: various cleanups, line reduction
no semantic change intended

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-07 15:39:57 +02:00
Thomas Lamprecht
97184e14b4 prune job: avoid duplicate code when assembling CLI keep options
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-07 15:37:32 +02:00
Dominik Csapak
cf91a072ef fix #4274: implement prune notifications
we converted the prune settings of datastores to prune-jobs, but did
not actually implement the notifications for them, even though
we had the notification options in the gui (they did not work).

implement the basic ok/error notification for prune jobs

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-07 08:35:56 +02:00
Dominik Csapak
1053a71e81 tape/inventory: make 'load_media_db' a method
and use self.inventory_path. This is only used internally (not pub) so there
is no need to have it as a static function.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-05 19:26:36 +02:00
Thomas Lamprecht
6088459570 api: disk SMART: fix details for depreacated return value comment
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-09-23 10:33:17 +02:00
Matthias Heiserer
500014e83b fix #4165: SMART: add raw field
This makes it consistent with the naming scheme in PVE/GUI.
Keep value for API stability reasons, and remove it in next major version.

Signed-off-by: Matthias Heiserer <m.heiserer@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.cspak@proxmox.com>
2022-09-23 10:32:30 +02:00
Fabian Grünbichler
37cfd7b336 daily update: fix subscription check
this wrongly triggered for non-subscribed systems as well.

Fixes: dd16e1dac8
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-23 07:58:17 +02:00
Thomas Lamprecht
1e85f97b9f api/pull: drop unused ApiMethod info
the API macro can handle this

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-09-23 07:57:03 +02:00
Matthias Heiserer
4d226f963f pbs-manager: fix pull not respecting local ns setting
It pulls to root namespace instead.

Signed-off-by: Matthias Heiserer <m.heiserer@proxmox.com>
2022-09-20 12:19:00 +02:00
Fabian Grünbichler
3259c85ad1 apt: add proxmox-offline-mirror-helper package
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-15 09:08:32 +02:00
Thomas Lamprecht
84df915ef5 proxy: inline args in format strings
in most cases anyway. Allows for shorter code, which allows to drop
some line breaks.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-09-09 14:00:29 +02:00
Fabian Grünbichler
f3cfeb20a1 update proxmox-router to 1.3.0
no real change for PBS usage - the ApiHandler enum is marked
non_exhaustive now because it has extra values if the new (enabled by
default) "server" feature is enabled.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-07 14:17:12 +02:00
Fabian Grünbichler
951cbb96a2 update to proxmox-http 0.7
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-07 14:17:00 +02:00
Fabian Grünbichler
6512017f8b update to nom 7
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-08-22 10:16:05 +02:00
Wolfgang Bumiller
5574114a2a more stable clippy fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-08-17 09:22:32 +02:00
Fabian Grünbichler
75ecc27907 properly scope clippy allow
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-08-02 11:09:40 +02:00
Wolfgang Bumiller
968d3b4ff5 allow too_many_arguments in tape restore code
Some of them could easily be grouped in a kind of
RestoreWorker struct, but that'll still leave one bigger
function that's more annoying to change.
Let's just allow it for now.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-28 12:10:02 +02:00
Wolfgang Bumiller
5b6cb51dbc lower PullParameters to pub(crate), allow too_many_parameters
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-28 10:58:24 +02:00
Wolfgang Bumiller
223271e123 clippy: deal with some internal type complexity
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-28 10:53:38 +02:00
Wolfgang Bumiller
1654ab33b4 clippy auto-deref fixes
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-28 10:32:07 +02:00
Wolfgang Bumiller
c54aeedb8a clippy fixups for Default impls
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-28 10:30:02 +02:00
Wolfgang Bumiller
e589e2b790 ticket: box TfaChallenge, it's large
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-28 10:23:19 +02:00
Wolfgang Bumiller
e1ea913522 more low hanging clippy fruits
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-28 10:17:14 +02:00
Wolfgang Bumiller
10dac69391 clippy: deal with let bindings
In the auth code we rather #[allow] the binding, because in
this case we explicitly want to assert the type.

In fact, it would make more sense for clippy to not warn
about a unit type if the unit type is explicitly spelled
out.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-28 10:08:09 +02:00
Wolfgang Bumiller
8e70d421f2 more simpler clippy fixes
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-28 09:56:03 +02:00
Wolfgang Bumiller
92ead5d3f4 rrd: Entry type and clippy fixes
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-27 14:54:44 +02:00
Fabian Grünbichler
e1db06705e more clippy fixes and annotations
the remaining ones are:
- type complexity
- fns with many arguments
- new() without default()
- false positives for redundant closures (where closure returns a static
  value)
- expected vs actual length check without match/cmp

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-07-26 14:05:25 +02:00
Fabian Grünbichler
3be9106f1c clippy fixes
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-07-25 16:03:34 +02:00
Wolfgang Bumiller
8ab1d13140 tfa: allow modifying 'allow-subdomains'
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-25 13:39:41 +02:00
Fabian Grünbichler
0f281d6ca2 cargo fmt
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-07-25 10:03:44 +02:00
Fabian Grünbichler
68fda26787 adapt to proxmox-subscription changes
key location is now in a single place, missing key and no signature is
not fatal anymore.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-07-25 10:03:44 +02:00
Fabian Grünbichler
5616dca6b7 adapt to proxmox-apt change
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-07-25 10:03:44 +02:00
Fabian Grünbichler
da7b6af61c manager: add 'subscription set-offline-key' command
and disallow updating offline subscription keys via the regular
check/update code path.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-07-25 10:03:10 +02:00
Fabian Grünbichler
dd16e1dac8 extract proxmox-subscription crate
and add support for signed subscription keys.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-07-25 10:03:07 +02:00
Thomas Lamprecht
c2206e21e0 api daemons: periodically unpark a tokio thread to ensure progress
The underlying issue seems to be the case when the thread that runs
the IO driver is polling its own tasks, while that happens the IO
driver/poller won't run and thus work stealing won't happen, meaning
that idle and parked threads will keep being parked even if there's
pending work they could do.

A promising solution for tokio is proposed in its issue tracker [0],
but it wasn't yet implemented. So, as stop gap spawn a separate
thread that periodically spawns a no-op ready future in the runtime
which would unpark a worker in the aforementioned case and thus
should break the bogus idleness. Choose a 3s period for that without
any overly elaborate reasons, our main goal is to ensure we accept
incoming connections and 3s is well below a HTTP timeout and leaves
some room for high network latencies while not invoking to much
additional wakeups for systems that are really idling.

[0]: https://github.com/tokio-rs/tokio/issues/4730#issuecomment-1147975074

Link: https://github.com/tokio-rs/tokio/issues/4730
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-07-18 14:11:03 +02:00
Thomas Lamprecht
3f6a17b09f proxy: scheduler: only do a single round of time alignment and drop counter
not much value in waiting an extra minute, that doesn't really
guarantees better scheduling (as in, less impact on startup).

Dropping that also allows easily to drop the counter by just moving
the sleep to the beginning of the loop.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-07-14 18:20:44 +02:00
Thomas Lamprecht
9f7752f2b2 proxy: scheduler: code cleanup/bloat reduction
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-07-14 18:20:12 +02:00
Thomas Lamprecht
37f9b8cc09 proxy: scheduler: move fallback inside next_minute fn, drops result from return
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-07-14 17:59:24 +02:00
Stefan Sterz
5b5f18eb38 fix: api2: make tasks endpoint work with new prune job worker type
when prune jobs were refactored, their worker type was changed from
"prune" to "prunejob" this broke `check_job_store` and
`check_job_privs`

as reported in the forum: https://forum.proxmox.com/threads/problem-with-apirequest-on-pbs-2-2-3.112131/#post-483835

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2022-07-14 14:29:32 +02:00
Wolfgang Bumiller
91f4b11fa7 api2: optimize filtered snapshot listing
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-14 11:49:36 +02:00
Wolfgang Bumiller
5a2f7ea7cf api: wrap get_snapshots_count in spawn_blocking
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-13 14:02:38 +02:00
Wolfgang Bumiller
979b378460 api: wrap set_backup_owner in spawn_blocking
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-13 14:02:27 +02:00
Wolfgang Bumiller
67d7a59d68 api: wrap set_protection in spawn_blocking
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-13 14:02:27 +02:00
Wolfgang Bumiller
7beb27d428 api: wrap catalog in spawn_blocking
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-13 14:02:27 +02:00
Wolfgang Bumiller
af201d7a1d api: wrap delete_snapshot in spawn_blocking
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-13 14:02:13 +02:00
Wolfgang Bumiller
6cb674aaf5 api: wrap list_snapshot_files in spawn_blocking
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-13 14:02:13 +02:00
Wolfgang Bumiller
6f67dc1197 api: wrap delete_group in spawn_blocking
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-13 14:02:11 +02:00
Wolfgang Bumiller
a577d7d801 api: move 'list_snapshots' to a blocking thread
This is a stop-gap measure to prevent snapshot listing from
blocking the main async worker threads as it can potentially
do a *lot* of I/O.

Ideally we'll move to a proper streaming API, but this will
be an API break.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-12 11:09:53 +02:00
Wolfgang Bumiller
143ac7e69b wrap fs_info calls in spawn_blocking
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-11 10:41:16 +02:00
Dominik Csapak
36156038c9 tape: include used tapes in tape notification e-mails
by saving them in the pool-writer, and setting them in the
TapeBackupJobSummary

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-07-05 08:52:57 +02:00
Fabian Grünbichler
2b9cf927e3 move json_object_to_query to proxmox-http+http-helpers
it's used by the subscription code that will be extracted next.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-07-04 10:08:53 +02:00
Fabian Grünbichler
87b7fa0226 move to/write_canonical_json to proxmox-serde
as preparation for extracting subscription-related code into its own
crate.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-07-04 10:08:53 +02:00
Fabian Grünbichler
8e76134db9 cruft: remove tools::http
it's not used by anything.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-07-04 10:08:53 +02:00
Wolfgang Bumiller
b58dd1d413 rustfmt
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-30 10:18:32 +02:00
Dominik Csapak
4a2bb3e6cd api: tape/backup: improve behaviour for vanishing snapshots
when snapshots vanish during tape backup, we skip them. Until now,
we also warned with the error and failed the task at the end.

Since deleting snapshots during tape backup does not really interfere
with it, don't fail the whole task, and only add a log line that it
was skipped.

To differentiate from different errors (e.g. permission problems),
introduce a 'SnapshotBackupResult' which is returned by 'backup_snapshot'.

Also remove the 'pub' there since we don't want to leak the
SnapshotBackupResult type and it's not used anywhere outside this file.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-30 10:17:05 +02:00
Hannes Laimer
0608b36b30 replace print with log macro
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-21 10:43:19 +02:00
Hannes Laimer
955aea8ae8 bins: init cli logger
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-21 10:43:19 +02:00
Hannes Laimer
be2604109d disks: use builder pattern for querying disk usage
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-06-15 10:59:34 +02:00
Hannes Laimer
6a6ba4cdac api2: disks endpoint return partitions
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-06-15 10:59:21 +02:00
Wolfgang Bumiller
a1896bffb6 remove outdated comment
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-14 13:00:14 +02:00
Stefan Sterz
9e8daa1d10 fix #3867: server/api: send emails on certificate renewal failure
the superuser's email will be used to notify them that certificate
renewal has failed.

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-14 12:59:17 +02:00
Dominik Csapak
173b479b08 api: add metricserver endpoints
but in contrast to pve, we split the api by type of the section config,
since we cannot handle multiple types in the updater

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-13 10:01:05 +02:00
Dominik Csapak
4d397849b5 proxmox-backup-proxy: send metrics to configured metrics server
and keep the data as similar as possible to pve (tags/fields)

datastores get their own 'object' type and reside in the "blockstat"
measurement

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-13 10:01:05 +02:00
Dominik Csapak
759c4c87af backup-proxy: decouple stats gathering from rrd update
that way we can reuse the stats gathered

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-13 10:01:05 +02:00
Thomas Lamprecht
87005234c6 cargo fmt/format variable inlining
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-07 14:35:00 +02:00
Thomas Lamprecht
ce9b00e7e3 api: tape restore: warn if snapshot could not get parsed
unlikely, but cost is far too low to not do it.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-07 14:29:50 +02:00
Thomas Lamprecht
72b550a836 api: tape restore: code cleanup to reduce indentation level
No semantic change intended. IMO the interface of "both a datastore
and NS mapping must be present" is still a bit weird, at least in how
its used here to decide what to skip and what not, maybe we can
implement this in a more clear way (or maybe I'm just overlooking
something that makes it clearer as is).

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-07 14:12:02 +02:00
Thomas Lamprecht
1e319bcb06 access: fix typo in RPC env wrong-type error
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-07 08:53:24 +02:00
Dominik Csapak
1cc73a43a3 replace 'disk_usage' with 'fs_info' from proxmox-sys
Use the moved 'fs_info' helpers from the proxmox-sys crate (available
from there since proxmox-sys 0.3.0) as replacement for 'disk_usage'
in the workspace local tools crate and remove the latter as we do not
need it anymore.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 [ T: squashed in removal of now unused import and reworded commit
  message to include version availability info, among other things ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-07 08:42:22 +02:00
Thomas Lamprecht
5a2e6ccf77 api: tape restore: avoid throwing away ns mapping, use target_store instead
avoid assembling a hash mapping of namespaces only to not use it,
i.e., throw it away then anyway

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-05 16:59:57 +02:00
Thomas Lamprecht
f31e32a006 api: tape restore: some code cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-05 16:55:13 +02:00
Thomas Lamprecht
2ad96e1635 api: tape restore: split/rework datastore/namespace map implementation
The split out helpers will (partially) be used in later patches for
call sites where we only need parts of the info assembled here.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-05 16:47:27 +02:00
Thomas Lamprecht
7bc2e240b1 api: tape restore: use HumanByte for friendlier total/throughput reporting
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-05 10:45:13 +02:00