Commit Graph

2744 Commits

Author SHA1 Message Date
Thomas Lamprecht
67769843da tools: file logger: avoid some possible unwraps in log method
writing to a file can explode quite easily.
time formatting to rfc3339 should be more robust, but it has a few
conditions where it could fail, so catch that too (and only really
do it if required).

The writes to stdout are left as is, it normally is redirected to
journal which is in memory, and thus breaks later than most stuff,
and at that point we probably do not care anymore anyway.

It could make sense to actually return a result here..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-02 19:32:22 +01:00
Wolfgang Bumiller
ad8fc652f6 make u2f registration challenge Deserialize
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-02 13:07:38 +01:00
Wolfgang Bumiller
a6f8f84632 make u2f attestation certificate optional
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-02 12:49:47 +01:00
Thomas Lamprecht
66b9170dda file logger: allow reopening file
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-02 10:03:10 +01:00
Wolfgang Bumiller
31b7620f20 bump proxmox to 0.7.0-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-30 15:15:32 +01:00
Wolfgang Bumiller
6255336f04 totp: rename step to period
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-30 15:13:42 +01:00
Thomas Lamprecht
95ac6d87c3 server/rest: accept also = as token separator
Like we do in Proxmox VE

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-30 13:34:26 +01:00
Thomas Lamprecht
7b2f5672c5 server/rest: user constants for HTTP headers
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-30 13:33:36 +01:00
Fabian Grünbichler
e57660691d api tokens: add authorization method
and properly decode secret (which is a no-op with the current scheme).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-10-30 13:15:14 +01:00
Fabian Grünbichler
971dc0e6bc REST: extract and handle API tokens
and refactor handling of headers in the REST server while we're at it.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-10-29 15:14:27 +01:00
Fabian Grünbichler
ed512bc26f replace Userid with Authid
in most generic places. this is accompanied by a change in
RpcEnvironment to purposefully break existing call sites.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-10-29 15:11:39 +01:00
Wolfgang Bumiller
0cd88fd832 bump proxmox to 0.6.0-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-29 15:04:07 +01:00
Wolfgang Bumiller
8cbf9cb7c8 add proxmox::tools::tfa
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-29 15:01:26 +01:00
Wolfgang Bumiller
05749ab419 test fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-29 14:51:22 +01:00
Fabian Grünbichler
51f5acea22 rpcenv: rename user to auth_id
since it does no longer store just a userid, but potentially an API
token identifier as well

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-29 13:59:24 +01:00
Thomas Lamprecht
7d34e6f3a4 api macro: reuse generated default const for "unwrap_or"
Instead of setting a default value to a const and inside an
.unwrap_or_else closure, lets set it only to the const and reuse that
later in .unwrap_or

To achieve that we move the "unrwap_or" code for param plumbing code generation
a bit later so that we have easy access to the generated const name.
As all this code is related to optional/default-value stuff it does read still
relatively OK with that change, IMO.

This has the advantage of not getting a warning like:

>  warning: constant is never used: `API_METHOD_EXAMPLE_FOO_PARAM_DEFAULT_FORCE`
>   --> src/api2/node/foo.rs
>    |
> XY |             force: {
>    |             ^^^^^
>    = note: `#[warn(dead_code)]` on by default

When one has a API endpoint like:

> #[api(
>     input: {
>         properties: {
>             force: {
>                 type: bool,
>                 optional: true,
>                 default: false,
>             },
>         },
>     },
>     ...
> )]
> /// Example
> fn example_foo(force: bool) -> Result<(), Error> {
>     if force {
>         // do something
>     }
>     Ok(())
> }

It effectively changes the output for optional parameters with a default set
and no Option<T> from

> let p = p.unwrap_or_else(|| #default_value);

to

> let p = p.unwrap_or(#const_name_for_default);

where the "#const_name_for_default" is a pub const with value
"#default_value"

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-27 09:17:12 +01:00
Thomas Lamprecht
7575f890ed tools fs: fix comment typo and strange glyph
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-20 18:46:50 +02:00
Fabian Grünbichler
9af79677b2 REST: rename token to csrf_token
for easier differentiation with (future) api_token

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-10-19 14:02:19 +02:00
Wolfgang Bumiller
fe652c9263 bump proxmox version to 0.5.0-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-19 12:32:51 +02:00
Wolfgang Bumiller
9cd97f9851 proxmox::tools::fs: remove deprecated functions
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-19 12:32:37 +02:00
Fabian Grünbichler
32bb67cf60 build: bump nix dependency
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-10-19 12:12:33 +02:00
Fabian Grünbichler
daab5a9faf proxmox: bump nix dependency
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-10-19 12:11:50 +02:00
Thomas Lamprecht
f10a722eda file logger: add option to make the backup user the log file owner
and use that in ApiConfig to avoid that it is owned by root if the
proxmox-backup-api process creates it first.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-19 10:37:26 +02:00
Thomas Lamprecht
98f91a4d24 server/rest: also log user agent
allows easily to see if a request is from a browser or a proxmox-backup-client
CLI

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-16 11:23:49 +02:00
Thomas Lamprecht
287659dcda server/rest: implement request access log
reuse the FileLogger module in append mode.
As it implements write, which is not thread safe (mutable self) and
we use it in a async context we need to serialize access using a
mutex.

Try to use the same format we do in pveproxy, namely the one which is
also used in apache or nginx by default.

Use the response extensions to pass up the userid, if we extract it
from a ticket.

The privileged and unprivileged dameons log both to the same file, to
have a unified view, and avoiding the need to handle more log files.
We avoid extra intra-process locking by reusing the fact that a write
smaller than PIPE_BUF (4k on linux) is atomic for files opened with
the 'O_APPEND' flag. For now the logged request path is not yet
guaranteed to be smaller than that, this will be improved in a future
patch.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-16 11:23:49 +02:00
Thomas Lamprecht
1d07c62e74 tools file logger: fix example and comments
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-16 11:16:29 +02:00
Thomas Lamprecht
3b2aaa1878 tools: file logger: use option struct to control behavior
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-16 10:48:36 +02:00
Thomas Lamprecht
43ab000e92 server: rest: also log the query part of URL
As it is part of the request and we do so in our other products

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-16 10:41:05 +02:00
Thomas Lamprecht
da8439b25e server: rest: implement max URI path and query length request limits
Add a generous limit now and return the correct error (414 URI Too
Long). Otherwise we could to pretty larger GET requests, 64 KiB and
possible bigger (at 64 KiB my simple curl test failed due to
shell/curl limitations).

For now allow a 3072 characters as combined length of URI path and
query.

This is conform with the HTTP/1.1 RFCs (e.g., RFC 7231, 6.5.12 and
RFC 2616, 3.2.1) which do not specify any limits, upper or lower, but
require that all server accessible resources mus be reachable without
getting 414, which is normally fulfilled as we have various length
limits for stuff which could be in an URI, in place, e.g.:
 * user id: max. 64 chars
 * datastore: max. 32 chars

The only known problematic API endpoint is the catalog one, used in
the GUI's pxar file browser:
GET /api2/json/admin/datastore/<id>/catalog?..&filepath=<path>

The <path> is the encoded archive path, and can be arbitrary long.

But, this is a flawed design, as even without this new limit one can
easily generate archives which cannot be browsed anymore, as hyper
only accepts requests with max. 64 KiB in the URI.
So rather, we should move that to a GET-as-POST call, which has no
such limitations (and would not need to base32 encode the path).

Note: This change was inspired by adding a request access log, which
profits from such limits as we can then rely on certain atomicity
guarantees when writing requests to the log.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-16 10:40:39 +02:00
Thomas Lamprecht
e4bece494c server/rest: forward real client IP on proxied request
needs new proxmox dependency to get the RpcEnvironment changes,
adding client_ip getter and setter.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-16 10:36:54 +02:00
Dietmar Maurer
f3c10cc9df bump proxmox version to 0.4.4-1 2020-10-16 10:30:22 +02:00
Thomas Lamprecht
e30fd48080 server: rest: refactor code to avoid multiple log_response calls
The 'Ok::<_, Self::Error>(res)' type annotation was from a time where
we could not use async, and had a combinator here which needed
explicity type information. We switched over to async in commit
df52ba5e45 and, as the type annotation
is already included in the Future type, we can safely drop it.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-15 13:58:47 +02:00
Thomas Lamprecht
009737844c code cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-15 13:58:47 +02:00
Thomas Lamprecht
88586e9343 api: RPC environment: add client IP getter/setter to trait
This is similar to what we have in PVE and PMG now. Will be used to
set the real client IP for proxied connections.

with a dummy implementation, which avoids the need to implement it
for the CLI or Backup environments, which do not have or care for a
client IP

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-14 13:53:04 +02:00
Wolfgang Bumiller
4d8bd987a4 clippy fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-14 11:20:07 +02:00
Wolfgang Bumiller
e5456f873d clippy fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-14 11:13:20 +02:00
Thomas Lamprecht
649ff6f67f server/REST: check auth: code cleanup, better variable names
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-12 18:39:45 +02:00
Thomas Lamprecht
1fa5b1108d server/REST: make handle_request private
it's not used anywhere else, so do not suggest so

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-12 18:39:00 +02:00
Thomas Lamprecht
4becd202c5 server: get index: make content-type non mutable
feels more idiomatic

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-12 13:36:45 +02:00
Thomas Lamprecht
a049949d14 server/rest: code cleanup: use async
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-12 13:36:45 +02:00
Fabian Grünbichler
16f05d6649 REST: don't print CSRF token
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-10-08 15:57:22 +02:00
Dietmar Maurer
75bc02d9db bump proxmox version to 0.4.3-1 2020-10-08 09:00:22 +02:00
Thomas Lamprecht
82172720a4 tools: change constnamemap to a more automatic constnamedbitmap
We only used this for the privileges for now, and there it's a
nuisance to alter all bit definitions manually if something is added.

This change makes it count the bits up automatically.

Rename the macro to indicate that this is not a generic name map but
a more specific named bit mapping.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-08 08:54:26 +02:00
Wolfgang Bumiller
19dc729b7b formatting fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-05 12:04:22 +02:00
Wolfgang Bumiller
9fa9c76a9c improve error messages in parse_rfc3339
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-05 09:16:51 +02:00
Thomas Lamprecht
d2b63c504a REST server: avoid hard coding world readable API endpoints
while we probably do not add much more to them, it still looks ugly.

If this was made so that adding a World readable API call is "hard"
and not done by accident, it rather should be done as a test on build
time. But, IMO, the API permission schema definitions are easy to
review, and not often changed/added - so any wrong World readable API
call will normally still caught.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-05 08:29:43 +02:00
Thomas Lamprecht
c6ab240333 rest server: cleanup use statements
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-02 13:04:08 +02:00
Wolfgang Bumiller
307cb04795 big formatting cleanup
had to be done, sorry not sorry

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-02 10:00:04 +02:00
Wolfgang Bumiller
ef5c719150 formatting fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-02 09:59:52 +02:00
Stefan Reiter
6eff0b289e rrd: fix integer underflow
Causes a panic if last_update is smaller than RRD_DATA_ENTRIES*reso,
which (I believe) can happen when inserting the first value for a DB.

Clamp the value to 0 in that case.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-10-01 14:30:32 +02:00