Commit Graph

2097 Commits

Author SHA1 Message Date
Wolfgang Bumiller
c01b08fea9 bump proxmox-compression to 0.1.1-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 09:35:31 +02:00
Wolfgang Bumiller
f8fe8f59a6 compression: limit ZstdEncoder constructors to usable ones
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 09:31:57 +02:00
Wolfgang Bumiller
79ac8d7344 compression: don't use tokio::main in doctest
because we have no rt feature enabled

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 09:31:26 +02:00
Wolfgang Bumiller
99add1733c compression: style changes
use where clauses where the parameter list is short enough
to become a single line

easier to read

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 09:31:10 +02:00
Wolfgang Bumiller
d4a09de520 compression: fmt
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 09:08:44 +02:00
Wolfgang Bumiller
fa5373c5c0 compression: clone_from_slice -> copy_from_slice
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 09:07:33 +02:00
Dominik Csapak
e7e4411f44 proxmox-compression: add streaming zstd encoder
similar to our DeflateEncoder, takes a Stream and implements it itself,
so that we can use it as an adapter for async api calls

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 09:02:54 +02:00
Dominik Csapak
d107d0b2eb proxmox-compression: add async tar builder
inspired by tar::Builder, but limited to the things we need and using
AsyncRead+AsyncWrite instead of the sync variants.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 09:02:51 +02:00
Wolfgang Bumiller
ac50b068de bump proxmox-schema to 1.3.1-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 08:20:35 +02:00
Wolfgang Bumiller
e734143380 bump proxmox-schema dependency to 1.3.1 for streaming attribute
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 08:20:27 +02:00
Wolfgang Bumiller
8291d9ed81 schema: bump api macro to 1.0.2 for the streaming attribute
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 08:19:00 +02:00
Wolfgang Bumiller
97c5095486 bump proxmox-router dependency to 1.2
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 08:17:08 +02:00
Dominik Csapak
922d61d276 adapt to the new ApiHandler variants
namely 'StreamingSync' and 'StreamingAsync'
in rest-server by using the new formatter function,
and in the debug binary by using 'to_value'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 08:13:40 +02:00
Dominik Csapak
cd4e485600 proxmox-rest-server: OutputFormatter: add new format_data_streaming method
that takes the data in form of a `Box<dyn SerializableReturn + Send>`
instead of a Value.

Implement it in json and extjs formatter, by starting a thread and
stream the serialized data via a `BufWriter<SenderWriter>` and use
the Receiver side as a stream for the response body.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 08:13:36 +02:00
Wolfgang Bumiller
bcf6abaa0d bump proxmox-api-macro to 1.0.2-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 14:27:11 +02:00
Wolfgang Bumiller
6ce3a73681 bump proxmox-router to 1.2.0-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 14:26:07 +02:00
Wolfgang Bumiller
930bb59d84 proxmox-router: depend on proxmox-async 0.4.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 14:24:27 +02:00
Wolfgang Bumiller
61d6541ce2 router: deduplicate some code
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 14:23:05 +02:00
Dominik Csapak
ca3b25869c proxmox-api-macro: add 'streaming' option
to generate the `Streaming` variants of the ApiHandler

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 14:23:05 +02:00
Dominik Csapak
f585722aad proxmox-router: add new ApiHandler variants for streaming serialization
they should behave like their normal variants, but return a
`Box<dyn SerializableReturn + Send>` instead of a value. This is useful
since we do not have to generate the `Value` in-memory, but can
stream the serialization to the client.

We cannot simply use a `Box<dyn serde::Serialize>`, because that trait
is not object-safe and thus cannot be used as a trait-object.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 14:23:05 +02:00
Dominik Csapak
2c9272945e promxox-router: add SerializableReturn Trait
this will be useful as a generic return type for api calls which
must implement Serialize.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 14:23:05 +02:00
Wolfgang Bumiller
27c8106d7b bump proxmox-async to 0.4.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 14:22:59 +02:00
Dominik Csapak
9471ba9969 proxmox-async: add SenderWriter helper
this wraps around a tokio Sender for Vec<u8>, but implements a blocking
write. We can use thas as an adapter for something that only takes a
writer, and can read from it asynchonously

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 14:05:14 +02:00
Thomas Lamprecht
9661defb0f tree wide: some stylistic clippy fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-11 08:14:28 +02:00
Thomas Lamprecht
4cdeee64dc sys: make acl constants rustfmt safe
there's not much better one can do here..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 17:39:31 +02:00
Thomas Lamprecht
0a651e00a9 sys: rust fmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 17:39:31 +02:00
Thomas Lamprecht
3a3dd296cc schema: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 12:42:09 +02:00
Thomas Lamprecht
ae9d6e255c lang: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 12:41:44 +02:00
Thomas Lamprecht
0eeb0dd17c http: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 12:41:21 +02:00
Thomas Lamprecht
05cad8926b router: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 12:40:39 +02:00
Thomas Lamprecht
0ec1c684ae shared memory: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 12:37:38 +02:00
Thomas Lamprecht
6f8173f67a tfa: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 12:34:41 +02:00
Thomas Lamprecht
4554034d32 time: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 12:34:04 +02:00
Thomas Lamprecht
800cf63a8a uuid: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 12:34:01 +02:00
Thomas Lamprecht
d3b387f1a7 update gitignore
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 12:29:03 +02:00
Thomas Lamprecht
6b06cc6839 rest server: log rotation: refactor and comment improvements
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-07 14:04:18 +02:00
Thomas Lamprecht
5eaab7864a rest server: log rotation: fix off-by-one for max_days
The entries in a file go from oldest end-time in the first time to
newest end-time in the last line. So, just because the first line is
older than the cut-off time, the remaining one doesn't necessarily
have to be old enough too. What we can know for sure that older than
the current checked rotations of the task archive are definitively up
for deletion.

Another possibility would be to check the last line, but as scanning
backwards is more expensive/complex to do while only being an actual
improvement in a very specific edge case (it's more likely to have a
mixed time-cutoff vs. task-log-file boundary than that those are
aligned)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-07 12:58:32 +02:00
Dominik Csapak
bd8fd62de4 rest-server: add option to rotate task logs by 'max_days' instead of 'max_files'
and use it with the configurable: 'task_log_max_days' of the node config

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-04-06 17:12:49 +02:00
Dominik Csapak
6c856eed5e rest-server: cleanup_old_tasks: improve error handling
by not bubbling up most errors, and continuing on. this avoids that we
stop cleaning up because e.g. one directory was missing.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-04-06 17:10:02 +02:00
Thomas Lamprecht
2bda552b55 rest server: rust fmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-06 16:55:39 +02:00
Thomas Lamprecht
0afe853119 bump version to 0.9.5-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-01 15:57:00 +02:00
Mira Limbeck
4aff0d7c95 fix Open ID with Azure as provider
Azure doesn't accept `Transfer-Encoding: chunked` on their token endpoint,
but with the switch to ureq we always send requests with this set.

Fix by switching to `Content-Length` in the header instead. ureq only
sets `Transfer-Encoding: chunked` when the body length is not known
beforehand, which is the case when using `send`. See
https://docs.rs/ureq/2.4.0/ureq/index.html#content-length-and-transfer-encoding

See https://forum.proxmox.com/threads/openid-401-with-azure-ad.105892/
for the issue.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2022-04-01 15:53:44 +02:00
Wolfgang Bumiller
f04eb949d1 tfa: serde tools: improve variance and dropck
`FoldSeqVisitor` doesn't actually own a `T` and therefore
cannot drop a `T`, we only use it via the `Fn(&mut Out, T)`,
so use `fn(T)` in the `PhantomData` to keep `T`
contravariant.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-22 12:31:54 +01:00
Thomas Lamprecht
1f47f7d3eb bump version to 0.9.4-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-03-22 11:37:59 +01:00
Thomas Lamprecht
23e6c398a2 http client: rust format and whitespace cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-03-22 10:57:21 +01:00
Mira Limbeck
e0535e56ad add http proxy support
ureq has support for a HTTP proxy, but no support for HTTPS proxy yet.
ureq doesn't query `all_proxy` and `ALL_PROXY` environment variables by
itself, the way curl does. So set the proxy in code if any of the above
environment variables are set.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2022-03-22 10:42:57 +01:00
Wolfgang Bumiller
6221d86c64 schema: add another test case
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-09 14:29:53 +01:00
Wolfgang Bumiller
d7283d5aeb schema: don't accept unterminated quoted strings
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-09 13:41:09 +01:00
Wolfgang Bumiller
cdf4326e43 schema: factor out string verify fn and improve docs
We'll need to name the type when we add more perl bindings.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-04 14:51:16 +01:00
Wolfgang Bumiller
c01fbafae0 bump proxmox-schema dep to 1.3
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-04 09:50:21 +01:00