Commit Graph

2014 Commits

Author SHA1 Message Date
Wolfgang Bumiller
0c448080c8 bump proxmox-http to 0.2.0-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-17 11:25:42 +02:00
Wolfgang Bumiller
76c25a9612 add some more module level docs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-17 11:25:42 +02:00
Wolfgang Bumiller
c9b4a4f39b uri: drop anyhow::Error and improve docs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-17 11:25:42 +02:00
Wolfgang Bumiller
ca9dd7ec36 reorganize crate
it's a 'http' utility crate
* the extra 'http' module is redundant
* the module name 'helpers' as a public path doesn't say
  much about what it does

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-17 11:03:31 +02:00
Dietmar Maurer
3a161119c2 update proxmox/debian/control 2021-05-17 10:21:54 +02:00
Dietmar Maurer
47e6ae52ff bump proxmox version to 0.11.5-1 2021-05-17 10:07:28 +02:00
Fabian Grünbichler
37ae288efd http: update d/control
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 09:54:32 +02:00
Fabian Grünbichler
94378967e3 http: rustfmt
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 09:54:01 +02:00
Fabian Grünbichler
910949db01 http: make clippy happy
again.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 09:53:42 +02:00
Fabian Grünbichler
667bd8ab83 http: takeover simple HTTP client from proxmox_backup
adapted to use already moved helpers/code.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 09:53:18 +02:00
Fabian Grünbichler
80b423f30f http: takeover tools::http from proxmox_backup
the parts that were not already moved.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 09:53:04 +02:00
Fabian Grünbichler
7f56e0774c http: takeover ProxyConfig from proxmox_backup
adapted to use moved build_authority helper.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 09:52:29 +02:00
Fabian Grünbichler
f305be9583 http: takeover build_authority helper from proxmox_backup
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 09:45:35 +02:00
Fabian Grünbichler
6c5028512b http: takeover MaybeTlsStream from proxmox_backup
this is just a (rather HTTP specific) wrapper, so put it into a
'wrapper' module for now.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 09:45:04 +02:00
Fabian Grünbichler
caef9a3b20 proxmox: takeover socket helper from proxmox_backup
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 09:43:51 +02:00
Fabian Grünbichler
cbaa3b45bc http: make clippy happy
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 09:43:28 +02:00
Fabian Grünbichler
5153e68cab http: takeover websocket feature from proxmox
adapted: use statements for proxmox::*, use statements for doctests

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 09:43:19 +02:00
Fabian Grünbichler
7b7e1413ff meta: add empty proxmox-http sub-crate
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 09:42:16 +02:00
Fabian Grünbichler
ebeaf483ab proxmox: add missing +router -> futures dep
this is shadowed by the websocket feature being enabled by default.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 09:41:26 +02:00
Thomas Lamprecht
10d2453ed3 d/control: update deb-cargo generated file
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-14 12:17:09 +02:00
Thomas Lamprecht
91e7604ddc proxmox: bump version to 0.11.4-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-14 12:11:48 +02:00
Thomas Lamprecht
89d14b4729 buildsys: upload to both, buster and bullseye for now
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-14 12:08:50 +02:00
Thomas Lamprecht
92d7292920 d/control: update
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-14 11:43:02 +02:00
Thomas Lamprecht
40e232f4d6 drop bogus tests
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-14 11:36:52 +02:00
Dominik Csapak
02acce2dd8 proxmox/tools/email: adapt tests to changes to sendmail
with test1/3 we only tested the regex, which is now gone

transform the tests to valid emails, so we test this also
and rename them to know what they should test

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-14 10:55:46 +02:00
Dominik Csapak
a7f4002306 fix #3302: allow for more characters for email
by removing the regex check here, that is responsibility of the caller

this is ok since we pass the args directly and not via shell, so
command injection should not be possible

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-12 21:18:28 +02:00
Wolfgang Bumiller
fa8194e0bb refactor send_command
- refactor the combinators,
- make it take a `&T: Serialize` instead of a Value, and
  allow sending the raw string via `send_raw_command`.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-11 16:28:08 +02:00
Dietmar Maurer
fa3b5374ed api_type_macros.rs: disable failing doctest (set it to ignore instead of no_run) 2021-05-05 08:03:35 +02:00
Dietmar Maurer
64ac45ba91 api_type_macros.rs: disable failing doctest 2021-05-05 07:56:53 +02:00
Dietmar Maurer
5a3ac01b2e bump proxmox version to 0.11.3-1 2021-05-05 07:47:25 +02:00
Wolfgang Bumiller
86ad53256a add api_string_type macro
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-05 07:39:24 +02:00
Wolfgang Bumiller
e3a78d6354 simplify const_regex macro
The :vis macro matcher has been stable for a long time now.
Let's use it, and replace the recursion with a plus pattern.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-05 07:38:37 +02:00
Dietmar Maurer
9c786006df bump proxmox version to 0.11.2-1 2021-05-03 08:35:03 +02:00
Dominik Csapak
77b85f7dc1 proxmox/api/cli: add extract_output_format
just getting it is not enough, sometimes we want it to remove from
params (if possble) too

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-03 08:01:51 +02:00
Thomas Lamprecht
610b147ba7 server/rest: fix new type ambiguity
basically the same as commit 5fb15b781e
Will be required once we get to use a newer rustc, at least the
client build for archlinux was broken due to this.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-22 21:24:44 +02:00
Wolfgang Bumiller
4dda9b5865 proxmox: constify CreateOptions methods and add helpers
Adds convenience helpers `.owner_root()`.
Unfortunately for the group equivalent `nix` needs to have
const `Gid` constructors...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-04-21 12:00:57 +02:00
Wolfgang Bumiller
6eef946e20 websocket: remove AsyncReadExt
style wise, bounds should refer to AsyncRead instead

also we don't actually use any of the Ext methods

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-04-15 13:14:55 +02:00
Wolfgang Bumiller
31402407c4 bump proxmox to 0.11.1-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-04-14 12:04:35 +02:00
Wolfgang Bumiller
6729d41188 bump api-macro to 0.3.4-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-04-14 12:00:01 +02:00
Wolfgang Bumiller
cde2863a68 api-macro: fix Updatable path in generated derive attribute
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-04-14 11:58:32 +02:00
Wolfgang Bumiller
6fc95b8628 add proxmox::tools::serde::string_as_base64url_nopad
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-04-14 11:32:24 +02:00
Thomas Lamprecht
ac49fbd472 enable tape backup by default
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 12:31:56 +02:00
Stefan Reiter
c80251ecca server/rest: add ApiAuth trait to make user auth generic
This allows switching the base user identification/authentication method
in the rest server. Will initially be used for single file restore VMs,
where authentication is based on a ticket file, not the PBS user
backend (PAM/local).

To avoid putting generic types into the RestServer type for this, we
merge the two calls "extract_auth_data" and "check_auth" into a single
one, which can use whatever type it wants internally.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 13:57:57 +02:00
Thomas Lamprecht
1b2a34166d server: rest: collapse nested if for less indentation
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-07 17:57:46 +02:00
Thomas Lamprecht
27fca61f1b server: rest: switch from fastest to default deflate compression level
I made some comparision with bombardier[0], the one listed here are
30s looped requests with two concurrent clients:

[ static download of ext-all.js ]:
  lvl                              avg /   stdev  / max
 none        1.98 MiB  100 %    5.17ms /  1.30ms / 32.38ms
 fastest   813.14 KiB   42 %   20.53ms /  2.85ms / 58.71ms
 default   626.35 KiB   30 %   39.70ms /  3.98ms / 85.47ms

[ deterministic (pre-defined data), but real API call ]:
  lvl                              avg /   stdev  / max
 none      129.09 KiB  100 %    2.70ms / 471.58us / 26.93ms
 fastest    42.12 KiB   33 %    3.47ms / 606.46us / 32.42ms
 default    34.82 KiB   27 %    4.28ms / 737.99us / 33.75ms

The reduction is quite better with default, but it's also slower, but
only when testing over unconstrained network. For real world
scenarios where compression actually matters, e.g., when using a
spotty train connection, we will be faster again with better
compression.

A GPRS limited connection (Firefox developer console) requires the
following load (until the DOMContentLoaded event triggered) times:
  lvl        t      x faster
 none      9m 18.6s   x 1.0
 fastest   3m 20.0s   x 2.8
 default   2m 30.0s   x 3.7

So for worst case using sligthly more CPU time on the server has a
tremendous effect on the client load time.

Using a more realistical example and limiting for "Good 2G" gives:

 none      1m  1.8s   x 1.0
 fastest      22.6s   x 2.7
 default      16.6s   x 3.7

16s is somewhat OK, >1m just isn't...

So, use default level to ensure we get bearable load times on
clients, and if we want to improve transmission size AND speed then
we could always use a in-memory cache, only a few MiB would be
required for the compressable static files we server.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-07 17:57:42 +02:00
Dominik Csapak
692de3a903 server/rest: compress static files
compress them on the fly
and refactor the size limit for chunking files

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-07 12:34:31 +02:00
Dominik Csapak
376d90679c server/rest: compress api calls
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-07 12:34:31 +02:00
Dominik Csapak
c2a1c87039 server/rest: add helper to extract compression headers
for now we only extract 'deflate'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-07 12:34:31 +02:00
Dominik Csapak
a06d3892fe tools/compression: add DeflateEncoder and helpers
implements a deflate encoder that can compress anything that implements
AsyncRead + Unpin into a file with the helper 'compress'

if the inner type is a Stream, it implements Stream itself, this way
some streaming data can be streamed compressed

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-07 12:34:31 +02:00
Dominik Csapak
d28249c290 tools: add compression module
only contains a basic enum for the different compresssion methods

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-07 12:34:31 +02:00