Commit Graph

2405 Commits

Author SHA1 Message Date
Thomas Lamprecht
27fca61f1b server: rest: switch from fastest to default deflate compression level
I made some comparision with bombardier[0], the one listed here are
30s looped requests with two concurrent clients:

[ static download of ext-all.js ]:
  lvl                              avg /   stdev  / max
 none        1.98 MiB  100 %    5.17ms /  1.30ms / 32.38ms
 fastest   813.14 KiB   42 %   20.53ms /  2.85ms / 58.71ms
 default   626.35 KiB   30 %   39.70ms /  3.98ms / 85.47ms

[ deterministic (pre-defined data), but real API call ]:
  lvl                              avg /   stdev  / max
 none      129.09 KiB  100 %    2.70ms / 471.58us / 26.93ms
 fastest    42.12 KiB   33 %    3.47ms / 606.46us / 32.42ms
 default    34.82 KiB   27 %    4.28ms / 737.99us / 33.75ms

The reduction is quite better with default, but it's also slower, but
only when testing over unconstrained network. For real world
scenarios where compression actually matters, e.g., when using a
spotty train connection, we will be faster again with better
compression.

A GPRS limited connection (Firefox developer console) requires the
following load (until the DOMContentLoaded event triggered) times:
  lvl        t      x faster
 none      9m 18.6s   x 1.0
 fastest   3m 20.0s   x 2.8
 default   2m 30.0s   x 3.7

So for worst case using sligthly more CPU time on the server has a
tremendous effect on the client load time.

Using a more realistical example and limiting for "Good 2G" gives:

 none      1m  1.8s   x 1.0
 fastest      22.6s   x 2.7
 default      16.6s   x 3.7

16s is somewhat OK, >1m just isn't...

So, use default level to ensure we get bearable load times on
clients, and if we want to improve transmission size AND speed then
we could always use a in-memory cache, only a few MiB would be
required for the compressable static files we server.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-07 17:57:42 +02:00
Dominik Csapak
692de3a903 server/rest: compress static files
compress them on the fly
and refactor the size limit for chunking files

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-07 12:34:31 +02:00
Dominik Csapak
376d90679c server/rest: compress api calls
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-07 12:34:31 +02:00
Dominik Csapak
c2a1c87039 server/rest: add helper to extract compression headers
for now we only extract 'deflate'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-07 12:34:31 +02:00
Dominik Csapak
a06d3892fe tools/compression: add DeflateEncoder and helpers
implements a deflate encoder that can compress anything that implements
AsyncRead + Unpin into a file with the helper 'compress'

if the inner type is a Stream, it implements Stream itself, this way
some streaming data can be streamed compressed

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-07 12:34:31 +02:00
Dominik Csapak
d28249c290 tools: add compression module
only contains a basic enum for the different compresssion methods

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-07 12:34:31 +02:00
Wolfgang Bumiller
c735b4bde0 api: enable Updater for usize, isize, f32, f64, char
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-04-07 10:48:46 +02:00
Thomas Lamprecht
7334107fae server/rest: drop now unused imports
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-01 11:53:13 +02:00
Stefan Reiter
25a14e8eb6 server/rest: extract auth to seperate module
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-01 11:26:28 +02:00
Thomas Lamprecht
5fb15b781e server/rest: fix type ambiguity
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-29 12:02:30 +02:00
Thomas Lamprecht
45a652c26f server/rest: rust format
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-29 08:17:26 +02:00
Wolfgang Bumiller
248335df36 api-macro: fix dev-dep versions
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-24 11:20:23 +01:00
Wolfgang Bumiller
afc59f6d15 add more of the ACME workflow to 'Account'
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-18 11:36:42 +01:00
Wolfgang Bumiller
02ecbb499c bump version to 0.1.4-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-17 15:28:55 +01:00
Wolfgang Bumiller
5aee14ac00 collect extra account fields in AccountData
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-17 15:20:50 +01:00
Wolfgang Bumiller
d369de8636 bump version to 0.1.3-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-17 13:35:49 +01:00
Wolfgang Bumiller
937a99a2e1 fix ec signature padding
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-17 13:32:57 +01:00
Wolfgang Bumiller
549b52cf68 don't serialize 'null' for 'expires' when ordering
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-17 12:24:21 +01:00
Wolfgang Bumiller
15fae62c15 bump version to 0.1.2-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-12 15:43:40 +01:00
Wolfgang Bumiller
ca0f57290f explicitly pass Content-Length header
pebble refuses to cooperate without it

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-10 12:01:00 +01:00
Wolfgang Bumiller
c9f137a093 bump version to 0.1.1-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-09 13:23:27 +01:00
Wolfgang Bumiller
cbfeb58ce4 make AccountData fields public
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-09 13:22:52 +01:00
Wolfgang Bumiller
aa23068293 import
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-09 13:07:59 +01:00
Dietmar Maurer
9dbc5f47ca ui: enable experimental tape UI if tape.cfg exists 2021-03-03 09:02:02 +01:00
Wolfgang Bumiller
1fce0ff41d bump proxmox to 0.11.0
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-25 09:21:14 +01:00
Wolfgang Bumiller
53146006b5 fmt
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-25 09:16:44 +01:00
Wolfgang Bumiller
8ce46932d6 use Send+Sync on dyn ObjectSchemaType
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-25 09:16:38 +01:00
Wolfgang Bumiller
42eb380917 support AllOfSchemas in section_config
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-24 14:48:21 +01:00
Wolfgang Bumiller
e2975598ad get rid of associated type in ObjectSchemaType trait
The AllOfProperties Iterator is capable of also being the
Iterator for the regular ObjectSchemas by simply filling it
like it was the "last" object type in an AllOf.

This makes the ObjectSchemaType trait more usable later.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-24 14:43:21 +01:00
Wolfgang Bumiller
adcbf7cab8 AllOfSchema support in parse_property_string
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-24 13:40:50 +01:00
Wolfgang Bumiller
12a6846a03 fix AllOfSchema::lookup to allow nesting
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-24 13:34:06 +01:00
Wolfgang Bumiller
bd4f9a313b proxmox: bump api-macro dep to 0.3.3
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-24 09:38:32 +01:00
Wolfgang Bumiller
1f6a2a66c9 bump api-macro to 0.3.3-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-24 09:37:30 +01:00
Wolfgang Bumiller
857b8ab2b9 Updater: take serde renames into account
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-24 09:21:52 +01:00
Wolfgang Bumiller
d3636d45d9 bump proxmox to 0.10.7
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-23 14:45:14 +01:00
Wolfgang Bumiller
e2d9f676a4 rustfmt
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-23 14:35:12 +01:00
Wolfgang Bumiller
5c120f6abe proxmox::tools: rustfmt
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-23 14:34:48 +01:00
Wolfgang Bumiller
866ec28116 proxmox::test: rustfmt
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-23 14:34:22 +01:00
Wolfgang Bumiller
944ea1e8b7 remove u2f example for now
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-23 14:33:46 +01:00
Wolfgang Bumiller
69a51c9af0 minor cleanup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-23 14:33:08 +01:00
Wolfgang Bumiller
0d68daab86 proxmox::test::io: properly forward flush
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-23 14:31:02 +01:00
Dominik Csapak
e0dad25af4 proxmox: add sparse_copy(_async) to tools::io
this is able to seek the target instead of writing zeroes, which
generates sparse files where supported

also add tests for it

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-23 14:28:54 +01:00
Dominik Csapak
840e5fda89 proxmox: add test/{io, task} modules
contains:
* AsyncBlocking{Reader,Writer} for dummy async code
  by wrapping a 'standard reader/writer'
* poll_result_once for pulling a future once (copied from pxar)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-23 14:28:50 +01:00
Dietmar Maurer
be03039ba9 bump proxmox version to 0.10.6-1 2021-02-21 15:49:40 +01:00
Dietmar Maurer
69ab6bd69d make get_property_string_type_text public
We need that to generate nice docs.
2021-02-21 15:44:13 +01:00
Stefan Reiter
8635b165b2 rest: implement tower service for UnixStream
This allows anything that can be represented as a UnixStream to be used
as transport for an API server (e.g. virtio sockets).

A tower service expects an IP address as it's peer, which we can't
reliably provide for unix socket based transports, so just fake one.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-17 07:50:35 +01:00
Dietmar Maurer
7212fd3290 bump proxmox version to 0.10.5 2021-02-11 10:46:11 +01:00
Dietmar Maurer
133209c14a implement dump_section_config 2021-02-11 10:44:04 +01:00
Dietmar Maurer
67c7699b23 rename dump_api_parameters to dump_properties
And do not dump object description automatically.
2021-02-11 10:32:39 +01:00
Wolfgang Bumiller
4e375ba95c bump api-macro to 0.3.2-1
This got lost in a rebase...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-11 09:00:44 +01:00