Commit Graph

78 Commits

Author SHA1 Message Date
Maximiliano Sandoval
4d1c4ec829 http: teach the Client how to decode deflate content
The Backup Server can compress the content using deflate so we teach the
client how to decode it.

If a request is sent with the `Accept-Encoding` [2] header set to
`deflate`, and the response's `Content-Encoding` [1] header is equal to
`deflate` we wrap the Body stream with a stream that can decode `zlib`
on the run.

Note that from the `Accept-Encoding` docs [2], the `deflate` encoding is
actually `zlib`.

This can be also tested against
http://eu.httpbin.org/#/Response_formats/get_deflate by adding the
following test:

```rust
    #[tokio::test]
    async fn test_client() {
        let client = Client::new();
        let headers = HashMap::from([(
            hyper::header::ACCEPT_ENCODING.to_string(),
            "deflate".to_string(),
        )]);
        let response = client
            .get_string("https://eu.httpbin.org/deflate", Some(&headers))
            .await;
        assert!(response.is_ok());
    }
```

at `proxmox-http/src/client/simple.rs` and running

```
cargo test --features=client,client-trait
```

[1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding
[2] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding

Suggested-by: Lukas Wagner <l.wagner@proxmox.com>
Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Reviewed-by: Max Carrara <m.carrara@proxmox.com>
Tested-by: Max Carrara <m.carrara@proxmox.com>
2024-07-10 11:37:36 +02:00
Wolfgang Bumiller
0652d81977 tree-wide: enable doc_cfg and doc_auto_cfg for docs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-07-02 11:59:53 +02:00
Maximiliano Sandoval
6b1e4b83bb http: remove unnecessary cast
Fixes the clippy warning:

warning: casting to the same type is unnecessary (`usize` -> `usize`)
   --> proxmox-http/src/websocket/mod.rs:446:40
    |
446 |             mask.copy_from_slice(&data[mask_offset as usize..payload_offset as usize]);
    |                                        ^^^^^^^^^^^^^^^^^^^^ help: try: `mask_offset`
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_cast
    = note: `#[warn(clippy::unnecessary_cast)]` on by default

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-06-28 10:22:58 +02:00
Maximiliano Sandoval
b006e66361 http: remove redundant redefinition of binding
Fixes the clippy error:

error: redundant redefinition of a binding `data`
   --> proxmox-http/src/websocket/mod.rs:375:9
    |
375 |         let data = data;
    |         ^^^^^^^^^^^^^^^^
    |
help: `data` is initially defined here
   --> proxmox-http/src/websocket/mod.rs:369:27
    |
369 |     pub fn try_from_bytes(data: &[u8]) -> Result<Option<FrameHeader>, WebSocketError> {
    |                           ^^^^
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#redundant_locals
    = note: `#[deny(clippy::redundant_locals)]` on by default

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-06-28 10:22:58 +02:00
Wolfgang Bumiller
9be9d4b6ab http: support ALPN negotiated http2
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-03-28 11:19:56 +01:00
Wolfgang Bumiller
fe8b11eeec http: concat! user agent instead of format!
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-01-15 11:17:57 +01:00
Lukas Wagner
3ac6f2d9c0 http: rate limited stream: fix typo in rustdoc comment
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-11-08 09:56:26 +01:00
Fabian Grünbichler
39453abb8f http: sync: drop unused &self parameter
these are just internal helpers, changing their signature is fine.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-03-07 09:30:13 +01:00
Fabian Grünbichler
6a1be173a6 http: sync: derive default user-agent from crate version
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-03-07 09:30:13 +01:00
Fabian Grünbichler
5ba9d9b2c2 http: sync: remove redundant calls for setting User-Agent
the requests are all created via the agent that already contains the user
agent, so this internal helper isn't needed anymore.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-03-07 09:30:13 +01:00
Fabian Grünbichler
d69fee254a http: sync: set user-agent via ureq agent
this allows us to slim down our code, and once
https://github.com/algesten/ureq/pull/597 is merged upstream (and/or we update
to a version containing the fix) it also means the custom user agent is used
for requests to the proxy host, if one is configured.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-03-07 09:30:13 +01:00
Wolfgang Bumiller
d7ed04f8e5 http: add RateLimitedStream::inner, drop peer_addr
instead of implementing 'peer_addr' specifically for
RateLimitedStream<tokio::net::TcpStream>, just provide
.inner() and .inner_mut() so the user can reach the inner
stream directly.

This way we can drop the tokio/net feature as well

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-03-02 16:14:04 +01:00
Wolfgang Bumiller
10a3ab222b http: move rate-limiting out of client feature
this can now be used separately

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-03-02 16:14:04 +01:00
Fabian Grünbichler
08a6d56eae http: client_trait: make request body generic
like the response body, instead of hard-coding Read.
2022-09-07 09:25:47 +02:00
Fabian Grünbichler
891dcfda2f http: add extra_headers to post
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-07 09:17:45 +02:00
Fabian Grünbichler
ec77785578 http: sync: add HttpClient for Box<dyn Read>
for use cases where the full request body is not available from the
start, or the response doesn't need to be fully read in one go.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-07 09:17:45 +02:00
Fabian Grünbichler
00f5eca155 http: make post() take Read, not &str
for more flexibility.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-07 09:17:45 +02:00
Fabian Grünbichler
7863eff2a5 http: fix typo
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-07 09:17:45 +02:00
Fabian Grünbichler
da49b98d15 http: rename SimpleHttp to Client
so we have proxmox_http::client::Client for the async, hyper-based
client and proxmox_http::client::sync::Client for the sync, ureq-based
one.

this is a breaking change.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-07 09:17:45 +02:00
Fabian Grünbichler
7ffb895062 http: add "raw" sync client
and switch String one over to use it.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-07 09:17:45 +02:00
Fabian Grünbichler
9c444d7a94 http: add ureq-based sync client
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-07 09:17:45 +02:00
Fabian Grünbichler
f429fcb592 http: extend HttpClient trait
to allow get requests with extra headers (such as `Authorization`) and a
generic `request` fn to increase flexibility even more.

this is a breaking change.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-07 09:17:45 +02:00
Fabian Grünbichler
ab5d5b39f6 http: move SimpleHttpOptions to http-helpers feature
and rename it to HttpOptions, since it's not specific to the "Simple"
client at all.

this is a breaking change.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-07 09:17:45 +02:00
Wolfgang Bumiller
28ee8bc6d0 http: clippy fixes
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-27 10:52:50 +02:00
Mira Limbeck
a8a838754d http: fix proxy authorization header to include type
and encode the username:password string as base64 [0]. This fixes the
error 407 issue when using proxy authentication [1].

[0] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Proxy-Authorization
[1] https://forum.proxmox.com/threads/checking-the-subscription-behind-a-proxy-fails.112063/

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2022-07-21 12:54:59 +02:00
Wolfgang Bumiller
94d388b988 http: clippy fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-20 13:31:58 +02:00
Fabian Grünbichler
bd1f9f103e http: implement HttpClient for SimpleHttp
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-06-30 12:42:17 +02:00
Fabian Grünbichler
3c0486be50 http: add HttpClient trait
gated behind feature "client-trait"

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-06-30 12:42:12 +02:00
Fabian Grünbichler
94456ee4b1 http: move TLS helper to client feature
it's only used there and pulls in hyper and tokio, which we don't
want/need in http-helpers.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-06-29 10:32:44 +02:00
Fabian Grünbichler
210d4fdb68 http: take over json_object_to_query 2022-06-29 10:32:44 +02:00
Wolfgang Bumiller
0d30720907 http: clippy fixes
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-29 10:14:33 +02:00
Thomas Lamprecht
0eeb0dd17c http: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 12:41:21 +02:00
Dominik Csapak
d4b4115400 move io error helpers to proxmox-lang
this removes proxmox_sys as a dependecy for proxmox-async

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 13:35:14 +01:00
Wolfgang Bumiller
70142e607d proxmox-http: drop 'mut' on specialized request methods
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-11 08:56:34 +01:00
Fabian Grünbichler
ca563a8cfd misc clippy fixes
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-02-08 14:28:44 +01:00
Thomas Lamprecht
5cc4ce3b4d http: websocket: code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-04 17:16:36 +01:00
Thomas Lamprecht
1edb52411e http: websocket: drop Text frame auto-detection from docs
was forgotten in commit 232d87531e when
dropping the bogus frame auto detection

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-04 17:15:49 +01:00
Thomas Lamprecht
42b6f4331f http: websocket: avoid modulo for power of 2
even for the small cases this can matter

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-04 17:12:07 +01:00
Thomas Lamprecht
425b52586e http: websocket: rustfmt and small cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-04 17:05:45 +01:00
Thomas Lamprecht
170564dd77 http: websocket: doc wording cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-04 17:04:50 +01:00
Dominik Csapak
4826ff99d8 proxmox-http: websocket: fix comment about callback
this was once a callback in an early version, but it changed to a
channel, but the comment was not updated

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-02-04 16:10:31 +01:00
Wolfgang Bumiller
c609a58086 doc fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-02 14:14:09 +01:00
Wolfgang Bumiller
179515c5b2 http: clippy fixes
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-12-07 11:45:52 +01:00
Dietmar Maurer
82245339b8 use new proxmox-sys 0.2.0 for all crates
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-24 10:00:38 +01:00
Dietmar Maurer
81e959548b proxmox-http: impl RateLimiterVec
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-15 11:01:03 +01:00
Dietmar Maurer
b9a1d62e47 proxmox-http: RateLimit - remove average_rate
Instead, add a method to return overall traffic.
2021-11-14 08:15:42 +01:00
Dietmar Maurer
13276cc619 proxmox-http: use repr(C) for RateLimiter
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-13 17:38:18 +01:00
Dietmar Maurer
8734d0c2f9 proxmox-http: use SharedRateLimit trait object for RateLimitedStream
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-13 17:38:10 +01:00
Dietmar Maurer
937d1a6095 proxmox-http: define a RateLimit trait
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-13 17:38:00 +01:00
Dietmar Maurer
2f221df863 RateLimiter: add update_rate method 2021-11-10 09:51:08 +01:00