Commit Graph

18 Commits

Author SHA1 Message Date
Wolfgang Bumiller
5b55ea797a compression: match style fixup
if the match arms are this far away from the actual `match`
keyword, this needs to be split up...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-06-15 10:37:59 +02:00
Dominik Csapak
af46b655d4 compression: zip_directory: improve error handling
when zipping a directory, our intention was to skip over files that
cannot be zipped (e.g. the file can't be read/vanished/etc.), so we
ignored errors and simply logged it.

but when 'add_entry' fails, we will never actually restore, since every
error there is fatal to the point that the zip cannot be finished thats
because we take the 'target' sink out of self, and only insert it again
after all writes succeeded. so if an error occurs in between 'target' is
not put into self again (and never will be) and the zip cannot be
finished (even if we would catch all those intermediate errors and
restore 'target', we don't know in which state the output was, so we're
unable to finish a valid zip)

to fix that, split the actual 'add_entry' part there out of the async
move block and treat its errors always as fatal

without this, we generate heaps of log lines even after an error
occurred, and can never recover

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2023-06-15 10:28:50 +02:00
Wolfgang Bumiller
818ac8e708 update zstd 0.6 -> 0.12 for bookworm
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-05-23 13:02:51 +02:00
Fabian Grünbichler
538578c558 clippy 1.65 fixes
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-05 11:17:37 +01:00
Wolfgang Bumiller
ff86aa5f8a compression: more cleanups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-05 13:23:14 +02:00
Wolfgang Bumiller
2d37cd92e0 compression: indentation cleanup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-05 12:17:04 +02:00
Wolfgang Bumiller
cc6e5d7372 compression: minor cleanup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-05 12:13:38 +02:00
Dominik Csapak
6e989a1c29 proxmox-compression: add 'tar_directory'
similar to 'zip_directory', this is intended to tar a local directory,
e.g. when we're in a restore vm.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-07-05 12:12:02 +02:00
Dominik Csapak
38db37dc5f proxmox-compression: make ZstdEncoder stream a bit more generic
by not requiring the 'anyhow::Error' type from the source, but only
that it implements 'Into<Error>'. This way, we can also accept a stream
that produces e.g. an io::Error

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-07-05 12:11:57 +02:00
Wolfgang Bumiller
30901c60f5 compression: clippy fixes
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-29 10:08:49 +02:00
Wolfgang Bumiller
f8fe8f59a6 compression: limit ZstdEncoder constructors to usable ones
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 09:31:57 +02:00
Wolfgang Bumiller
79ac8d7344 compression: don't use tokio::main in doctest
because we have no rt feature enabled

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 09:31:26 +02:00
Wolfgang Bumiller
99add1733c compression: style changes
use where clauses where the parameter list is short enough
to become a single line

easier to read

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 09:31:10 +02:00
Wolfgang Bumiller
d4a09de520 compression: fmt
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 09:08:44 +02:00
Wolfgang Bumiller
fa5373c5c0 compression: clone_from_slice -> copy_from_slice
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 09:07:33 +02:00
Dominik Csapak
e7e4411f44 proxmox-compression: add streaming zstd encoder
similar to our DeflateEncoder, takes a Stream and implements it itself,
so that we can use it as an adapter for async api calls

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 09:02:54 +02:00
Dominik Csapak
d107d0b2eb proxmox-compression: add async tar builder
inspired by tar::Builder, but limited to the things we need and using
AsyncRead+AsyncWrite instead of the sync variants.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 09:02:51 +02:00
Dominik Csapak
b8bf6a5c81 split out compression code into new crate 'proxmox-compression'
this removes quite a bit of dependecies of proxmox-async

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
[set proxmox-lang dep to 1.1]
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 14:10:53 +01:00