Commit Graph

3472 Commits

Author SHA1 Message Date
Fiona Ebner
e97f41e290 section config: add test for array schema
where duplicate keys are allowed.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-12-12 14:03:23 +01:00
Wolfgang Bumiller
aaf4b72839 deps: bump api-macro to current version
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-12-12 11:51:08 +01:00
Wolfgang Bumiller
7bc85c05c9 bump proxmox-api-macro to 1.0.4-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-12-12 11:34:18 +01:00
Wolfgang Bumiller
38a60d3acb api: support #[default] attribute
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-12-12 11:34:18 +01:00
Fabian Grünbichler
0719e1db1c update/extend README.rst
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-12 11:05:30 +01:00
Fabian Grünbichler
ee8419cf2d workspace: switch remaining dependencies
while these are (currently) only used by a single member each, having *all*
dependency versions specified in the top level Cargo.toml only makes the whole
process of managing them less error-prone.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-12 11:05:30 +01:00
Fabian Grünbichler
1380182538 update d/control
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-12 11:05:30 +01:00
Wolfgang Bumiller
2610208794 io: add boxed module for boxed bytes like vec::zeroed...
- proxmox_io::boxed::uninitialized(len) -> Box<[u8]>
  same as vec::uninitialized, but as a box

- proxmox_io::boxed::zeroed(len) -> Box<[u8]>
  same as vec::zeroed, but as a box

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-12-12 11:00:22 +01:00
Wolfgang Bumiller
a7d84effc5 io: deny unsafe_op_in_unsafe_fn
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-12-12 11:00:22 +01:00
Fabian Grünbichler
8316fd3899 more workspace dependencies
regex was missed in the first pass, and two intra-workspace dev-dependencies as
well.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-12 09:30:30 +01:00
Fabian Grünbichler
10f56e9358 sort dependencies
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-12 09:08:56 +01:00
Fabian Grünbichler
89052a009d switch remaining member dependencies to workspace
these are only used by a single member at the moment, but we can move them to
the workspace to have a single location for version + base feature set
specification.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-12 09:08:36 +01:00
Fabian Grünbichler
32504b78db switch remaining member dependencies to workspace
these are only used by a single member at the moment, but we can move them to
the workspace to have a single location for version + base feature set
specification.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-12 09:08:36 +01:00
Fabian Grünbichler
8dbc2d7311 switch regular dependencies to workspace ones
where applicable.

notable changes:
- serde now uses 'derive' feature across the board
- serde removed from pbs-tools (not used)
- openssl bumped to 0.40 (and patched comment removed)
- removed invalid zstd comment

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-12 09:07:12 +01:00
Fabian Grünbichler
b659deb529 switch regular dependencies to workspace ones
where applicable.

notable changes:
- serde now uses 'derive' feature across the board
- serde removed from pbs-tools (not used)
- openssl bumped to 0.40 (and patched comment removed)
- removed invalid zstd comment

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-12 09:07:12 +01:00
Fabian Grünbichler
580399b26c switch proxmox dependencies to workspace
besides harmonizing versions, the only global change is that the tokio-io
feature of pxar is now implied since its default anyway, instead of being
spelled out.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-12 09:05:27 +01:00
Fabian Grünbichler
d75e305162 switch proxmox dependencies to workspace
besides harmonizing versions, the only global change is that the tokio-io
feature of pxar is now implied since its default anyway, instead of being
spelled out.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-12 09:05:27 +01:00
Fabian Grünbichler
6623ebdf2a workspace: inherit metadata
pbs-buildcfg is the only one that needs to inherit the version as well, since
it stores it in the compiled crate.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-12 09:05:27 +01:00
Fabian Grünbichler
219af02796 workspace: inherit metadata
pbs-buildcfg is the only one that needs to inherit the version as well, since
it stores it in the compiled crate.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-12 09:05:27 +01:00
Fabian Grünbichler
485ed1a2a2 switch exclude to workspace in README
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-09 13:31:51 +01:00
Wolfgang Bumiller
d3f2a86f80 buildsys: get crate list via cargo metadata in Makefile
so we don't have to keep this in sync manually

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-12-09 13:18:23 +01:00
Fabian Grünbichler
e6d1e6440d add bump.sh
for bumping crates in this workspace (it requires cargo-edit being installed).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-07 10:36:27 +01:00
Fabian Grünbichler
de6a59289a proxmox-time: drop TryFrom use statement
no longer needed with edition 2021

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-07 09:48:47 +01:00
Fabian Grünbichler
46a675830d update d/control files
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-07 09:48:47 +01:00
Fabian Grünbichler
48abc5afa3 update outdated workspace dependencies
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-07 09:48:47 +01:00
Fabian Grünbichler
bdca6de588 update d/control files
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-07 09:48:47 +01:00
Fabian Grünbichler
e5abc0590e define workspace dependencies in workspace
so that we no longer have to (or forget to) bump the version in multiple places.

notable changes:
- outdated versions have been unified
- proxmox-metrics -> proxmox-async no longer uses explicit empty features
  (proxmox-async doesn't provide any anyway)
- proxmox-subscription -> proxmox-http no longer uses explicit default_features
  = false (proxmox-http has an empty default feature anyway)
- missing path dependencies added (mainly proxmox-rest-server)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-07 09:48:38 +01:00
Fabian Grünbichler
6c161bd5ab update d/control files
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-07 09:48:38 +01:00
Fabian Grünbichler
4189221470 inherit shared, external dependencies
noteworthy changes:
- proxmox-http had a default_features_false dep on hyper, which is dropped (the
  default feature is empty anyway)
- hyper, libc, nix, tokio and url versions are unified
- missing (cosmetic) bindgen feature on zstd enabled everywhere

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-07 09:48:38 +01:00
Fabian Grünbichler
64959d9ae0 move common metadata to workspace
and switch all crates to 2021 edition as well as a unified "authors" value.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-07 09:48:25 +01:00
Wolfgang Bumiller
378b763408 tree-wide: bump edition to 2021
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-12-06 13:31:01 +01:00
Wolfgang Bumiller
5acca01947 tree-wide: bump edition to 2021
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-12-06 13:31:01 +01:00
Fabian Grünbichler
5ec765f842 update d/control files
debcargo 2.6 changed some minor details

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-06 11:21:43 +01:00
Wolfgang Bumiller
6a3a3f0413 use statement fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-12-05 14:11:16 +01:00
Fabian Grünbichler
1aa6f0ea22 clippy 1.65 fixes
and rustfmt

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-05 11:40:02 +01:00
Fabian Grünbichler
538578c558 clippy 1.65 fixes
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-05 11:17:37 +01:00
Fabian Grünbichler
50aa62b764 proxmox-schema: bump to 1.3.5
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-01 11:10:01 +01:00
Fabian Grünbichler
30388b7256 schema: update to textwrap 0.16
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-01 11:06:41 +01:00
Dominik Csapak
d70ae30a43 use derive 'Default' for ChunkOrder
instead of hardcoding the default deep inside the code. This makes it
much easier to see what is the actual default

the first instance of ChunkOrder::None was only for the test case, were
the ordering doe not matter

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-28 15:59:55 +01:00
Wolfgang Bumiller
854fb5c08f api-types: add MaintenanceType::Delete
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-28 14:45:39 +01:00
Wolfgang Bumiller
cf10742842 api-types: derive Display and FromStr for MaintenanceType
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-28 14:45:39 +01:00
Wolfgang Bumiller
5196e0bbee api-types: make Operation Eq
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-28 14:45:39 +01:00
Dominik Csapak
017a0652cd datastore: make 'filesystem' the default sync-level
rationale is that it makes the backup much safer than 'none', but does not
incur a big of a performance hit as 'file'.

here some benchmark:

data to be backed up:
~14GiB semi-random test images between 12kiB and 4GiB
that results in ~11GiB chunks (more than ram available on the target)

PBS setup:
virtualized (on an idle machine), PBS itself was also idle
8 cores (kvm64 on Intel 12700k) and 8 GiB memory

all virtual disks are on LVM with discard and iothread on
the HDD is a 4TB Seagate ST4000DM000 drive, and the NVME is a 2TB
Crucial CT2000P5PSSD8

i tested each disk with ext4/xfs/zfs (default created with the gui)
with 5 runs each, inbetween the caches are flushed and the filesystem synced
i removed the biggest and smallest result and from the remaining 3
results built the average (percentage is relative to the 'none' result)

result:

test         none     filesystem         file
hdd - ext4   125.67s  140.39s (+11.71%)  358.10s (+184.95%)
hdd - xfs    92.18s   102.64s (+11.35%)  351.58s (+281.41%)
hdd - zfs    94.82s   104.00s (+9.68%)   309.13s (+226.02%)
nvme - ext4  60.44s   60.26s (-0.30%)    60.47s (+0.05%)
nvme - xfs   60.11s   60.47s (+0.60%)    60.49s (+0.63%)
nvme - zfs   60.83s   60.85s (+0.03%)    60.80s (-0.05%)

So all in all, it does not seem to make a difference for nvme drives,
for hdds 'filesystem' increases backup time by ~10%, while
for 'file' it largely depends on the filesystem, but always
in the range of factor ~3 - ~4

Note that this does not take into account parallel actions, such as gc,
verify or other backups.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-28 09:49:55 +01:00
Wolfgang Bumiller
d513ef7836 bump proxmox-section-config to 1.0.2-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-28 09:30:03 +01:00
Wolfgang Bumiller
c39d04c33e minor doc fixup
an 'ignore' block assumes rust syntax, a 'text' block should
just be plain text

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-28 09:27:28 +01:00
Fiona Ebner
0cd0d16c1e section config: support allowing unknown section types
Similar to commit c9ede1c ("support unknown types in section config")
in pve-common.

Unknown sections are parsed as String-JSON String key-value pairs
without any additional checks and also written as-is.

Suggested-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-11-28 09:19:05 +01:00
Thomas Lamprecht
bb7519af3b subscription: recognize 'Suspended' status
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-25 09:53:58 +01:00
Wolfgang Bumiller
3193237afd rrd: add Entry::get() to access the data
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-24 13:53:49 +01:00
Wolfgang Bumiller
b5708459d3 api-types: derive Ord for BackupDir
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-15 10:16:54 +01:00
Dominik Csapak
7a98c5d50c datastore: improve sync level code a bit
fixups for DatastoreFSyncLevel:
* use derive for Default
* add some more derives (Clone, Copy)

chunk store:
* drop to_owned for chunk_dir_path

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-28 13:04:22 +02:00