Commit Graph

2599 Commits

Author SHA1 Message Date
Dietmar Maurer
8619b21e4d proxmox-rrd: make rrd load callback configurable 2021-10-14 11:41:26 +02:00
Dietmar Maurer
52d5f340f2 proxmox-rrd: add more regression tests 2021-10-14 10:55:12 +02:00
Dietmar Maurer
f2e1ab2d44 proxmox-rrd: add regression tests and two minor fixes 2021-10-14 10:17:07 +02:00
Dietmar Maurer
a1eede6918 proxmox-rrd: pass time and value to update function 2021-10-14 08:12:56 +02:00
Dietmar Maurer
4231e2d5f5 proxmox-rrd: add some integration tests (file format tests) 2021-10-13 18:21:23 +02:00
Dietmar Maurer
0cf4129204 use complete_file_name from proxmox-router 1.1 2021-10-13 14:10:02 +02:00
Dietmar Maurer
432374d024 use complete_file_name from proxmox-router 1.1 2021-10-13 14:10:02 +02:00
Thomas Lamprecht
919ccf713a proxmox-rrd: move unshipped cli tool to examples
it's a rather low-level tool mostly useful for debugging and some of
it is rather "dumb" (by design) anyway, e.g., it does not
transparently applies journal but really only operates on the DB
files as is (which can conflict with daemon operations).

In summary, not (yet) a tool meant for end user consumption.
Move it to examples folder to avoid compilation on packaging (we do
not ship it anyway) which allows us to move the rather expensive
proxmox-router (pulls in hyper) to the dev-dependencies section.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 13:36:02 +02:00
Dietmar Maurer
392d646f7b proxmox-rrd: add more commands to the rrd cli tool
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 13:36:02 +02:00
Dietmar Maurer
7edea7e08c proxmox-rrd: rename last_counter to last_value
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 13:36:02 +02:00
Dietmar Maurer
66dfd1f08f proxmox-rrd: protect against negative update time
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 13:36:02 +02:00
Dietmar Maurer
507f19dd33 proxmox-rrd: new helpers: slot, slot_start_time & slot_end_time
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 13:36:02 +02:00
Dietmar Maurer
334eb9ce48 proxmox-rrd: avoid expensive modulo (%) inside loop
Modulo is very slow, so we try to avoid it inside loops.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 13:36:02 +02:00
Dietmar Maurer
4cd28918e2 proxmox-rrd: add binary to create/manage rrd files
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 13:36:02 +02:00
Dietmar Maurer
2c72c6a7ba proxmox-rrd: split out load_rrd (cleanup)
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 13:36:02 +02:00
Dietmar Maurer
e928c24948 proxmox-rrd: support CF::Last
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 13:36:02 +02:00
Dietmar Maurer
4bf2db8666 remove proxmox-rrd-api-types crate, s/RRDTimeFrameResolution/RRDTimeFrame/
Because the types used inside the RRD have other requirements
than the API types:

- other serialization format
- the API may not support all RRD features

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 13:36:02 +02:00
Dietmar Maurer
ab567561b5 proxmox-rrd: extract_data: include values from current slot
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 13:36:02 +02:00
Dietmar Maurer
cf097c5a89 proxmox-rrd: remove dependency to proxmox-rrd-api-types
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 13:36:02 +02:00
Dietmar Maurer
bc68dee171 proxmox-rrd: implement new CBOR based format
Storing much more data points now got get better graphs.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 13:36:02 +02:00
Dietmar Maurer
0355554905 proxmox-rrd: use a journal to reduce amount of bytes written
Append pending changes in a simple text based format that allows for
lockless appends as long as we stay below 4 KiB data per write.

Apply the journal every 30 minutes and on daemon startup.

Note that we do not ensure that the journal is synced, this is a
perfomance optimization we can make as the kernel defaults to
writeback in-flight data every 30s (sysctl vm/dirty_expire_centisecs)
anyway, so we lose at max half a minute of data on a crash, here one
should have in mind that we normally expose 1 minute as finest
granularity anyway, so not really much lost.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 13:36:02 +02:00
Dietmar Maurer
48ef839043 bump proxmox-router version to 1.1.0-1 2021-10-13 12:28:16 +02:00
Dietmar Maurer
417b7159d2 add filename completions helper (moved from pbs-tools)
Depend on 'nix' now.
2021-10-13 12:28:16 +02:00
Wolfgang Bumiller
17adc570db bump proxmox-borrow to 1.0.1-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-13 10:41:01 +02:00
Wolfgang Bumiller
087bf31567 borrow: update to ManuallyDrop::take
and fixup into_boxed_inner along the way

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-13 10:39:12 +02:00
Wolfgang Bumiller
d18292192d bump proxmox-api-macro to 1.0.1-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-12 14:51:35 +02:00
Wolfgang Bumiller
5988a1adf1 drop automatically_derived attribute for now
new rustc seems to *sometimes* complain about it

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-12 14:49:11 +02:00
Wolfgang Bumiller
f63ce12b66 lang: prepare c_str for const fns with 1.56
provides an api compatible const-fn-compatible c_str
alternative working on 1.56

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-12 14:29:03 +02:00
Wolfgang Bumiller
8f89f9ad60 lang: get offsetof const fn ready
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-12 14:14:25 +02:00
Wolfgang Bumiller
af33a97547 lang: deprecate ops::ControlFlow
as we now have rustc 1.55

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-12 14:13:59 +02:00
Wolfgang Bumiller
d65b2df750 update to proxmox split and bump version to 0.8.0-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-11 12:37:13 +02:00
Wolfgang Bumiller
4ccd6256a8 update proxmox-http to 0.5 for the split
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-11 12:34:14 +02:00
Wolfgang Bumiller
336dab0177 update proxmox crate to the current split
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-11 12:34:14 +02:00
Wolfgang Bumiller
c7b17de1b5 update to proxmox split and bump version to 0.8.0-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-11 12:00:19 +02:00
Wolfgang Bumiller
09046671ed update to first proxmox crate split
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-11 11:58:49 +02:00
Wolfgang Bumiller
e0ce41b03a update to first proxmox crate split
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-11 11:58:49 +02:00
Wolfgang Bumiller
1d24555b28 drop u2f-api file
used to be used by examples at some point

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-11 11:39:59 +02:00
Wolfgang Bumiller
f35dbbd651 add proxmox-section-config crate
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-11 11:39:59 +02:00
Wolfgang Bumiller
41f3fdfeb9 add proxmox-schema and proxmox-router crates
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-11 11:39:59 +02:00
Wolfgang Bumiller
01a8b6f1bf add proxmox-io crate
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-11 10:07:53 +02:00
Wolfgang Bumiller
91f59f9f59 add proxmox-lang crate
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-08 15:22:17 +02:00
Wolfgang Bumiller
9b6fe4aceb add proxmox-time crate
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-08 15:22:17 +02:00
Wolfgang Bumiller
7db0a3c6df add proxmox-borrow crate
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-08 15:22:17 +02:00
Wolfgang Bumiller
bd67ccc1b3 add proxmox-uuid crate
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-08 15:22:17 +02:00
Wolfgang Bumiller
77dc52c047 add proxmox-tfa crate
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-08 15:22:17 +02:00
Wolfgang Bumiller
2859858f59 fix systemd::escape_unit's hex encoding
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-08 15:22:17 +02:00
Wolfgang Bumiller
6ad1bcaf89 bump proxmox dependency to 0.14.0 and proxmox-http to 0.5.0
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-08 11:18:22 +02:00
Wolfgang Bumiller
fa9757e67f bump proxmox dependency to 0.14.0 and proxmox-http to 0.5.0
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-08 11:18:22 +02:00
Dominik Csapak
fb6823b54b rest-server: add cleanup_old_tasks
this is a helper that removes task log files that are not referenced
by the task archive anymore

it gets the oldest task archive file, gets the first endtime (the
oldest) and removes all files in the taskdir where the mtime is older
than that

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-10-08 06:38:52 +02:00
Dietmar Maurer
071cb7aa8b proxmox-rrd: use correct directory options in create_rrdb_dir 2021-10-07 08:50:50 +02:00