proxmox-mirrors/proxmox
1
0
Fork 1
mirror of https://git.proxmox.com/git/proxmox synced 2025-05-31 23:42:10 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
2,871 Commits 3 Branches 0 Tags 17 MiB
ca04e129e4
Commit Graph

107 Commits

Author SHA1 Message Date
Fabian Grünbichler
5b19368000 tree-wide: fix needless borrows 
found and fixed via clippy

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-12-30 13:55:33 +01:00
Wolfgang Bumiller
cb32acc703 cleanup schema function calls 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2021-12-16 11:25:02 +01:00
Wolfgang Bumiller
e207a84d93 bump proxmox-schema to 1.1 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2021-12-16 11:25:02 +01:00
Dietmar Maurer
a092ef9c32 update to proxmox-sys 0.2 crate 
- imported pbs-api-types/src/common_regex.rs from old proxmox crate
- use hex crate to generate/parse hex digest
- remove all reference to proxmox crate (use proxmox-sys and
  proxmox-serde instead)

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-24 10:32:27 +01:00
Dietmar Maurer
0fdc15a3f8 use proxmox::tools::fd::fd_change_cloexec from proxmox 0.15.3 
Depend on proxmox 0.15.3

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-18 13:43:41 +01:00
Dominik Csapak
8e344d3d67 rrd: use saturating_sub to avoid underflow 
Without this, the tests fail in debug mode.
Also having start (u64) underflow to a value greater than end does
not really make sense

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2021-10-28 12:54:54 +02:00
Dietmar Maurer
75ca726c29 use new fsync parameter to replace_file and atomic_open_or_create 
Depend on proxmox 0.15.0 and proxmox-openid 0.8.1

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-10-21 07:28:32 +02:00
Dietmar Maurer
412712029c proxmox-rrd: use fsync instead of syncfs 
syncfs can sync unrelated data, and we do not want that.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-10-20 11:46:59 +02:00
Dietmar Maurer
a7ee3455da proxmox-rrd: fix regression tests 2021-10-19 18:41:03 +02:00
Dietmar Maurer
86b50e18ed proxmox-rrd: improve dev docs 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-10-19 11:17:09 +02:00
Dietmar Maurer
ed6a7f52e5 proxmox-rrd: cleanup - impl FromStr for JournalEntry 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-10-19 11:17:09 +02:00
Dietmar Maurer
75bb60e7b3 proxmox-rrd: add option to avoid page cache for load/save 
use fadvice(.., POSIX_FADV_DONTNEED) for RRD files. We read those files only once,
and always rewrite them.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-10-19 11:17:09 +02:00
Dietmar Maurer
336e8f3e7f proxmox-rrd: use syncfs after writing rrd files 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-10-19 11:17:09 +02:00
Dietmar Maurer
77c2e4668b proxmox-rrd: use fine grained locking in commit_journal_impl 
Aquire the rrd_map lock for each file (else we block access for a long time)
 2021-10-18 14:55:47 +02:00
Dietmar Maurer
cc0bb59788 proxmox-rrd: log all errors from apply_and_commit_journal_thread (but only once) 2021-10-18 11:57:19 +02:00
Dietmar Maurer
e23f3ec774 proxmox-rrd: cleanup list_old_journals 2021-10-18 10:00:58 +02:00
Dietmar Maurer
a74384f725 proxmox-rrd: cleanup - use struct instead of tuple 2021-10-16 12:45:03 +02:00
Dietmar Maurer
4393b93a8b proxmox-rrd: move RRDMap into extra file 2021-10-16 12:45:03 +02:00
Dietmar Maurer
9dcc64b71a proxmox-rrd: move JournalState into extra file 2021-10-16 12:45:03 +02:00
Dietmar Maurer
30b4800f4f proxmox-rrd: implement non blocking journal 
Do not block while applying the journal.
 2021-10-16 12:45:03 +02:00
Dietmar Maurer
2c24c1dd22 proxmox-rrd: rename RRDCacheState to JournalState 2021-10-15 09:35:44 +02:00
Dietmar Maurer
2be07c2250 proxmox-rrd: avoild blocking readers while applying the journal 
By using and extra RwLock<RRDMap> on the rrd data.
 2021-10-15 09:22:07 +02:00
Dietmar Maurer
3275f1ac16 proxmox-rrd: log journal apply/flush times, split apply and flush 
We need to apply the journal only once.
 2021-10-15 07:16:41 +02:00
Dietmar Maurer
ec5d84f4d3 proxmox-rrd: cleanup - use slot_end_time() 2021-10-14 16:29:00 +02:00
Dietmar Maurer
2b9fb32de1 proxmox-rrd: cleanup - use staturating_add instead of if/else 2021-10-14 16:10:55 +02:00
Dietmar Maurer
26bd6a4f77 proxmox-rrd: improve dev docs 2021-10-14 11:53:54 +02:00
Dietmar Maurer
8619b21e4d proxmox-rrd: make rrd load callback configurable 2021-10-14 11:41:26 +02:00
Dietmar Maurer
52d5f340f2 proxmox-rrd: add more regression tests 2021-10-14 10:55:12 +02:00
Dietmar Maurer
f2e1ab2d44 proxmox-rrd: add regression tests and two minor fixes 2021-10-14 10:17:07 +02:00
Dietmar Maurer
a1eede6918 proxmox-rrd: pass time and value to update function 2021-10-14 08:12:56 +02:00
Dietmar Maurer
4231e2d5f5 proxmox-rrd: add some integration tests (file format tests) 2021-10-13 18:21:23 +02:00
Dietmar Maurer
432374d024 use complete_file_name from proxmox-router 1.1 2021-10-13 14:10:02 +02:00
Thomas Lamprecht
919ccf713a proxmox-rrd: move unshipped cli tool to examples 
it's a rather low-level tool mostly useful for debugging and some of
it is rather "dumb" (by design) anyway, e.g., it does not
transparently applies journal but really only operates on the DB
files as is (which can conflict with daemon operations).

In summary, not (yet) a tool meant for end user consumption.
Move it to examples folder to avoid compilation on packaging (we do
not ship it anyway) which allows us to move the rather expensive
proxmox-router (pulls in hyper) to the dev-dependencies section.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar Maurer
392d646f7b proxmox-rrd: add more commands to the rrd cli tool 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar Maurer
7edea7e08c proxmox-rrd: rename last_counter to last_value 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar Maurer
66dfd1f08f proxmox-rrd: protect against negative update time 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar Maurer
507f19dd33 proxmox-rrd: new helpers: slot, slot_start_time & slot_end_time 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar Maurer
334eb9ce48 proxmox-rrd: avoid expensive modulo (%) inside loop 
Modulo is very slow, so we try to avoid it inside loops.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar Maurer
4cd28918e2 proxmox-rrd: add binary to create/manage rrd files 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar Maurer
2c72c6a7ba proxmox-rrd: split out load_rrd (cleanup) 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar Maurer
e928c24948 proxmox-rrd: support CF::Last 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar Maurer
4bf2db8666 remove proxmox-rrd-api-types crate, s/RRDTimeFrameResolution/RRDTimeFrame/ 
Because the types used inside the RRD have other requirements
than the API types:

- other serialization format
- the API may not support all RRD features

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar Maurer
ab567561b5 proxmox-rrd: extract_data: include values from current slot 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar Maurer
cf097c5a89 proxmox-rrd: remove dependency to proxmox-rrd-api-types 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar Maurer
bc68dee171 proxmox-rrd: implement new CBOR based format 
Storing much more data points now got get better graphs.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar Maurer
0355554905 proxmox-rrd: use a journal to reduce amount of bytes written 
Append pending changes in a simple text based format that allows for
lockless appends as long as we stay below 4 KiB data per write.

Apply the journal every 30 minutes and on daemon startup.

Note that we do not ensure that the journal is synced, this is a
perfomance optimization we can make as the kernel defaults to
writeback in-flight data every 30s (sysctl vm/dirty_expire_centisecs)
anyway, so we lose at max half a minute of data on a crash, here one
should have in mind that we normally expose 1 minute as finest
granularity anyway, so not really much lost.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Wolfgang Bumiller
e0ce41b03a update to first proxmox crate split 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2021-10-11 11:58:49 +02:00
Wolfgang Bumiller
fa9757e67f bump proxmox dependency to 0.14.0 and proxmox-http to 0.5.0 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2021-10-08 11:18:22 +02:00
Dietmar Maurer
071cb7aa8b proxmox-rrd: use correct directory options in create_rrdb_dir 2021-10-07 08:50:50 +02:00
Dietmar Maurer
9c64c09c92 proxmox-rrd: cleanup error handling 2021-10-07 08:01:12 +02:00
Dietmar Maurer
5165bed8c2 proxmox-rrd: use log crate instead of eprintln, avoid duplicate logs 2021-10-06 18:19:22 +02:00
Dietmar Maurer
9c7fd3c936 proxmox-rrd: fix update (do not update) when time is in the past 2021-10-06 18:01:48 +02:00
Dietmar Maurer
881d8f85ea proxmox-rrd: improve developer docs 2021-10-06 12:19:54 +02:00
Dietmar Maurer
54f7a80f97 proxmox-rrd: remove serde dependency 2021-10-06 10:55:46 +02:00
Dietmar Maurer
538e6f66f3 split out RRD api types into proxmox-rrd-api-types crate 2021-10-06 09:49:51 +02:00
Dietmar Maurer
ac17698e4a proxmox-rrd: use create_path instead of std::fs::create_dir_all 
To ensure correct file ownership.
 2021-10-06 08:37:14 +02:00
Dietmar Maurer
8d1a9d2ec6 move RRD code into proxmox-rrd crate 2021-10-06 08:13:28 +02:00