Commit Graph

7 Commits

Author SHA1 Message Date
Lukas Wagner
073b53ae71 metrics: add /cluster/metrics/export endpoint
This new endpoint returns node, storage and guest metrics in JSON
format. The endpoint supports history/max-age parameters, allowing
the caller to query the recent metric history as recorded by the
PVE::PullMetric module.

The returned data format is quite simple, being an array of
metric records, including a value, a metric name, an id to identify
the object (e.g. qemu/100, node/foo), a timestamp and a type
('gauge', 'derive', ...). The latter property makes the format
self-describing and aids the metric collector in choosing a
representation for storing the metric data.

    [
        ...
        {
            "metric": "cpu_avg1",
            "value": 0.12,
            "timestamp": 170053205,
            "id": "node/foo",
            "type": "gauge"
        },
        ...
    ]

Some experiments were made in regards to making the format
more 'efficient', e.g. by grouping based on timestamps/ids, resulting
in a much more nested/complicated data format. While that
certainly reduces the size of the raw JSON response by quite a bit,
after GZIP compression the differences are negligible (the
simple, flat data format as described above compresses by a factor
of 25 for large clusters!). Also, the slightly increased CPU load
of compressing the larger amount of data when e.g. polling once a
minute is so small that it's indistinguishable from noise in relation
to a usual hypervisor workload. Thus the simpler, format was
chosen. One benefit of this format is that it is more or less already
the exact same format as the one Prometheus uses, but in JSON format -
so adding a Prometheus metric scraping endpoint should not be much
work at all.

The API endpoint collects metrics for the whole cluster by calling
the same endpoint for all cluster nodes. To avoid endless request
recursion, the 'local-only' request parameter is provided. If this
parameter is set, the endpoint implementation will only return metrics
for the local node, avoiding a loop.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
[WB: remove unused $start_time leftover from benchmarks]
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-08-14 14:18:48 +02:00
Dominik Csapak
5964deeae3 api: metrics: use complete config for testing the connection
not only the given parameters, e.g. at the moment, the gui will
never send a 'verify-certificate' parameter, even if set in the config

by using the complete resulting config, we test the actual settings.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-15 16:17:51 +01:00
Dominik Csapak
27bc5e8e02 status/plugin: extend with add/update/delete hooks
like we do in it for the storage section configs

we will need this to store the token for influxdbs http api

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 17:32:23 +01:00
Dominik Csapak
161d904f13 api: cluster/metricserver: prevent simultaneosly setting and deleting of property
like we do in other apis of section configs (e.g. storage)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 17:32:23 +01:00
Dominik Csapak
dadba141a8 api: metrics/server: test connection on add/update
just a basic check, but better than not checking at all

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-25 14:55:25 +01:00
Thomas Lamprecht
d57f8d9482 ui/api: metrics: move server to subpath, post to path with ID
We actually wanted to use that scheme for more new API paths, lets
see if it is really fitting starting with this.

Use the new widget-toolkit submitUrl helper to add the ID on create.

And unify the edit/create window creation, which may fit better in a
separate commit, it's quite small and was to cumbersome to untangle,
so just go against my one rules here... :(

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-21 20:53:05 +01:00
Thomas Lamprecht
9dedf1e2d5 api: drop Config from MetricServer module name
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-21 20:39:33 +01:00