Commit Graph

4813 Commits

Author SHA1 Message Date
Thomas Lamprecht
1ff3fef0b0 fix #844: allow to pre-delay start-all-marked guests on boot
Add a simple ExecStartPre command which reads the local node config,
and if a delay is set the helper sleeps that long then exists.

The systemd-unit approach was chosen as this ensures that we really
only delay when doing the startall on node boot. The pve-guests
service does not allows manual stops, starts or restarts, it can only
be pulled in by the multi-user.target

Mark this command with "-" to tell systemd that errors of it should
not cause an abort, it's a best-effort approach.

The journal from a 2 second delay would look like:
> Nov 19 13:13:48 dev6 systemd[1]: Starting PVE guests...
> Nov 19 13:13:48 dev6 pve-startall-delay[2318]: Delaying on-boot 'startall' command for 2 second(s).
> ...
> Nov 19 13:13:50 dev6 pve-guests[2339]: <root@pam> starting task UPID:dev6:00000924:00000529:5DD3DC7E:startall::root@pam:

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-20 20:20:58 +01:00
Dominik Csapak
f36240c507 gui: pci passthrough: consider domain in PCISelector
but remove the default domain '0000' before sending to the backend,
and add it if no domain is given in the config

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-11-20 19:43:31 +01:00
Thomas Lamprecht
4b527db9e5 d/control: bump version dependency to qemu-server
for mon_cmd refactoring

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-20 19:43:31 +01:00
Stefan Reiter
7a108020b3 refactor: vm_mon_cmd is now Monitor::mon_cmd
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-11-20 18:25:49 +01:00
Thomas Lamprecht
cd3b6b0177 allow to run some tests in parallel
the replication test are not yet ready for that, but the others can
be run in parallel

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-19 15:02:50 +01:00
Thomas Lamprecht
f1ea8ca4f1 followup: cleanup/refactor mail test
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-19 15:02:50 +01:00
Dominic Jäger
ce84a9506d Fix #352: Limit the length of backup logs for mails
When creating a backup the log part can make the mail too big to be
transferred. To ensure delivery, two measures are taken:
1. Always omit the status lines
2. Omit the whole log part if a mail becomes (too) big

Additionally, add a check for missing log files.

Co-developed-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
2019-11-19 15:02:50 +01:00
Thomas Lamprecht
1b025c2fa0 api: node/config: allow to return only a specific config property
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-19 12:04:52 +01:00
Thomas Lamprecht
db2ce4886c status/graphite: fix memory leak, avoid cyclic closure reference
The data passed to this closure was never free'd, depending on the
count of VM/CTs one could get >1 MB of RSS (!) memory leaked per
statd status cycle update run...

We could also use Scalar::Util's weaken, to weak a copy of this
variable, but as a simple undef works lets do that with a comment..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-19 09:11:47 +01:00
Thomas Lamprecht
1e4ae7d44c fixup: graphite: use correct variable in closure
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-19 09:10:54 +01:00
Thomas Lamprecht
e35f1d37d7 Ext. Metrics: module global variable does not help with memory leak
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-18 19:15:13 +01:00
Thomas Lamprecht
2112d31092 statd: increase RSS difference required for restart
it seems that we have a reference leak or the like somewhere in the
(graphite?) status plugin, while the recent transaction based update
mechanism made it slightly better, it's still bad with a lot of VMs..

Until we can track that down, or abandon perl for good, avoid to
frequent restarts by allowing statd to grow 15 MB of memory usage
after initial calibration (it's memory usage at the 10th cycle)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-18 19:07:24 +01:00
Thomas Lamprecht
cc3d280b98 statd: report memory usage in KB
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-18 19:04:29 +01:00
Thomas Lamprecht
87be2c19e3 ext. metric: move to a transaction model
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-18 19:04:29 +01:00
Fabian Grünbichler
83cab72f62 bump version to 6.0-12
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-18 12:25:35 +01:00
Fabian Grünbichler
0721bc786c d/control: bump versioned b-d on libpve-guest-common-perl
otherwise the replication tests fail to load datacenter.cfg

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-11-18 12:25:35 +01:00
Fabian Grünbichler
82afbfdd2c d/control: add (build-)depends on libpve-cluster-api-perl
it contains PVE::API2::ClusterConfig

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-11-18 12:25:35 +01:00
Fabian Grünbichler
f02546f772 d/control: add (build-)depends on libpve-cluster-perl
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-11-18 12:25:35 +01:00
Fabian Grünbichler
33cd5dfefe test: also mock cfs_read_file
needed for tests to work without a functional running pmxcfs instance.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-11-18 12:25:35 +01:00
Fabian Grünbichler
57f110df07 pveproxy.service: take over pvecm call from pve-cluster.service
to ensure certificate/key is present prior to loading them.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-11-18 12:25:35 +01:00
Fabian Grünbichler
516a7948cd use PVE::RRD for RRD data
refactored from PVE::Cluster. same code, same semantics, different file.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-11-18 12:25:35 +01:00
Fabian Grünbichler
fe57e096cc test: mock PVE::SSHInfo module
since it is now used for getting a cluster node's migration IP address.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-11-18 12:25:35 +01:00
Fabian Grünbichler
3ac3653e63 use PVE::DataCenterConfig
to make sure that the corresponding cfs_read_file works() works.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-11-18 12:25:35 +01:00
Fabian Grünbichler
0f9ac2dfc5 takeover CertCache from pve-cluster
same code, same semantics, different file/module

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-11-18 12:25:35 +01:00
Thomas Lamprecht
f1f4bfefc7 move common metric server management part to own module
For now it only handles the plugin registration and the two recently
integrated helpers.
But, this is a prepartation to move the external metrics server
update mechanic from a stateless always-newly-connect-send-disconnect
to a statefull transaction based mechanis; see later patches

keep the PVE::Status::Plugin use in pvestatd, as we read the cfs
hosted status.cfg there, and the parser is defined by the common
status plugin base module.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-16 16:19:42 +01:00
Thomas Lamprecht
e1c6aaeb16 makefile: sort perlsources module list
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-16 16:10:24 +01:00
Thomas Lamprecht
df598da578 ui: ceph/Services: do some housekeeping
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-16 14:31:31 +01:00
Thomas Lamprecht
f4643f1782 bump version to 6.0-12
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-15 19:04:53 +01:00
Thomas Lamprecht
1aaca6fde7 api: ceph/metadata: add structured node versions
include the version as string and as parts, as we do the split
already. Also include the build commit, so if we re-release a ceph
version, we can differ here too.

Use node as key, to make the new entry a bit more general, could be
easily expanded with other infos, if required.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-15 18:36:37 +01:00
Thomas Lamprecht
3248590d53 api: ceph version: actually get full version
add and change the return signature for the wantarray case, which can
safely done as this is only used once (statd), and there only the
first elemen, the full version string, is used - so no breakage
potential there

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-15 18:35:39 +01:00
Thomas Lamprecht
2a8e514947 statd: adapt ceph update error message
"getting ceph services" sound a bit vague, like the download of those
failed, or the like..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-15 11:33:47 +01:00
Thomas Lamprecht
a6dff455f6 statd: refactor out updating ceph metadata
makes no sense to do half in line and half in a extra update_method

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-15 11:30:14 +01:00
Thomas Lamprecht
e051836377 status: cleanup config parser regsistration
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-15 10:49:05 +01:00
Thomas Lamprecht
68f58b5d59 status plugins: add _connect to plugin method interface
in preparation of doing real transactions, with one batch connect +
send + disconnect, and not hundreds of those per update cycle..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-14 19:27:28 +01:00
Thomas Lamprecht
5e82aaac89 status plugins: add update_all and foreach_plug helper
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-14 19:24:24 +01:00
Thomas Lamprecht
b25f645957 remove some useless empty lines
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-13 17:05:44 +01:00
Thomas Lamprecht
0dd73a7fec statd: refactor update_node_status
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-13 08:42:00 +01:00
Thomas Lamprecht
7887310045 statd: cleanup update_node_status
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-13 08:40:13 +01:00
Aaron Lauterer
ff4d18e1b1 ui: vm opts: add hint for spice foldersharing
Spice foldersharing needs the webdavd daemon installed inside the guest.
This patch adds a hint to remind the user to install it in the VM.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2019-11-12 11:28:48 +01:00
Stefan Reiter
a202f65a95 gui: fix onlineHelp for join window
We have an entire chapter written about this, why not link it too.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-11-11 11:47:45 +01:00
Thomas Lamprecht
108e0c8b9f status/graphite: refactor write_graphite to send all at once
Instead of doing multiple sends, for each status metric line one,
assemble it all in a string and send it out in a single go.
Per VM/CT/Node we had >10 lines to send, so this is quite the
reduction. But, also note that thanks to Nagler's delay algorithm
this may not had a big effect for TCP, as it buffered those small
writes anyhow.
For UDP it can reduce the packet count on the line dramatically,
though.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-07 18:59:06 +01:00
Thomas Lamprecht
8dcf2cac46 status/graphite: just use setsockopt to set timeouts
after rethinking this it felt weird, sockets already can to this
themself, so I checked out the IO::Socket::Timeout module, and yeah,
it's just a OOP wrapper for this, hiding the "scary" struct pack.

So instead of adding that as dependency lets do it ourself.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-07 16:27:50 +01:00
Thomas Lamprecht
228f017ee4 status/graphite: record missing module-use
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-07 16:26:54 +01:00
Dominik Csapak
f9b633668f gui: lxc/MPEdit: save and use the mountpoint propertyString
this way, backend only settings do not get lost (like 'size', 'shared')
when editing in the gui

this was most obvious with the new pending options, as every time
we edited a mp, we lost its size, and even setting the options
to exactly the same as the originals, we still had the mp as
'pending', but without the size

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-11-06 21:12:35 +01:00
Dominik Csapak
e4e36020aa gui: qemu/HDEdit: use propertyStringSet helper
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-11-06 21:12:35 +01:00
Dominik Csapak
49dfba72c8 gui: Utils: add propertyString helper
this helper conditionally sets the given value to the given property
on the given object, optionally a different value

this is useful for our MP/HD Edit panels, where we set the options
of the drive/mp this way for every gui option we have

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-11-06 21:12:35 +01:00
Dominik Csapak
69f36699b6 gui: lxc/Resources: improve reload behaviour
reload when a new disk was added, and set the button status
on a reload

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-11-06 21:12:35 +01:00
Dominik Csapak
18f57a2319 gui: qemu/HardwareView: use builtin reload function
ObjectGrid (an ancestor of PendingObjectGrid) does already have
a 'reload' function which does exactly the same, so get rid
of the local one here

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-11-06 21:12:35 +01:00
Thomas Lamprecht
c44a0c01c8 status/graphite: reduce default timeout to 1 second
This is for TCP only, and TCP needs roughly 1.5 time of the Round
Trip Time for connection setup, So, with 1 second timeout we're still
good for connections with 660 ms latency in-between.

The assumption is that most of the time the status server is
relatively near (same datacenter, or region), and connections to it
are datacenter grade, and not like a spotty GPRS modem.
So, reduce this timeout to ensure that we do not block to long.

If anybody needs higher timeouts they can just change the default
anyway.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-06 21:06:00 +01:00
Thomas Lamprecht
dd4268e50e status/graphite: refactor default assignments, no ternary
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-06 21:02:59 +01:00