Commit Graph

6074 Commits

Author SHA1 Message Date
Thomas Lamprecht
c20699493a api: cluster: add jobs sub-directory index and set its permissions
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-11 21:02:03 +01:00
Thomas Lamprecht
7cced7126b api: cluster: sort index
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-11 21:02:03 +01:00
Thomas Lamprecht
584a57ea3c calendar event simulator: rename number parameter to iterations
in sync with the same option of `systemd-analyze calendar` and number
is so generic.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-11 20:53:13 +01:00
Dominik Csapak
f493f5685d remove pvescheduler manpage generation
pve-docs-generator does this now

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 18:35:52 +01:00
Dominik Csapak
4d6215bcf5 ui: dc/Backup: add schedule simulator button
so that a user can simply simulate the schedule

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 18:35:52 +01:00
Dominik Csapak
e190bc2cc8 api: cluster: add jobs/schedule-analyze api call
a simple api call to simulate calendar event triggers
takes a schedule, an optional number (default 10), an optional starttime
(default 'now') and returns a list with unix timestamps, as well as
humanly readable utc timestamps.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 18:35:52 +01:00
Dominik Csapak
611ef47558 ui: dc/Backup: fix comment sort
without this, jobs without comments will not be included in the sort
properly

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 18:35:52 +01:00
Dominik Csapak
3446e5bb26 ui: dc/Backup: never show id input field, autogenerate id
similar to pbs sync jobs

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 18:35:52 +01:00
Thomas Lamprecht
4243d7490f d/control: bump versioned dependency on libpve-cluster-api-perl
for new webauthn config

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-11 18:35:29 +01:00
Dominik Csapak
02c1e98e83 pveceph: add 'fs destroy' command
with 'remove-storages' and 'remove-pools' as optional parameters

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 17:52:08 +01:00
Dominik Csapak
fe681357e6 ui: ceph/fs: allow creating multiple cephfs
but only if there are any standby mds

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 17:52:08 +01:00
Dominik Csapak
786e7d8328 ui: storage/cephfs: make ceph fs selectable
by adding a CephFSSelector and using it in the CephFSEdit window
(similar to the poolselector/textfield)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 17:52:08 +01:00
Dominik Csapak
6ec3dc437b api: cephfs: add 'fs-name' for cephfs storage
so that we can uniquely identify the cephfs (in case of multiple)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 17:52:08 +01:00
Dominik Csapak
999df1c25e ui: ceph/fs: show fs for active mds
so that the user can see which mds is responsible for which cephfs

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 17:52:08 +01:00
Dominik Csapak
89c0472127 ui: ceph/ServiceList: refactor controller out
we want to reuse that controller type by overriding some functionality
in the future, so factor it out.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 17:52:08 +01:00
Dominik Csapak
352a0e5c93 api: cephfs: add fs_name to 'is mds active' check
so that we check the mds for the correct cephfs we just added

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 17:52:08 +01:00
Dominik Csapak
028af34e3d api: cephfs: more checks on fs create
namely if the fs is already existing, and if there is currently a
standby mds that can be used for the new fs
previosuly, only one cephfs was possible, so these checks were not
necessary. now with pacific, it is possible to have multiple cephfs'
and we should check for those.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 17:52:08 +01:00
Dominik Csapak
0ab69d6e88 api: cephfs: refactor {ls, create}_fs
no function change intended

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 17:52:08 +01:00
Dominik Csapak
7d88e195b2 ui: ceph: catch missing version for service list
when a daemon is stopped, the version here is 'undefined'. catch that
instead of letting the template renderer run into an error.
this fixes the rendering of the grid backgrounds

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 17:52:08 +01:00
Dominik Csapak
c51db18c42 api: ceph-mds: get mds state when multple ceph filesystems exist
by iterating over all of them and saving the name to the active ones
this fixes the issue that an mds that is assigned to not the first
fs in the list gets wrongly shown as offline

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 17:52:08 +01:00
Thomas Lamprecht
82536c370d ui: form/CalendarEvent: remove 'first saturday each month' example
Originally-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-11 17:50:47 +01:00
Thomas Lamprecht
a2e0725adb ui: dc/options: show depreacation notice in u2f settings
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-11 17:44:03 +01:00
Thomas Lamprecht
cca2aea8b2 ui: dc/options: display WebAuthn and add editor
mostly copied 1:1 from PBS, we could move this to widget toolkit
sometimes though..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-11 17:38:15 +01:00
Wolfgang Bumiller
e90127be83 www: redirect user TFA button to TFA view
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-11 14:38:31 +01:00
Wolfgang Bumiller
8ff8f43512 www: add TFA view to config
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-11 14:38:31 +01:00
Wolfgang Bumiller
3c02565699 www: use af-address-book-o for realms
we do this in PBS, and use the key for TFA

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-11 14:38:31 +01:00
Wolfgang Bumiller
f744796310 www: switch to new tfa login format
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-11 14:38:31 +01:00
Wolfgang Bumiller
69c33cffe8 www: use UserSelector from wtk
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-11 14:38:23 +01:00
Wolfgang Bumiller
d35665de6c www: use render_u2f_error from wtk
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-11 14:38:23 +01:00
Fabian Ebner
358d3efd1d ui: calendar event: fix descriptions for examples
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-11-11 10:09:47 +01:00
Wolfgang Bumiller
6b2028cb73 depend on and use libjs-qrcodejs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-11 08:37:31 +01:00
Thomas Lamprecht
e82d35836c ui: storage: avoid deleting preallocation setting on creation
reported in the forum: https://forum.proxmox.com/threads/.99471/

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-10 22:03:13 +01:00
Wolfgang Bumiller
a6a681b950 pvestatd: broadcast version info
broadcast the built-in, statically available version info, e.g.:
{
   "release" : "7.0",
   "repoid" : "3ce05d40",
   "version" : "7.0-14"
}

We can expand this by more actual package version info in the future,
but that certainly needs more elaborate update control mechanisms as
the oneshot at boot we have now.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-10 21:44:32 +01:00
Thomas Lamprecht
9a621f7177 ui: form/calendar event: add some more complex examples
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-10 21:32:14 +01:00
Thomas Lamprecht
8e57774e2a ui: form/calendar event: increase width of combobox picker
makes it more readable and will allow for some more complex examples
without wrapping (which is hard to grasp without row styling
otherwise)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-10 21:31:18 +01:00
Thomas Lamprecht
01c9294f37 ui: backup job: limit ID to 20 chars for now
we're thinking of maybe just auto-filling it or allowing to edit it,
in any way, the comment field can/should be used for longer, free
form info.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-10 21:29:40 +01:00
Thomas Lamprecht
91380e55e6 ui: backup job: ellipsize read-only ID on edit
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-10 19:26:56 +01:00
Thomas Lamprecht
04da73f851 ui: backup job: avoid row wrapping due to overly long label
We can express that it's a notification in the combobox.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-10 19:25:56 +01:00
Thomas Lamprecht
3ce05d40af api: version: expand return schema info
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-10 17:34:39 +01:00
Thomas Lamprecht
b2eec1b652 api: version: only include the console option of the datacenter config
Especially with the new description the result can get quite huge and
we do not really require any other info, the console one is only
returned here anyway as a hack due to the dc config API path requires
higher privileges.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-10 17:26:52 +01:00
Dominik Csapak
d9e05d3404 ui: dc/Backup: add comment field and column
and hide the ID column by default

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-10 16:11:42 +01:00
Dominik Csapak
998b61fbff api: backup/jobs: add comment field to jobs
and encode them with PVE::Tools::encode_text in the config

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-10 16:11:42 +01:00
Dominik Csapak
e177268741 ui: dc/BackupEdit: use correct validation
we use a 'pve-configid', so use the 'ConfigId' vtype

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-10 16:11:42 +01:00
Dominik Csapak
b34220461d Jobs/VZDump: encode/decode 'prune-backups' where needed
we want to write it out to the config as propertyString,
but want it as object in the api (for compatiblity)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-10 16:11:42 +01:00
Dominik Csapak
20d15804f6 ui: dc/backup: show id+schedule instead of dow+starttime
we can now show the id (since its not autogenerated anymore),
and we can always show/edit the schedule instead of the dow+starttime

also add an 'ID' field to the edit/create window and update the
backupinfo window as well

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-10 16:11:00 +01:00
Dominik Csapak
305921b1ab api/backup: handle new vzdump jobs
in addition to listing the vzdump.cron jobs, also list from the
jobs.cfg file.

updates/creations go into the new jobs.cfg only now
and on update, starttime+dow get converted to a schedule
this transformation is straight forward, since 'dow'
is already in a compatible format (e.g. 'mon,tue') and we simply
append the starttime (if any)

id on creation is optional for now (for api compat), but will
be autogenerated (uuid). on update, we simply take the id from before
(the ids of the other entries in vzdump.cron will change but they would
anyway)

as long as we have the vzdump.cron file, we must lock both
vzdump.cron and jobs.cfg, since we often update both

we also change the backupinfo api call to read the jobs.cfg too

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-10 16:11:00 +01:00
Dominik Csapak
85b9ba88f1 api/backup: refactor string for all days
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-10 16:11:00 +01:00
Dominik Csapak
fa7d54564a pvescheduler: run jobs from jobs.cfg
PVE/Jobs is responsible to decide if the job must run (e.g. with a
schedule)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-10 16:11:00 +01:00
Dominik Csapak
76c6ee8a6f add PVE/Jobs to handle VZDump jobs
this adds a SectionConfig handling for jobs (only 'vzdump' for now) that
represents a job that will be handled by pvescheduler and a basic
'job-state' handling for reading/writing state json files

this has some intersections with pvesrs state handling, but does not
use a single state file for all jobs, but seperate ones, like we
do it in the backup-server.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-10 16:11:00 +01:00
Thomas Lamprecht
6385fb8183 replace systemd timer with pvescheduler daemon
The whole thing is already prepared for this, the systemd timer was
just a fixed periodic timer with a frequency of one minute. And we
just introduced it as the assumption was made that less memory usage
would be generated with this approach, AFAIK.

But logging 4+ lines just about that the timer was started, even if
it does nothing, and that 24/7 is not to cheap and a bit annoying.

So in a first step add a simple daemon, which forks of a child for
running jobs once a minute.
This could be made still a bit more intelligent, i.e., look if we
have jobs tor run before forking - as forking is not the cheapest
syscall. Further, we could adapt the sleep interval to the next time
we actually need to run a job (and sending a SIGUSR to the daemon if
a job interval changes such, that this interval got narrower)

We try to sync running on minute-change boundaries at start, this
emulates systemd.timer behaviour, we had until now. Also user can
configure jobs on minute precision, so they probably expect that
those also start really close to a minute change event.
Could be adapted to resync during running, to factor in time drift.
But, as long as enough cpu cycles are available we run in correct
monotonic intervalls, so this isn't a must, IMO.

Another improvement could be locking a bit more fine grained, i.e.
not on a per-all-local-job-runs basis, but per-job (per-guest?)
basis, which would improve temporary starvement  of small
high-periodic jobs through big, less peridoci jobs.
We argued that it's the user fault if such situations arise, but they
can evolve over time without noticing, especially in compolexer
setups.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-10 16:11:00 +01:00