Commit Graph

937 Commits

Author SHA1 Message Date
Aaron Lauterer
fca1900c76 api: ceph pools: add type to returned properties
The osd dump already contains the pool type in numerical format.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-02 15:43:11 +02:00
Thomas Lamprecht
ec63b237de api: ceph: fix description indentation style
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-29 14:28:12 +02:00
Thomas Lamprecht
07316e6d04 api: followup: code locality
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-29 14:26:32 +02:00
Aaron Lauterer
4605d6fd22 api: ceph ec pools: make add_storages overridable default
The behavior of always adding the storage config was lost in commit
23c407e. But it is more sensible to make it a default that can be
changed if needed.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2022-04-29 14:24:04 +02:00
Aaron Lauterer
136f761ba7 api: ceph ec pools: schema fixes and enhancements
Ceph has a min value for 'k' of 2. Adding default and description where
missing.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2022-04-29 14:24:04 +02:00
Thomas Lamprecht
f35e7fcd8e api: ceph ec pools: move to format-str, create ec in worker, reuse $rados
moved to a format string 'erasurce-coded', that allows also to drop
most of the param existence checking as we can set the correct
optional'ness in there.  Also avoids bloating the API to much for
just this.

Reuse the $rados connection more often to avoid to much
overhead/lingering sockets (the rados connection stays around in the
background to allow efficient reuse)

really should be three separate commits, but too intertwined and too
late for me to care tbh.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-28 20:26:38 +02:00
Aaron Lauterer
3bd128d7a0 ceph pools: allow to create erasure code pools
To use erasure coded (EC) pools for RBD storages, we need two pools. One
regular replicated pool that will hold the RBD omap and other metadata
and the EC pool which will hold the image data.

The coupling happens when an RBD image is created by adding the
--data-pool parameter. This is why we have the 'data-pool' parameter in
the storage configuration.

To follow already established semantics, we will create a 'X-metadata'
and 'X-data' pool. The storage configuration is always added as it is
the only thing that links the two together (besides naming schemes).

Different pg_num defaults are chosen for the replicated metadata pool as
it will not hold a lot of data.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2022-04-28 20:26:38 +02:00
Aaron Lauterer
29fe1eea7a api: ceph: $get_storages check if data-pool too
When removing a pool, we check against any storage that might have that
pool configured.
We need to check if that pool is used as data-pool too.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2022-04-28 20:26:38 +02:00
Fabian Grünbichler
c971330601 api2: network: anybridge: re-add regular bridges
commit 89d146f207 introduced permission
checks here that caused all regular bridges to be removed from the
returned list as soon as the SDN package is installed, unless the user
is root@pam or there exists a VNET with the same ID.

this is arguably a breaking change, so limit the priv check to actually
defined VNETs for the time being, and add ALL regular bridges
uncondtionally like before.

get_local_vnets already filters by the same prvs, so we need to get the
full config to find out which IDs are VNETs and which are not.

once/iff we introduce ACL paths for *all* bridges in the future, we can
limit accordingly here.

CC: Alexandre Derumier <aderumier@odiso.com>
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-04-27 13:55:57 +02:00
Thomas Lamprecht
ca65e09904 api: next-id: honor new datacenter.cfg upper/lower range option
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-27 13:52:01 +02:00
Thomas Lamprecht
5ac1eaa041 replication: split out error handling and include more info in mail
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-27 10:18:30 +02:00
Thomas Lamprecht
fa4bb659a3 replication: sent always mail for first three tries and move helper
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-27 09:48:28 +02:00
Fabian Ebner
237f00be03 api: replication: don't send mails about failed replication only once
but rather multiple times becoming exponentially less frequent.

Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-26 16:09:43 +02:00
Dominik Csapak
81ba0803d5 fix #3976: api/backup: make schedule/starttime truly optional on update
on create we require either starttime (+dow) or a schedule, but when
updating an existing job, this is not necessary

before we changed to schedules, the starttime was not optional either on
update, but i think there is no reason to require the user to send the
schedule/startime along every time.

the gui will send all values every time, so that was never a problem there

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-04-01 17:52:10 +02:00
Fabian Ebner
c53d5c5ec1 api: vzdump: extract config: add content type check
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-01 09:24:35 +02:00
Fabian Ebner
0bd224e56b api: vzdump: extract config: check for VM.Backup privilege
In preparation to have check_volume_access() always allow access for
users with Datastore.Allocate privilege. As to not automatically give
all such users permission to extract the config too.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-01 09:24:35 +02:00
Thomas Lamprecht
42f82359c1 api: cluster: refactor and cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-03-28 14:04:35 +02:00
Wolfgang Bumiller
fd0a63153e api: fill in nodeconfig schema
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-17 14:24:20 +01:00
Wolfgang Bumiller
9ee9991032 switch to Proxmox::RS::CalendarEvent
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-17 14:02:16 +01:00
Thomas Lamprecht
7851068108 api: sdn: code/whitespace refactor/fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-03-16 17:00:50 +01:00
Alexandre Derumier
89d146f207 api2 : network: anybridge: don't display bridges if user have access to vnets.
This remove vmbr* from bridgeselector if user have access to vnets.
if user need to have also access to vmbr, we can add a permission
in path "/sdn/vnets/vmbrX"

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2022-03-16 16:43:42 +01:00
Fabian Ebner
d3cd3c9ca3 api: apt: repos: fix interfacing with perlmod
Using
    pvesh create /nodes/pve701/apt/repositories --path
    "/etc/apt/sources.list" --index 0 --enabled 1
reliably leads to
    error: invalid type: string "0", expected usize

Coerce to int to avoid this. I was not able to trigger the issue with
the "enabled" option being a string here (in PMG I was), but be on the
safe side and coerce there too. Otherwise it might get triggered by a
future, completely unrelated change further up in the API call
handling.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-02-17 14:34:18 +01:00
Oguz Bektas
dfde1eec16 api: query url metadata: change default user-agent to "Proxmox VE"
To avoid being blacklisted because of the default, quite popular,
libwww-perl user-agent like reported in community forum [0].

[0]: https://forum.proxmox.com/threads/104081/

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
Tested-by: Matthias Heiserer <m.heiserer@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-04 17:49:46 +01:00
Fabian Ebner
38496d8758 api: apt: list_updates: fix dependency detection for virtual package
A virtual package does not have SelectedState Install, but the
dependency will still be satisfied if a package providing it has.

Fixes a bug, wrongly showing that postfix will be installed, when a
different mail-transport-agent is installed and a pve-manager update
is available:
https://forum.proxmox.com/threads/103413/

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-02-04 17:44:03 +01:00
Hannes Laimer
09ba41362c api: add 'type' parameter to pool GET endpoint
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-01-31 13:30:50 +01:00
Dominik Csapak
c048ad30c8 vzdump: add new 'next-run' field for vzdump job listing
and calculate it by getting the next event after 'now' since
we currently have no way to get the last run time for jobs only running
on different cluster nodes

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-01-13 15:33:41 +01:00
Dominik Csapak
68df949690 api: journal: stream the journal data to the client
instead of accumulating the whole output of 'mini-journalreader' in
the api call (this can be quite big), use the download mechanic of the
http-server to stream the output to the client.

we lose some error handling possibilities, but we do not have
to allocate anything here, and since perl does not free memory after
allocating[0] this is our desired behaviour.

to keep api compatiblitiy, we need to give the journalreader the '-j'
flag to let it output json.

also tell the http server that the encoding is gzip and pipe
the output through it.

0: https://perldoc.perl.org/perlfaq3#How-can-I-free-an-array-or-hash-so-my-program-shrinks?

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-24 18:19:16 +01:00
Thomas Lamprecht
5cc0c3a008 api: backup: code reduction
had it lying around and did not felt condensed/code-golfed to me,
rather a bit more expressive (surely biased though)..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-17 15:57:55 +01:00
Thomas Lamprecht
de92b4d783 api: backup: add comment regarding normalizing DOW list
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-17 15:36:06 +01:00
Dominik Csapak
307b2608a5 api: backup: normalize 'dow' format when converting
the old web ui sends the days as seperate parameters, which will
be concatenated by a null-byte in the api, causing it to land it this
way in the jobs.cfg

to fix this, split+join the list to get a well-formed dow list

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-17 15:22:45 +01:00
Fabian Grünbichler
1c87d3447c fix use statement
else this happens:

"Use of inherited AUTOLOAD for non-method PVE::API2::Backup::uuid() is
no longer allowed at /usr/share/perl5/PVE/API2/Backup.pm line 198.
(500)"

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-11-17 15:07:12 +01:00
Dominik Csapak
ca62f164cd api: backup: correclty use uuid package
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-17 15:07:12 +01:00
Thomas Lamprecht
2b42965020 api: apt/versions: track swtpm
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-15 16:35:08 +01:00
Dominik Csapak
5964deeae3 api: metrics: use complete config for testing the connection
not only the given parameters, e.g. at the moment, the gui will
never send a 'verify-certificate' parameter, even if set in the config

by using the complete resulting config, we test the actual settings.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-15 16:17:51 +01:00
Dominik Csapak
7aa5b131ff api/services: add pvescheduler to the service list
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-12 16:17:36 +01:00
Dominik Csapak
b188fcd8d4 api/services: sort service list
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-12 16:17:36 +01:00
Fabian Ebner
cffeb11592 api: ceph: create osd: set correct partition type
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-11-11 21:50:51 +01:00
Fabian Ebner
5161a0c2f0 partially fix #2285: api: ceph: create osd: allow using partitions
Note that this does not only allow partitions to be used, but for DB
and WAL disks, one more type of disk, that wasn't allowed before.
Namely, GPT-partitioned disks with any partitions detected as used.
The reason is get_disks' behavior:
  * Without $include_partitions=1, the disk will have the same usage
    as it's first used partition, and thus wasn't allowed. (Except in
    the case that usage was LVM, where the check was bypassed, but
    luckily OSD creation just failed later because no Ceph volume
    group would be detected).
  * With $include_partitions=1, the disk will have usage 'partitions'
    and thus be allowed.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-11-11 21:50:38 +01:00
Fabian Ebner
46b1ccc357 api: ceph: create osd: set correct parttype for DB/WAL
The get_ceph_journals function in pve-storage uses this information.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-11-11 21:50:33 +01:00
Thomas Lamprecht
c20699493a api: cluster: add jobs sub-directory index and set its permissions
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-11 21:02:03 +01:00
Thomas Lamprecht
7cced7126b api: cluster: sort index
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-11 21:02:03 +01:00
Thomas Lamprecht
584a57ea3c calendar event simulator: rename number parameter to iterations
in sync with the same option of `systemd-analyze calendar` and number
is so generic.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-11 20:53:13 +01:00
Dominik Csapak
e190bc2cc8 api: cluster: add jobs/schedule-analyze api call
a simple api call to simulate calendar event triggers
takes a schedule, an optional number (default 10), an optional starttime
(default 'now') and returns a list with unix timestamps, as well as
humanly readable utc timestamps.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 18:35:52 +01:00
Dominik Csapak
6ec3dc437b api: cephfs: add 'fs-name' for cephfs storage
so that we can uniquely identify the cephfs (in case of multiple)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 17:52:08 +01:00
Dominik Csapak
352a0e5c93 api: cephfs: add fs_name to 'is mds active' check
so that we check the mds for the correct cephfs we just added

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 17:52:08 +01:00
Dominik Csapak
028af34e3d api: cephfs: more checks on fs create
namely if the fs is already existing, and if there is currently a
standby mds that can be used for the new fs
previosuly, only one cephfs was possible, so these checks were not
necessary. now with pacific, it is possible to have multiple cephfs'
and we should check for those.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 17:52:08 +01:00
Dominik Csapak
0ab69d6e88 api: cephfs: refactor {ls, create}_fs
no function change intended

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-11 17:52:08 +01:00
Dominik Csapak
998b61fbff api: backup/jobs: add comment field to jobs
and encode them with PVE::Tools::encode_text in the config

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-10 16:11:42 +01:00
Dominik Csapak
305921b1ab api/backup: handle new vzdump jobs
in addition to listing the vzdump.cron jobs, also list from the
jobs.cfg file.

updates/creations go into the new jobs.cfg only now
and on update, starttime+dow get converted to a schedule
this transformation is straight forward, since 'dow'
is already in a compatible format (e.g. 'mon,tue') and we simply
append the starttime (if any)

id on creation is optional for now (for api compat), but will
be autogenerated (uuid). on update, we simply take the id from before
(the ids of the other entries in vzdump.cron will change but they would
anyway)

as long as we have the vzdump.cron file, we must lock both
vzdump.cron and jobs.cfg, since we often update both

we also change the backupinfo api call to read the jobs.cfg too

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-10 16:11:00 +01:00
Dominik Csapak
85b9ba88f1 api/backup: refactor string for all days
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-10 16:11:00 +01:00