Commit Graph

599 Commits

Author SHA1 Message Date
Dominik Csapak
3e4c0f06e7 ceph: init: lock ceph config
so that not multiple users can create a ceph cluster at the same time

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-18 16:17:35 +02:00
Dominik Csapak
b3d8b5f527 ceph: init: check for nautilus
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-18 16:17:35 +02:00
Dominik Csapak
52fea280cf ceph: init: update inital config for nautilus
since we do not support creating filestore osds anymore, drop
the journal size from the config

and move the keyring from global to client
this makes it possible to omit the osd keyring path
(which was the default but got overwritten from the global section)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-18 16:17:35 +02:00
Thomas Lamprecht
7618913027 perl: fix some common typos found with codespell
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-15 10:40:20 +02:00
Alexandre Derumier
f31cc71c03 network api : reload : don't allow openswitch at all
ifupdown2 reload can't work with openswitch until we implement
ovs.

I don't think that too much users are mixing ovs && bridge anyway.

It's possible to use ifupdown2 with ovs for ifup/down with ifupdown script,
but config need to be changed, and I don't have tested too much.
(maybe add a conflict in ifupdown2 package with openvswitch package for now)

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2019-06-14 09:14:58 +02:00
Alexandre Derumier
feaf369e77 network api : reload : always reload
reload also if interfaces.new don't exist
(for vnet or other user change in different file)

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2019-06-14 09:14:58 +02:00
Alexandre Derumier
bf32b90e89 network api : reload : don't check bridge delete
we are doing it in ifupdown2 directly

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2019-06-14 09:14:58 +02:00
Alexandre Derumier
e46bf624d5 network api : reload : remove ifdown/up hack
This was for vxlan interfaces and fixed in ifupdown2 with my last patches.

simply reload network, and if we still have errors, we can use ifquery to check them later

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2019-06-14 09:14:58 +02:00
Thomas Lamprecht
f79372c0dc api: cluster/resources: add information about guest locks
using the new get_guest_config_property helper from pve-cluster,
which allows us to get this info with relatively low overhead.

With a somewhat realistic setup of 303 guest configurations here my
API call timing changes from ~ 24 to 26 ms without this to 26 to 28
ms with this patch applied, which seems reasonable.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-13 11:11:38 +02:00
Thomas Lamprecht
68f371d4d1 fixup: while not an error here a trailing semicolon is good to have
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-13 11:08:28 +02:00
Thomas Lamprecht
0f6e6f6b5c api network: make IPv6 address checking spew out useful errors
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-13 11:06:15 +02:00
Dominik Csapak
4ce045788a ceph: osd create: add encrypted as parameter
uses cpeh-volumes --dmcrypt parameter to encrypt the osd

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-11 12:58:24 +02:00
Dominik Csapak
46fb9c5017 ceph: a little luminous backwards compatibility
ceph luminous does not use the 'name' property in the metadata
everywhere, so fall back to 'id'

this makes the ceph dashboard usable while having still luminous
(relevant for upgrading)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-11 12:58:24 +02:00
Thomas Lamprecht
970f96fdbb api osd create: followup code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-06 13:43:32 +02:00
Dominik Csapak
45d45a63cd ceph: make ceph osd create api more readable
The aim of this patch is to reorder/rework the code of the api call
so that it gets more readable

it adds comments of what/why something is done, removes
code duplication between db/wal checks/creation

There are two changes in behaviour:
* when a device is given more than once via the api,
  the user gets a parameter exception for the db or wal
  with the information that the explicit defined devices must be
  different

* we check the usage for db/wal before the worker, so that the user
  gets instant feedback if a device is already in use
  (this is more for api users than for gui users, since we do those
  checks there also)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-06 12:41:14 +02:00
Dominik Csapak
3d7b3992dd ceph: osd create: add missing gpt check
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-05 18:23:49 +02:00
Dominik Csapak
ab62d137e1 ceph: osd create: round size down to the next kib
since the size of an LV can only be a multiple of 512b, we round
down to the next kib

we then have to mulitply it by 1024 for the partition, since
append_partition expects bytes and not kib

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-05 18:23:49 +02:00
Thomas Lamprecht
48983cdb8c api: create mgr: also have id in path for create
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-05 12:41:11 +02:00
Thomas Lamprecht
a91fa58ebd api: mgr create: code cleanup, redue heavy empty line usage
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-05 12:41:11 +02:00
Thomas Lamprecht
a435eaf9a0 api: create mon: also have monid in path for create
similar to the MDS api, so that DELETE and POST calls can operate on
the same path. This does not changes the CLI pveceph interface

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-05 12:41:11 +02:00
Thomas Lamprecht
7760a8beba api: mon create: code cleanup, redue heavy empty line usage
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-05 12:41:11 +02:00
Thomas Lamprecht
6dc136a178 api: ceph mgr list: check if active_name is really set
As in a situation where we /had/ a manager but destroyed it this
key's value is a empty string, and if we pass that to the WebUI we
get strange results form of a ghost MGR entry with ExtJS auto-ID
generation as name -> pretty confusing.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-05 12:41:11 +02:00
Thomas Lamprecht
b32e925587 api: osd destroy: try to remove PVs directly on the fly
no point in first building a list if we can just remove it directly
afterwards, it's eval-ed anyway and $osd_list did not get touched
in-between.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-05 10:50:54 +02:00
Thomas Lamprecht
5ebb945c3c api: osd destroy: pull out cleanup param
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-05 10:49:53 +02:00
Dominik Csapak
9b44d03dad ceph: osd: rework osd destroy to work with ceph-volume
with this, osd destruction is left to ceph-volume if the osd was created
with ceph-volume, else our old code remains mostly the same since
we want to be able to destroy upgraded osds

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-05 10:43:12 +02:00
Thomas Lamprecht
0154e79558 followup: api: osd create: code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-05 10:01:45 +02:00
Thomas Lamprecht
0e5f83badc api: osd create: use verbose_description and document defaults directly
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-05 10:01:01 +02:00
Thomas Lamprecht
afa09e02c7 fixup: ceph osd create: also put real UUID when adding a lv
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-05 09:46:38 +02:00
Dominik Csapak
7783f755e3 ceph: osd: rework creation with ceph-volume
this completely rewrites the ceph os creation api call using ceph-volume
since ceph-disk is not available anymore

breaking changes:
no filestore anymore, journal_dev -> db_dev

it is now possible to give a specific size for db/wal, default
is to read from ceph db/config and fallback is
10% of osd for block.db and 1% of osd for block.wal

the reason is that ceph-volume does not autocreate those itself
(like ceph-disk) but you have to create it yourself

if the db/wal device has an lvm on it with naming scheme 'ceph-UUID'
it uses that and creates a new lv

if we detect partitions, we create a new partition at the end

if the disk is not used at all, we create a pv/vg/lv for it

it is not possible to create osds on luminous with this api call anymore,
anyone needing this has to use ceph-disk directly

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-05 08:55:28 +02:00
Thomas Lamprecht
76bbe3e2d5 drop references to un-maintained sheepdog plugin
as already announced over two months ago[0], remove the unofficial
SheepDog plugin now completely. Besides that it was never fully
supported in Proxmox VE one of its main developer and ex-maintainer
declared it as abandoned[1], and thus just let's remove it, git
allows to resurrect it any time if a wonder happens anyway.

[0]: https://pve.proxmox.com/pipermail/pve-user/2019-March/170497.html
[1]: http://lists.wpkg.org/pipermail/sheepdog/2019-March/068449.html

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-04 17:45:38 +02:00
Dominik Csapak
da92b5fcce ceph: mon: use ceph_service_cmd to enable/disable the systemd service
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-04 17:07:43 +02:00
Dominik Csapak
919513b01b ceph: remove 'exclude-manager' api parameter
since we will have a seperate gui for the manager, we do not need this
anymore

this is a breaking api change

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-04 16:51:18 +02:00
Dominik Csapak
66fb085fc0 ceph: add an index for mgr services
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-04 16:45:04 +02:00
Dominik Csapak
d5373b7dc3 ceph: factor out the service info generation
and include a call to $type metadata to include the version

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-04 14:57:10 +02:00
Dominik Csapak
4e76dbd7b3 ceph: refactor broadcast_ceph_services and get_cluster_service
and use the broadcast when a service is added/removed
we will use 'get_cluster_service' in the future when we generate a list
of services of a specific type

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-04 14:56:24 +02:00
Dominik Csapak
2d7d6c9ac9 ceph: add missing link for 'restart' in ceph api
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-04 14:55:13 +02:00
Dominik Csapak
e02970235d gui/ceph: show versions in osd overview
and highlight the not current osds

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-05-31 15:45:48 +02:00
Dominik Csapak
6e251e550b add cluster wide ceph api calls
add two new api calls in /cluster/ceph

status:
the same as /nodes/NODE/ceph/status, but accessible without
nodename, which we don't need, as in the hyperconverged case, all nodes
have the ceph.conf which contains the info on how to connect to the
monitors

metadata:
combines data from the cluster filesystem about the services,
as well as the 'ceph YYY metadata' info we get from ceph.
with this info we can convieniently display which services exists,
which are running and which versions they have

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-05-27 16:17:20 +02:00
Thomas Lamprecht
180a86d39a pvecfg: adapt version and release semantic
Wit commit a74ba607d4 we switched over
to using the dpkg-dev provided helpers to set package version,
architecture and such in the buildsystem.

But unlike other repositories we used the version also for giving it
back over the API through the during build generated PVE::pvecfg
module, which wasn't fully updated to the new style.

This patch does that, and also cleans up semantics a bit, the
following two changed:

release is now the Debian release, instead of the "package release"
(i.e., the -X part of a full package version).
version is now simply the full (pve-manager) version, e.g., 6.0-1 or
the currently for testing used 6.0-0+1

This allows to do everything we used this information for even in a
slightly easier way (no  string concat needed anymore), and fits also
with the terminology we often used in our public channels (mailing
lists, forum, website)

Remove some cruft as we touch things.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-05-26 16:11:43 +02:00
Thomas Lamprecht
97ed02b8b6 api/ceph: create mon: drop now unsupported set-uid on autkey create
From Nautilus release changelog[0]:
> The auid property for cephx users and RADOS pools has been removed.
> This was an undocumented and partially implemented capability that
> allowed cephx users to map capabilities to RADOS pools that they
> “owned”. Because there are no users we have removed this support.

[0]: https://ceph.com/releases/v14-2-0-nautilus-released/

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-05-26 15:22:42 +02:00
Thomas Lamprecht
bba5c71217 ceph: drop systemd_managed - we now always are
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-05-26 13:35:39 +02:00
Thomas Lamprecht
fcb8022169 api apt versions: update package lists for PVE 6
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-05-24 13:23:46 +02:00
Thomas Lamprecht
961399a308 api/nodes journal: fix parameter types and add minimum
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-05-15 16:12:20 +02:00
Thomas Lamprecht
a03cb8b89e partially revert "api/nodes journal: add and enforce parameter conflicts"
This partially reverts commit f9b08743a5
as we had some wrong assumptions about lastentries and the other
params, so just note conflicts in the description but let the tool
itself make the checks

This reverts commit f9b08743a5.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-05-15 16:05:19 +02:00
Thomas Lamprecht
f9b08743a5 api/nodes journal: add and enforce parameter conflicts
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-05-15 15:16:22 +02:00
Thomas Lamprecht
952280b49a api/nodes: add journal to index
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-05-15 15:15:51 +02:00
Dominik Csapak
1d397a8394 node: add journal api
this uses the new journalreader instead of journalctl, which is a bit
faster and can read from/to cursor and returns a start/end cursor

also you can give an unix epoch as time parameters

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-05-15 12:09:26 +02:00
Dominik Csapak
69106e5cf3 add new parameters cidr(6) to the network api
to also allow to set it directly and not via address/netmask

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-05-08 14:56:40 +00:00
Thomas Lamprecht
fc1da3b020 api: handle undefined shellcmd parameter better
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-04-24 11:27:52 +00:00
Thomas Lamprecht
78c2d7f781 api: ceph/osd: conciser metadata array to hash mapping
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-04-24 10:24:21 +00:00