ceph: follow up rewords & fixes

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
This commit is contained in:
Thomas Lamprecht 2019-04-04 14:31:31 +02:00
parent 76f6eca4ee
commit 2f19a6b0b6

View File

@ -79,17 +79,16 @@ Check also the recommendations from
http://docs.ceph.com/docs/luminous/start/hardware-recommendations/[Ceph's website]. http://docs.ceph.com/docs/luminous/start/hardware-recommendations/[Ceph's website].
.CPU .CPU
As higher the core frequency the better, this will reduce latency. Among other Higher CPU core frequency reduce latency and should be preferred. As a simple
things, this benefits the services of Ceph, as they can process data faster. rule of thumb, you should assign a CPU core (or thread) to each Ceph service to
To simplify planning, you should assign a CPU core (or thread) to each Ceph provide enough resources for stable and durable Ceph performance.
service to provide enough resources for stable and durable Ceph performance.
.Memory .Memory
Especially in a hyper-converged setup, the memory consumption needs to be Especially in a hyper-converged setup, the memory consumption needs to be
carefully monitored. In addition to the intended workload (VM / Container), carefully monitored. In addition to the intended workload from virtual machines
Ceph needs enough memory to provide good and stable performance. As a rule of and container, Ceph needs enough memory available to provide good and stable
thumb, for roughly 1TiB of data, 1 GiB of memory will be used by an OSD. With performance. As a rule of thumb, for roughly 1 TiB of data, 1 GiB of memory
additionally needed memory for OSD caching. will be used by an OSD. OSD caching will use additional memory.
.Network .Network
We recommend a network bandwidth of at least 10 GbE or more, which is used We recommend a network bandwidth of at least 10 GbE or more, which is used
@ -97,15 +96,14 @@ exclusively for Ceph. A meshed network setup
footnote:[Full Mesh Network for Ceph {webwiki-url}Full_Mesh_Network_for_Ceph_Server] footnote:[Full Mesh Network for Ceph {webwiki-url}Full_Mesh_Network_for_Ceph_Server]
is also an option if there are no 10 GbE switches available. is also an option if there are no 10 GbE switches available.
To be explicit about the network, since Ceph is a distributed network storage, The volume of traffic, especially during recovery, will interfere with other
its traffic must be put on its own physical network. The volume of traffic services on the same network and may even break the {pve} cluster stack.
especially during recovery will interfere with other services on the same
network.
Further, estimate your bandwidth needs. While one HDD might not saturate a 1 Gb Further, estimate your bandwidth needs. While one HDD might not saturate a 1 Gb
link, a SSD or a NVMe SSD certainly can. Modern NVMe SSDs will even saturate 10 link, multiple HDD OSDs per node can, and modern NVMe SSDs will even saturate
Gb of bandwidth. You also should consider higher bandwidths, as these tend to 10 Gbps of bandwidth quickly. Deploying a network capable of even more bandwith
come with lower latency. will ensure that it isn't your bottleneck and won't be anytime soon, 25, 40 or
even 100 GBps are possible.
.Disks .Disks
When planning the size of your Ceph cluster, it is important to take the When planning the size of your Ceph cluster, it is important to take the
@ -114,16 +112,21 @@ might take long. It is recommended that you use SSDs instead of HDDs in small
setups to reduce recovery time, minimizing the likelihood of a subsequent setups to reduce recovery time, minimizing the likelihood of a subsequent
failure event during recovery. failure event during recovery.
In general SSDs will provide more IOPs then spinning disks. This fact and the In general SSDs will provide more IOPs than spinning disks. This fact and the
higher cost may make a xref:pve_ceph_device_classes[class based] separation of higher cost may make a xref:pve_ceph_device_classes[class based] separation of
pools appealing. Another possibility to speedup OSDs is to use a faster disk pools appealing. Another possibility to speedup OSDs is to use a faster disk
as journal or DB/WAL device, see xref:pve_ceph_osds[creating Ceph OSDs]. If a as journal or DB/WAL device, see xref:pve_ceph_osds[creating Ceph OSDs]. If a
faster disk is used for multiple OSDs, a proper balance between OSD and WAL / faster disk is used for multiple OSDs, a proper balance between OSD and WAL /
DB (or journal) disk must be selected, otherwise the faster disk becomes the DB (or journal) disk must be selected, otherwise the faster disk becomes the
bottleneck for all linked OSDs. bottleneck for all linked OSDs.
Aside from the disk type, Ceph best performs with an even sized and distributed Aside from the disk type, Ceph best performs with an even sized and distributed
amount of disks per node. For example, 4x disks à 500 GB in each node. amount of disks per node. For example, 4 x 500 GB disks with in each node is
better than a mixed setup with a single 1 TB and three 250 GB disk.
One also need to balance OSD count and single OSD capacity. More capacity
allows to increase storage density, but it also means that a single OSD
failure forces ceph to recover more data at once.
.Avoid RAID .Avoid RAID
As Ceph handles data object redundancy and multiple parallel writes to disks As Ceph handles data object redundancy and multiple parallel writes to disks
@ -137,8 +140,8 @@ the ones from Ceph.
WARNING: Avoid RAID controller, use host bus adapter (HBA) instead. WARNING: Avoid RAID controller, use host bus adapter (HBA) instead.
NOTE: Above recommendations should be seen as a rough guidance for choosing NOTE: Above recommendations should be seen as a rough guidance for choosing
hardware. Therefore, it is still essential to test your setup and monitor hardware. Therefore, it is still essential to adapt it to your specific needs,
health & performance. test your setup and monitor health and performance continuously.
[[pve_ceph_install]] [[pve_ceph_install]]