This patch adds support to change the default corosync pid file lock
path. This is useful to run corosync net namespace environment only and
since the pid lock file cannot be clarified over the conf because the
pid lock file exists before config parsing we allow the user to specify
it over the command line.
Signed-off-by: Alexander Aring <aahringo@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
As suggested by Christine Caulfield split long sentence so now paragraph
follows same formatting style as other options hopefully making
it less confusing.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
- Add default value 5405 for mcastport
- Add brief introduction for UDP/UDPU/KNET transport
- Keep format consistent (use uppercase) for above 3 transport types
Signed-off-by: xin liang <xliang@suse.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
The bits about IPv6 were out of date (for knet).
Added reference to the corosync-*tool utilities so that
people know they are there
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
totem.knet_mtu is new configuration option which allows setting
of automatic or manual knet MTU.
Also reload of totem.knet_pmtud_interval is fixed now, so it works when
key is deleted (and set back default value).
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
The consensus timeout is 1.2 * token_timeout,
which has been changeg from 1000 to 3000, so change also consensus
timeout.
Signed-off-by: miharahiro <hmihara@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Previously, existence of retransmit messages canceled holding
of token (and never allowed representative to enter token hold
state).
This makes token rotating maximum speed and keeps processor
resending messages over and over again - overloading network
and reducing chance to successfully deliver the messages.
Also there were reports of various Antivirus / IPS / IDS which slows
down delivery of packets with certain sizes (packets bigger than token)
what make Corosync retransmit messages over and over again.
Proposed solution is to allow representative to enter token hold
state when there are only retransmit messages. This allows network to
handle overload and/or gives Antivirus/IPS/IDS enough time scan and
deliver packets without corosync entering "FAILED TO RECEIVE" state and
adding more load to network.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Support for cgroup v2 is very similar to cgroup v1 just checking (and
writing) different file.
Because of all the problems described later with cgroup v2 new "auto"
mode (new default) is added. This mode first tries to set rr scheduling
and moves Corosync to root cgroup only if it fails.
Testing this feature is a bit harder than with cgroup v1 so it's
probably worh noting in this commit message.
1. Copy some service file (I've used httpd service) and set
CPUQuota=30% in the [service] section.
2. Check /sys/fs/cgroup/cgroup.subtree_control - there should be no
"cpu"
3. Start modified service
4. Check /sys/fs/cgroup/cgroup.subtree_control - there should be "cpu"
5. Start corosync - It should be able to get rt priority
When move_to_root_cgroup is disabled (applies only for kernels
with CONFIG_RT_GROUP_SCHED enabled), behavior differs:
- If corosync is started before modified service, so
there is no "cpu" in /sys/fs/cgroup/cgroup.subtree_control
corosync starts without problem and gets rt priority.
Starting modified service later will never add "cpu" into
/sys/fs/cgroup/cgroup.subtree_control (because corosync is holding
rt priority and it is placed in the non-root cgroup by systemd).
- When corosync is started after modified service, so "cpu"
is in /sys/fs/cgroup/cgroup.subtree_control, corosync is not
able to get RT priority.
It's worth noting problems when cgroup v2 is used together with systemd
logging described in corosync.conf(5) man page.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Internally knet is using just one link for localhost so for single node
configuration knet_link_get_link_list returns only one entry. This is
propagated to `corosync-cfgtool -s`.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Support for cgroup v2 is very similar to cgroup v1 just checking (and
writing) different file.
Testing this feature is a bit harder than with cgroup v1 so it's
probably worh noting in this commit message.
1. Copy some service file (I've used httpd service) and set
CPUQuota=30% in the [service] section.
2. Check /sys/fs/cgroup/cgroup.subtree_control - there should be no
"cpu"
3. Start modified service
4. Check /sys/fs/cgroup/cgroup.subtree_control - there should be "cpu"
5. Start corosync - It should be able to get rt priority
When move_to_root_cgroup is disabled, behavior differs:
- If corosync is started before modified service, so
there is no "cpu" in /sys/fs/cgroup/cgroup.subtree_control
corosync starts without problem and gets rt priority.
Starting modified service later will never add "cpu" into
/sys/fs/cgroup/cgroup.subtree_control (because corosync is holding
rt priority and it is placed in the non-root cgroup by systemd).
- When corosync is started after modified service, so "cpu"
is in /sys/fs/cgroup/cgroup.subtree_control, corosync is not
able to get RT priority.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Use knet_get_crypto_list to find knet supported crypto models and use
them instead of hardcoded list.
Also fix compression handling. Previously knet_compression_model
value was not checked at all and was directly passed to knet.
Use knet_get_compress_list to find knet supported compress models and
use them to check validity of config file and for more informative
error message.
Lastly enhance corosync version display with information
about available crypto/compression models.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Apostrophe as the first character of the input line indicates a
request, so groff complained: macro 'onwire'' not defined.
Signed-off-by: Ferenc Wágner <wferi@debian.org>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Retry knet_handle_new without privileged operations if it fails
knet_handle_new can fail with ENAMETOOLONG if its privileged operations
fail, which can happen if we're running as a user process or in an
unprivileged container.
This adds a cmap key 'allow_knet_handle_fallback' that defaults to no,
which is the current behavior of exiting with error if the knet_handle
can't be created with privileged operations. If the new cmap key is set
to 'yes' and the knet_handle creation fails, fallback to creating the
handle using unprivileged operations is tried.
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
CFG tracking was removed in 815375411e,
probably as a mistake, as part of the tidy up of cfg and the removal of
dynamic loading. This means that shutdown tracking (using
cfg_try_shutdown()) stopped working.
This patch restores the trackstart & trackstop API calls (renamed to be
more consistent with the exiting libraries) so that shutdown tracking
can be used again.
Change cfg.shutdown_timeout to be in milliseconds rather than seconds
nd use libqb macros for conversion.
Add --force option to corosync-cfgtool -H
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Current we horribly over-use totempg_ifaces_get() to
retrieve information about knet interfaces. This is an attempt to
improve on that.
All transports are supported (so not only Knet but also UDP(U)).
This patch builds best against the "onwire-upgrade" branch of knet
as that's what sparked my interest in getting more information out.
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Default token timeout of 1000 ms was often changed by users because of
other workloads on machine which may make corosync responding a bit
later than needed and resulting in token loss.
3000 ms was chosen as a compromise between token timeout increase
and allow live cluster upgrade (other nodes should receive token
by node with new default on time).
It doesn't affect token token_coefficient so final token timeout still
depends on number of configured nodes (just base is higher).
This change slows down failover a bit so for clusters where failover
times are important, please change the token timeout in configuration
file corosync.conf as a:
totem {
version: 2
token: 1000
...
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Backtick and apostrophe are formatted as directional quotes by plain
groff, but they behave literally in the body of a man page.
Signed-off-by: Ferenc Wágner <wferi@debian.org>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
By slightly rewording the documentation of knet_compression_model.
Signed-off-by: Ferenc Wágner <wferi@debian.org>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Current quorum callback contains only actual view list and there is no
way how to find out joined/left nodes. This cannot be emulated by user
app, because when corosync restarts before other nodes notices then view
list is unchanged (ring id is changed tho).
Solution is to implement similar callback as for cpg which contains ring
id, member list, joined list and left list.
To implement such callback and keep backwards compatibility,
quorum_model_initialize is introduced. Its behavior is similar to
cpg_model_initialize. This allows passing model v1, which contains
enhanced quorum (full ring id is passed instead of just seq number)
and nodelist callbacks.
To find out which events should be sent by corosync daemon, new message
MESSAGE_REQ_QUORUM_MODEL_GETTYPE is used. Quorum library on init was
sending MESSAGE_REQ_QUORUM_GETTYPE. Whem model v1 is requested the
MESSAGE_REQ_QUORUM_MODEL_GETTYPE is used, which contains model number
so corosync knows that client is using model v1 and can send enhanced
quorum and nodelist events.
Nodelist event is (for now) send both in case of change of membership
and also when requested, also when CS_TRACK_CURRENT is requested, but
then left_list and joined_list is left empty, because they don't make
too much sense there.
New test application testquorummodel is added as an example of new API
usage.
Also during patch developement, I found few bugs here and there, which
are also fixed:
- quorum_initialize was never returning error code returned by
MESSAGE_REQ_QUORUM_GETTYPE call (always returned CS_OK)
- Allocated memory in send_library_notification was based
on sizeof(unsigned int) instead of mar_uint32_t. That's not wrong,
but it make more sense to use sizeof(mar_uint32_t) instead
(big thanks to Chrissie for englishify the man pages)
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Make it clear that reloads during a rolling upgrade are not
supported.
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Some users found description of priority for passive link_mode
confusing (probably because "priority" word is too
overloaded) so add some redundancy to make description
unambiguous.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Totemknet is enhanced to use 'n' character for localhost and not adding
status, because it is safe to expect that localhost link is always
connectd. corosync-cfgtool is enhanced to properly decode 'n', '?' and
'd' characters and display its meaning for extended status. Special
characters are also documented in man page.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
This patch add a stats.schedmiss.* set of entries that
are a record of the last 10 times corosync was not scheduled
in time.
These entries are keypt in reverse order (so stats.schedmiss.0.* is
always the latest one kept) and the values, including the timestamp,
are in milliseconds.
It's also possible to use a cmap tracker to follow these events, which
might be useful.
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Section 8 is for "System administration commands", 7 is "Miscellaneous".
Signed-off-by: Ferenc Wágner <wferi@debian.org>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
commit 029b8ebad6 changed the default
of the KNET_PONG_COUNT from the kronosnet default of 5 to 2, as
corosync bring up was deemed to slow.
The documentation, and the comment stating that the totem config
default values match the knet ones were not updated, and thus now out
of date.
Fixhis by noting the correct default of 2 for KNET_PONG_COUNT and
note that all but that one are in sync with the korosync defaults in
the comment.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
corosync_cfg_ring_status_get returns string status, which is always OK
for UDP(U) and detailed status for Knet transport. Previously also
FAULTY status was returned for UDP(U) and cfgtool used to return error
code back to shell when one of the interfaces was faulty.
Because FAULTY is now not returned, it's not needed to have code for
handling it.
Also man page was misleading, so it is fixed too.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Thanks Christine Caulfield <ccaulfie@redhat.com> for
Englishify and refining the description.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Fix issues found by Ulrich Windl <Ulrich.Windl@rz.uni-regensburg.de>
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
This feature allows corosync to block packets received from unknown
nodes (nodes with IP address which is not in the nodelist). This is
mainly for situations when "forgotten" node is booted and tries to join
cluster which already removed such node from configuration. Another use
case is to allow atomic reconfiguration and rejoin of two separate
clusters.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
currently knet acl are only available on master
but they might be backported
to stable1 as they don´t break onwire protocol.
Signed-off-by: Fabio M. Di Nitto <fdinitto@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
1. Enable scripting of vqsim and add man page
I've added a 'sleep' command to help with scripting as well as
documentation on how to do it.
2. Make 'sync' operation much more robust and useful
Refactored a lot of code to make sure that in sync mode the
prompt appears at the 'right' time. What we do is wait for all
of the nodes in all partitions to have the same ring_id. If this
doesn't happen then the timeout will fire as before.
3. Rename binary to corosync-vqsim and add a sub-package for it
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Previously the reason field for the member_list items
in cpg_totem_confchg_fn was unset what may be little confusing.
Solution is to add a special value CPG_REASON_UNDEFINED and use it for
the member_list items.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
with the following semantics:
- default off
- implies crypto_hash SHA256 and crypto_cipher AES256
- crypto_* have higher precedence
- only applicable for knet, like crypto_*
this should make upgrading from Corosync 2.x less painful for users that
have an explicit secauth=on in their configuration.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Triple DES is considered as a "weak cipher" since 2016 so there is
really no need to support it in the corosync. Thanks to bug in
Corosync/Knet/NSS which caused 3des to not work at all,
no matter what library was used, we can just remove support for 3des
without braking the compatibility.
Also fix coroparse so:
- totem.crypto_type is removed (this is 1.x construct which was not used
even in 2.x)
- Add checking of totem.crypto_model.
- Enumarate possible values for crypto_model, crypto_cipher and
crypto_hash error messages
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Knet commit 1cb36f0cffd4559971826ca4774a88c5b05882fb reduced minimal
key length to 1024-bit. Keygen should keep compatibility with already
released 3.0.[0-1] so default key length should be 2048 bits. It's
possible to use -s argument to generate shorter key - keygen respects
minimum/maximum as defined by knet.
Also fix man page to reflect this change.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
A nozzle device is a pseudo ethernet device that routes network
traffic through a channel on the corosync knet network (NOT cpg or any
corosync internal service) to other nodes in the cluster. It allows
applications to take advantage of knet features such as multipathing,
automatic failover, link switching etc.
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
1. Use EXIT_SUCCESS and EXIT_FAILURE when possible
2. For -s option return EXIT_SUCCESS when no problem appeared and node
is quorate, EXIT_FAILURE if problem appeared and exit code 2
(EXIT_NOT_QUORATE) when no problem appeared but node is not quorate.
3. Document exit codes in the man page
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Previously it was required to use link id together with IP address (ex.
"0 127.0.0.1") as a -i parameter.
This was reported as not very user friendly. Solution is to split
returned interface name and try match link id and ip address
separately.
Also fix typo in description of parameter -s.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
