Patch tries to make nodestatusget really extendable. Following changes
are implemented:
- corosync_cfg_node_status_version_t is added with (for now) single
value CFG_NODE_STATUS_V1
- corosync_knet_node_status renamed to corosync_cfg_node_status_v1 (it
isn't really knet because it works as well for udp(u()
- struct res_lib_cfg_nodestatusget_version is added which holds only ipc
result header and version on same position as for
corosync_cfg_node_status_v1
- corosync_cfg_node_status_get requires version and pointer to one of
corosync_cfg_node_status_v structures
- request is handled in case switches to make adding new version easier
Also fix following bugs:
- totempg_nodestatus_get error was retyped to cs_error_t without any
meaning.
- header.error was not checked at all in the library
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Current we horribly over-use totempg_ifaces_get() to
retrieve information about knet interfaces. This is an attempt to
improve on that.
All transports are supported (so not only Knet but also UDP(U)).
This patch builds best against the "onwire-upgrade" branch of knet
as that's what sparked my interest in getting more information out.
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Previously only crypto cipher was used as a way to find out if crypto is
enabled or disabled.
This usually works ok until cipher is set to none and hash to some other
value (like sha1). Such config is perfectly valid and it was not
supported correctly.
As a solution, check both cipher and hash.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Fabio M. Di Nitto <fdinitto@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
At the same time simplify the overwrite logic and stop clearing the
umask (which is unexpected and quite pointless here, as applications
can't really protect the users from their own pathological settings).
Signed-off-by: Ferenc Wágner <wferi@debian.org>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
... when token and consensus timeouts pop.
Signed-off-by: Aleksei Burlakov <aburlakov@suse.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Default token timeout of 1000 ms was often changed by users because of
other workloads on machine which may make corosync responding a bit
later than needed and resulting in token loss.
3000 ms was chosen as a compromise between token timeout increase
and allow live cluster upgrade (other nodes should receive token
by node with new default on time).
It doesn't affect token token_coefficient so final token timeout still
depends on number of configured nodes (just base is higher).
This change slows down failover a bit so for clusters where failover
times are important, please change the token timeout in configuration
file corosync.conf as a:
totem {
version: 2
token: 1000
...
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Current quorum callback contains only actual view list and there is no
way how to find out joined/left nodes. This cannot be emulated by user
app, because when corosync restarts before other nodes notices then view
list is unchanged (ring id is changed tho).
Solution is to implement similar callback as for cpg which contains ring
id, member list, joined list and left list.
To implement such callback and keep backwards compatibility,
quorum_model_initialize is introduced. Its behavior is similar to
cpg_model_initialize. This allows passing model v1, which contains
enhanced quorum (full ring id is passed instead of just seq number)
and nodelist callbacks.
To find out which events should be sent by corosync daemon, new message
MESSAGE_REQ_QUORUM_MODEL_GETTYPE is used. Quorum library on init was
sending MESSAGE_REQ_QUORUM_GETTYPE. Whem model v1 is requested the
MESSAGE_REQ_QUORUM_MODEL_GETTYPE is used, which contains model number
so corosync knows that client is using model v1 and can send enhanced
quorum and nodelist events.
Nodelist event is (for now) send both in case of change of membership
and also when requested, also when CS_TRACK_CURRENT is requested, but
then left_list and joined_list is left empty, because they don't make
too much sense there.
New test application testquorummodel is added as an example of new API
usage.
Also during patch developement, I found few bugs here and there, which
are also fixed:
- quorum_initialize was never returning error code returned by
MESSAGE_REQ_QUORUM_GETTYPE call (always returned CS_OK)
- Allocated memory in send_library_notification was based
on sizeof(unsigned int) instead of mar_uint32_t. That's not wrong,
but it make more sense to use sizeof(mar_uint32_t) instead
(big thanks to Chrissie for englishify the man pages)
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Trigger token received callback only for valid token.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Check whether linknumber larger than INTERFACE_MAX and display error if
so.
Signed-off-by: liangxin1300 <XLiang@suse.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
While execute corosync-cfgtool -k <nodeid> to kill node:
* Check whether nodeid exists
* Check whether the node was joined
Signed-off-by: liangxin1300 <XLiang@suse.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Needs new knet crypto API.
If it's not available, then fall back to the old
API and forbid changing crypto while running.
To avoid us being dependant on the leader node, each
node sends its own crypto_reconfig_phase messages so
we can guarantee that the reconfiguration always completes
on each node.
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Have string values stored in char arrays in totem_config
so we don't get into a mess with the pointers.
Also remove vsftype (which hasn't been used since corosync 1)
Use strncpy even though we know the string is fine. Keep covscan happy
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
reload failed for UDP[U] because they had saved pointers
to the interfaces[] array. so memcpy into that rather then
re-allocate it.
Also, move the check for different IP address families so
it also gets run at reload time.
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Fix an 'error: success' stype message by propogating error_string
back down the stack.
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
To be more reliable & maintainable
The basic plan here is to fix reloads to be more stable
using read/parse/verify/build/commit stages, so that any errors
will not leave corosync in an unstable state. This should
also make the code more maintainable as currently the verify/commit
stages are horribly intertwined.
Also:
- Fix local_node_pos not being updated in the new map during validation
(broke adding and removing new nodes in the middle of the list).
- Fix reconfiguration so that nodes are indexed by nodeid and not their
position in the list. This is an old bug that's just been carried
over
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
This reverts commit 934c47ed43 which is
causing protocol incompatibility in needle. Master seems to be not
affected, but it needs more checking.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Previously reload of configuration with enabled wait_for_all result in
set of wait_for_all_status which set cluster_is_quorate to 0 but didn't
inform the quorum service so votequorum and quorum information may get
out of sync.
Example is 1 node cluster, which is extended to 3 nodes. Quorum service
reports cluster as a quorate (incorrect) and votequorum as not-quorate
(correct). Similar behavior happens when extending cluster in general,
but some configurations are less incorrect (3->4).
Discussed solution was to inform quorum service but that would mean
every reload would cause loss of quorum until all nodes would be seen
again.
Such behaviour is consistent but seems to be a bit too strict.
Proposed solution sets wait_for_all_status only on startup and
doesn't touch it during reload.
This solution fulfills requirement of "cluster will be quorate for
the first time only after all nodes have been visible at least
once at the same time." because node clears wait_for_all_status only
after it sees all other nodes or joins cluster which is quorate. It also
solves problem with extending cluster, because when cluster becomes
unquorate (1->3) wait_for_all_status is set.
Added assert is only for ensure that I haven't missed any case when
quorate cluster may become unquorate.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Previously value of new expected_votes was checked so newly computed
quorum value was in the interval <total_votes / 2, total_votes>. The
upper range prevented the cluster to become unquorate, but bottom check
was almost useless because it allowed to change expected_votes so it is
smaller than total_votes.
Solution is to check if expected_votes is bigger or equal to total_votes
and for quorate cluster only check if cluster doesn't become unquorate
(for unquorate cluster one can set upper range freely - as it is
perfectly possible when using config file)
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
This is useful for matching schedmiss event in stats map with logged
event.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
When user configure a specific interface like vlan
with the same IPv6 link-local address, Corosync should
compare sin6_scope_id with interface_num, to make sure got
the right interface to bind
Signed-off-by: liangxin1300 <XLiang@suse.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
sin6_scope_id was not present in totemip structure making impossible to
use link-local ipv6 address.
Patch adds sin6_scope_id and changes convert/copy functions to use it
(formally also comparator functions should be changed, but it seems to
cause more harm and it is not really needed).
This makes corosync work with link-local addresses fine for both UDPU
and Knet transport as long as interface specification is used (so
fe80::xxxx:xxxx:xxxx:xxxx%eth0).
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Totemknet is enhanced to use 'n' character for localhost and not adding
status, because it is safe to expect that localhost link is always
connectd. corosync-cfgtool is enhanced to properly decode 'n', '?' and
'd' characters and display its meaning for extended status. Special
characters are also documented in man page.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Using monotonic time is not working because it doesn't have to match
time from epoch.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
This patch add a stats.schedmiss.* set of entries that
are a record of the last 10 times corosync was not scheduled
in time.
These entries are keypt in reverse order (so stats.schedmiss.0.* is
always the latest one kept) and the values, including the timestamp,
are in milliseconds.
It's also possible to use a cmap tracker to follow these events, which
might be useful.
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
When 2Node mode is set, WFA is also set unless WFA is configured
explicitly. This behavior was not reflected on runtime change, so
restarted corosync behavior was different (WFA not set). Also when
cluster is reduced from 3 nodes to 2 nodes during runtime, WFA was not
set, what may result in two quorate partitions.
Solution is to set WFA depending on 2Node when WFA
is not explicitly configured.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Express intention to ignore icmap_get_* return
value and rely on default behavior of not changing the output
parameter on error.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Make the icmap*_r functions read from the specified map rather
than the global map.
Also include icmap_get_string_r() which seems to have been missed out.
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Some functions allocated memory on stack without clearing memory and
then send them on wire. This is not an issue, but valgrind reports this
as a problem so it is easy to miss real problem then.
Solution is to clear stack memory.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
commit 029b8ebad6 changed the default
of the KNET_PONG_COUNT from the kronosnet default of 5 to 2, as
corosync bring up was deemed to slow.
The documentation, and the comment stating that the totem config
default values match the knet ones were not updated, and thus now out
of date.
Fixhis by noting the correct default of 2 for KNET_PONG_COUNT and
note that all but that one are in sync with the korosync defaults in
the comment.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
