If one node is paused it can miss a config change and
thus report a larger old_members than expected.
The solution is to use the left_nodes field.
Master selection used to be "choose node with":
1) largest previous membership
2) (then as a tie-breaker) node with smallest nodeid
New selection:
1) largest (previous #nodes - #nodes know to have left)
2) (then as a tie-breaker) node with smallest nodeid
Signed-off-by: Angus Salkeld <asalkeld@redhat.com>
memb_state_gather_enter increase stats.continuous_gather only if
previous state was gather also. This should happen only if multicast is
not working properly (local firewall in most cases) and not if many
nodes joins at one time.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Angus Salkeld <asalkeld@redhat.com>
If library connect to service with no init function, coroipcs will try
to dereference NULL pointer. Now we correctly return error code
CS_ERR_NOT_EXIST.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Steven Dake <sdake@redhat.com>
Without refcounting the conn pointer here, corosync will segfault
if one kills a running instance of "corosync-cfgtool -r" (rhbz#695191)
Signed-off-by: Tim Serong <tserong@novell.com>
Reviewed-by: Steven Dake <sdake@redhat.com>
Align all ipc messages on 8 byte boundaries. This alignment will remove bus
errors on systems that can't access non-byte aligned data and should improve
performance.
Signed-off-by: Steven Dake <sdake@redhat.com>
Reviewed-by: Angus Salkeld <asalkeld@redhat.com>
This is to prevent nasty deadlocks between IPC and objdb.
Signed-off-by: Angus Salkeld <asalkeld@redhat.com>
Reviewed-by: Steven Dake <sdake@redhat.com>
A negative value for the message type (on systems where char is signed)
would cause a crash. This is highly probable if the cluster is, for example,
misconfigured to have encryption enabled on some nodes but not others.
Signed-off-by: Zane Bitter <zane.bitter@gmail.com>
Reviewed-by: Steven Dake <sdake@redhat.com>
If you are connected to corosync and registered for
object notifications then corosync is asked to shutdown
the IPC server will get stuck. This is because the pipe
is closed and the refcount is increased. This leaves ipcs
with a connection that it can't destroy.
Solution:
1) if a write to the pipe fails (pipe closed) decrement the refcounter.
2) fix the object_track_stop() - it was not working as the functions
did not match up. (this caused the late callbacks).
3) in ipcs call exit_fn() then stats_destroy_connection() so that
the service engine can have time to call object_track_stop()
before the object gets destroyed.
Signed-off-by: Angus Salkeld <asalkeld@redhat.com>
Reviewed-by: Steven Dake <sdake@redhat.com>
This helps to quickly identify what service the application
is connected to.
The object will now look like:
runtime.connections.corosync-objctl:CONFDB:19654:13.service_id=11
runtime.connections.corosync-objctl:CONFDB:19654:13.client_pid=19654
etc...
This also makes it clearer to receivers of the dbus/snmp events
what is going on.
Signed-off-by: Angus Salkeld <asalkeld@redhat.com>
Reviewed-by: Steven Dake <sdake@redhat.com>
Zero element array behavior is very different from normal array or
pointer. This behavior is root of problem in not returning correctly
filled array of addresses. This appeared only in rrp mode, where more
then one address is returned.
All memcpy's are now correctly converted to copy pointer to char.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Steven Dake <sdake@redhat.com>
Relying on messages_free may seem like it should work, but it leads to a
situation where every node has released the messages, yet some nodes think
messages are missing. The output then looks like "Retransmit: #" in
repitition. This patch frees those messages immediately during the transition
to the OPERATIONAL state and sets the internal variables totemsrp depends
upon to the proper values.
Signed-off-by: Steven Dake <sdake@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
The current code stores the current ring information every time a commit
token is generated. This causes the old ring id used for comparison purposes
to increase if a token is lost in commit or recovery, resulting in failure of
totem. This patch changes the behavior to only store the old ring id one
time when the commit token is received, and then further commit token ring
id saves are not done until OPERATIONAL is reached.
Signed-off-by: Steven Dake <sdake@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
The recv_flush code is no longer necessary because of the miss_count_count
addition. It can in some cases lead to register corruption because of
interactions with -fstack-protector, the recursive nature of how this code
works, and interactions with the optimizer in some versions of gcc.
Signed-off-by: Steven Dake <sdake@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
corosync-notifyd has exposed an issue with confdb notifications.
The normal state of affairs is:
IPC thread > lock > objdb > lock
objdb notification whilst really useful turn things around:
<middle of big call chain>
objdb > lock > confdb > ipc > lock
This reverse ordering of locks causes a horrible dead lock.
I see this patch as a work around until corosync-2.0
when most of the threads and locking disappear.
This patch adds a pipe to confdb service. When we get a
objdb notification a struct gets written to the pipe.
The poll loop then runs the dispatch in the main thread.
In the dispatch we call the real ipc_dispatch_send().
Signed-off-by: Angus Salkeld <asalkeld@redhat.com>
Reviewed-by: Steven Dake <sdake@redhat.com>
consider 5 nodes.
node 3,4 stopped (by random stopping) node 1,2,5 form new configuration
and during recovery node 1 and node 2 are stopped (via service service
corosync stop). This causes 5 never to finish recovery within the timeout
period, triggering a token loss in recovery. Bug #623176 resolved an assert
which happens because the full ring id was being restored. The resolution
to Bug #623176 was to not restore the full ring id, and instead operate
(according to specifications) the new ring id. Unfortunately this exposes
a problem whereby the restarting of nodes 1-4 generate the same ring id.
This ring id gets to the recovery failed node 5 which is now in gather,
and triggers a condition not accounted for in the original totem specification.
It appears later work from Dr. Agarwal's PHD dissertation considers this
scenario. That solution entails rejecting the regular token in the above
condition. Since the ring id is also used to make decisions for commit token
acceptance, we must also take care to reject the regular token in all cases
after transitioning from OPERATIONAL.
Signed-off-by: Steven Dake <sdake@redhat.com>
Reviewed-by: Steven Dake <sdake@redhat.com>
This is avoid getting stuck in the dispatch processing
messages when the user is trying to shutdown the service.
Signed-off-by: Angus Salkeld <asalkeld@redhat.com>
Reviewed-by: Steven Dake <sdake@redhat.com>
1) both IPv4 and IPv6 mcast should default to ttl=1
2) the range should be 0..255
0 is valid meaning localhost only (cluster of one)
Signed-off-by: Angus Salkeld <asalkeld@redhat.com>
Reviewed-by: Fabio M. Di Nitto <fdinitto@redhat.com>
This patch modifies most of the existing comments in header files to be
in a format that doxygen can interpret. This provides another
significant improvement to the web/pdf/etc generated documentation
without having to add new content.
Signed-off-by: Russell Bryant <russell@russellbryant.net>
Reviewed-by: Angus Salkeld <asalkeld@redhat.com>
This change paves the way for eliminating a copy within the Infiniband
driver in the future by transferring responsibility for allocating and
freeing message buffers to the transport driver layer.
Tested under valgrind on a single-node cluster.
Signed-off-by: Zane Bitter <zane.bitter@gmail.com>
Reviewed-by: Steven Dake <sdake@redhat.com>
A commit token should be rejected when a token is lost in the recovery
state. This occurs naturally because the ring id increases by 4 for
every new ring. Prior to this patch, if the token was lost, the old
ring id information was restored, causing a commit token to be accepted
when it should be rejected. This erronously accepted commit token would
lead to an assertion which is fixed by this patch.
Signed-off-by: Steven Dake <sdake@redhat.com>
Reviewed-by: Angus Salkeld <asalkeld@redhat.com>
This creates some content on the main page of the documentation
generated by doxygen. The main page includes the license and a link
to the project web site.
Signed-off-by: Russell Bryant <russell@russellbryant.net>
eviewed-by: Steven Dake <sdake@redhat.com>
This resolves a couple of doxygen warnings. First, the group needed a
name. Second, all of the functions in the file were added to the group
but doxygen complained about the lack of an end to the grouping.
Signed-off-by: Russell Bryant <russell@russellbryant.net>
Reviewed-by: Steven Dake <sdake@redhat.com>
The included doxygen configuration file was a bit stale. It included
some options that were obsolete and caused doxygen to generate some
warnings when running it. Most of the changes here were simply done by
running "doxygen -u" to automatically update the file. It added its
documentation for the options and removed the obsolete options.
This also includes one configuration change, which is to set EXTRACT_ALL
to yes. This instructs doxygen to generate documentation pages for all
files, public functions, and public data structures even if they are not
currently documented using doxygen syntax. Doxygen is capable of
generating some useful documentation on its own, such as dependency
graphs.
Signed-off-by: Russell Bryant <russell@russellbryant.net>
Reviewed-by: Steven Dake <sdake@redhat.com>
The configure script has been updated to check for the doxygen and dot
applications (from doxygen and graphviz). The results from these checks
are now used in the Makefile to ensure that the tools are installed when
you run "make doxygen". If they are not, it will generate a helpful
error message.
Signed-off-by: Russell Bryant <russell@russellbryant.net>
Reviewed-by: Steven Dake <sdake@redhat.com>
From the strcpy(3) man page, the following warning is given:
The strncpy() function is similar, except that at most n bytes of src
are copied. Warning: If there is no null byte among the first n bytes
of src, the string placed in dest will not be null-terminated.
The current corosync code base does not take this warning into account
when using strncpy, potentially resulting in non-null terminated strings.
Signed-off-by: Russell Bryant <russell@russellbryant.net>
Reviewed-by: Steven Dake <sdake@redhat.com>
This option (-l or --less-secure) causes corosync-keygen to read from
/dev/urandom instead of /dev/random to ensure that no input is required
from the user. It may be useful when this command is used from a
script.
Signed-off-by: Russell Bryant <russell@russellbryant.net>
Reviewed-by: Steven Dake <sdake@redhat.com>
