It turns out that there are some legitimate cases where fragmented
messages might be interrupted during sending (e.g. CS_ERR_TRY_AGAIN or
as in my case: CS_ERR_INTERRUPT). This creates a situation where
LIBCPG_PARTIAL_FIRST is sent multiple times before receiving
LIBCPG_PARTIAL_LAST.
Solution is to drop incomplete message and start assembly of new message
as libcpg should have reported error during sending of that
incomplete message.
Signed-off-by: Rytis Karpuška <rytisk@neurotechnology.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Packet assembly is done seperately for each nodeid, pid pair, therefore
multiple packets are not mixed into single buffer.
Signed-off-by: Rytis Karpuška <rytisk@neurotechnology.com>
Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
- Error returned by coroipcc_msg_send_reply_receive is now correctly
handled.
- If munmap fails, error is set to proper value and handle is put back
into handle_db
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Function in cpg_zcb_alloc (from code lib/cpg.c) creates
/dev/shm/corosync_zerocopy-XXXXX and does mmap
The memory is allocated by corosync service (function zcb_alloc
in exec/cpg.c) also and both shares this memory via mmap
(uses MAP_SHARED in mmap call)
Corosync calls unlink which deletes the file from /dev/shm while
closing the file descriptor, but unmap is not happening correctly
while calling cpg_zcb_free.
So:
- still the deleted file holds the memory
- As munmap is not happening correctly, the number of mappings per
process gets exceeded and corosync dies with ENOMEM
From gdb, the size passed to munmap appears to be zero and address
looks wrong. Also in the code return code of munmap is not checked.
The patch adds check for:
- munmap return code and getting correct address for munmap
Signed-off-by: Athira Rajeev <atrajeev@linux.vnet.ibm.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
If a cpg client sends a message larger than 1Mb (actually slightly
less to allow for internal buffers) cpg will now fragment that into
several corosync messages before sending it around the ring.
cpg_mcast_joined() can now return CS_ERR_INTERRUPT which means that the
cpg membership was disrupted during the send operation and the message
needs to be resent.
The new API call cpg_max_atomic_msgsize_get() returns the maximum size
of a message that will not be fragmented internally.
New test program cpghum was written to stress test this functionality,
it checks message integrity and order of receipt.
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com>
Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Code for zero-copy in cpg does following mmaps:
- Mmap anonymous, private memory to some address (-> malloc)
- Mmap shared memory of fd to address returned by first mmap
(effectively shadows first mapping)
This is not necessary and only one mapping is needed.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Steven Dake <sdake@redhat.com>
Reviewed-by: Fabio M. Di Nitto <fdinitto@redhat.com>
IPC is using buffer of CS_MAX_NAME_LENGTH for name. If user calls
function with longer string, such string can be passed to service
incomplete.
Solution is to not allow string larger then CS_MAX_NAME_LENGTH
and return error.
Same applies to cpg service.
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Fabio M. Di Nitto <fdinitto@redhat.com>
Sometimes calling xyz_finilize() within a dispatch would
cause a crash because the qb_ipcc_disconnect actually
disconnects immediatly and frees it't memory. whereas
the corosync structure is reference counted. So this
makes use of the reference counting to only call
qb_ipcc_disconnect when it is fully dereferenced.
Reviewed-by: Fabio M. Di Nitto <fdinitto@redhat.com>
Signed-off-by: Angus Salkeld <asalkeld@redhat.com>
1) remove BUSY loop from membership get
Note only cpg_join and cpg_leave ever set the
BUSY error code.
2) set the size correctly
3) copy the name in correctly
Signed-off-by: Angus Salkeld <asalkeld@redhat.com>
Reviewed-by: Steven Dake <sdake@redhat.com>
Add missing option for dispatch, which fills gap in combination of
block/nonblock and one/all dispatch types. New type doesn't mask
CS_ERR_TRY_AGAIN, and it means "no message was processed".
Signed-off-by: Jan Friesse <jfriesse@redhat.com>
Reviewed-by: Steven Dake <sdake@redhat.com>
the only user of those obsoleted defines is dlm master (already ported)
to use CS_ and cmirror (that needs full porting to new corosync either way).
Signed-off-by: Fabio M. Di Nitto <fdinitto@redhat.com>
Reviewed-by: Steven Dake <sdake@redhat.com>
Many functions do not require flowcontrol and are two-way
so they can get failures from corosync.
Only cpg_mcast_joined() _really_ needs the current level
of flowcontrol.
Signed-off-by: Angus Salkeld <asalkeld@redhat.com>
Reviewed-by: Steven Dake <sdake@redhat.com>
IPC: return 0/-ENOBUFS from message handler
IPC: use the new rate_limit API to improve perf.
CPG: add send_async API & hook up flow control
IPC: Fix flow control getting stuck.
IPC: Port the remaining libs to use libqb IPC
IPC: remove libqb flowcontrol API
TEST: put cpg_dispatch() in it's own thread
IPC: cleanup ipc_glue.c name everything cs_ipcs_*()
IPC: add back statistics
IPC: remove coroipcc_ symbols from lib*.versions
IPC: init each se's IPC as it is loaded.
IPC: use the new connection_closed() event to free the context.
IPC: re-add zero copy functionality back
IPC: remove cpg_mcast_joined_async() and make it the default
-> now cpg_mcast_joined() == cpg_mcast_joined_async()
libqb: expose a libqb error converter
libqb: add missing error conversions
libqb: remove repeat try loop in lib/cpg.c
CPG: fix zero copy mcast
CPG: use newer return codes
Add ENOTCONN to qb_to_cs_error()
libqb: fix error conversion from errno to cs_error_t in confdb
libqb: change errno_to_cs to qb_to_cs_error
libqb: add a cs_strerror() to get a more meaningful message
libqb: fix some confusing error conversions.
libqb: set the timeout on recv's to -1 (wait forever)
Signed-off-by: Angus Salkeld <asalkeld@redhat.com>
Reviewed-by: Steven Dake <sdake@redhat.com>
This prevents a shared mem leak if corosync dies while clients
are connected.
Calling cpg_finalize() did not release the shared mem as
coroipcc_msg_send_reply_receive() returned an error and
thus coroipcc_service_disconnect() did not get called.
Signed-off-by: Angus Salkeld <asalkeld@redhat.com>
Reviewed-by: Steven Dake <sdake@redhat.com>
Patch adds new function to initialize cpg, cpg_model_initialize. Model
is set of callbacks. With this function, future addions of models
should be possible without changing the ABI.
Patch also contains callback in CPG_MODEL_V1 for notification about
Totem membership changes.
git-svn-id: http://svn.fedorahosted.org/svn/corosync/trunk@2770 fd59a12c-fef9-0310-b244-a6a79926bd2f
Add support for MESSAGE_REQ_CPG_FINALIZE message. This will allow us
remove cpg_pd from list of active connections, and remove problem, when
cpg_finalize + cpg_initialize + cpg_join can result in CPG_ERR_EXIST
error.
git-svn-id: http://svn.fedorahosted.org/svn/corosync/trunk@2676 fd59a12c-fef9-0310-b244-a6a79926bd2f
This functions allows iterate available cpg groups
and their members. API is modelled like ckpt iteration
functions.
git-svn-id: http://svn.fedorahosted.org/svn/corosync/trunk@2399 fd59a12c-fef9-0310-b244-a6a79926bd2f
Attached patches handle NULL callbacks in *_initialize
and *_dispatch. Handling is same as in quorum service. Now, when
callback is NULL -> no memcpy -> instance callbacks will have all
items set to NULL and in *_dispatch function is not called.
It changes cfg so now we are using continue instead of exit.
git-svn-id: http://svn.fedorahosted.org/svn/corosync/trunk@2392 fd59a12c-fef9-0310-b244-a6a79926bd2f
