AMF is complemented to handle termination and instantiation with respect to
instantiation level also for the following scenarios:
- SU restart
- termination/instantiation errors during component/SU restart
- instantiation error during cluster start up
git-svn-id: http://svn.fedorahosted.org/svn/corosync/trunk@1352 fd59a12c-fef9-0310-b244-a6a79926bd2f
- improves error handling caused by the INSTANTIATE or CLEANUP command
while recovering with component_restart or su_restart
git-svn-id: http://svn.fedorahosted.org/svn/corosync/trunk@1347 fd59a12c-fef9-0310-b244-a6a79926bd2f
- AMF handles a component report of injurious health.
- AMF handles saAmfHealthcheckConfirm() SA_AIS_ERR_FAILED_OPERATION
so that if it's a recent recovery ongoing amf does nothing but if it's
no immediate recovery in progress, AMF invokes the recovery action
specified by the component when the health check is started If
the individual recommendation was SA_AMF_NO_RECOMMENDATION,
then AMF uses the configured recovery action for the component
(saAmfCompRecoveryOnError). If this recommendation also is
SA_AMF_NO_RECOMMENDATION, then AMF makes a component restart or
component/SU fail over counts on the value of
saAmfCompDisableRestart and saAmfSUFailover.
- Handling of cleanup of a component and health check response hardened.
- Time supervision and check return value of clc-cli CLEANUP command.
- Handle 'recommended recovery' specified by a component in an error
report. The potential recovery action to choose
implemented is - component restart - and - node fails over.
- The attribute saAmfCompDisableRestart is now recognizable which means
that if the component specifies 'Component restart' and restart is
disabled
then the SU in which the component is contained shall fall over.
- The attribute saAmfSUFailover will not be recognized. SU will always
fail
over as a single entity.
- A component can report an error on another component than itself.
- Implementation 'Instantiation Level' according to chapter 3.9.2 in the
AMF specification.
- Implementation of the escalation levels, component restart, SU
restart, SU fail over and Node fail over.
git-svn-id: http://svn.fedorahosted.org/svn/corosync/trunk@1321 fd59a12c-fef9-0310-b244-a6a79926bd2f
A mechanism to defer and recall simultaneous
events in the state machines for amf_cluster,
amf_application and amf_sg.
The implication of this defer and recall mechanism is
that it's now possible to to recover from e.g. several
simultaneous SU failures in an ordered serialized manner.
The events that could be deferred/recalled so far is
SG_FAILOVER_NODE_EV,SG_START_EV,SG_FAILOVER_SU_EV,
CLUSTER_SYNC_READY_EV,APPLICATION_START_EV and
APPLICATION_ASSIGN_WORKLOAD_EV.
Files involved:
Index: exec/amfnode.c
Index: exec/amfsg.c
Index: exec/amfutil.c
Index: exec/amfapp.c
Index: exec/amfcomp.c
Index: exec/amfcluster.c
Index: exec/amf.h
git-svn-id: http://svn.fedorahosted.org/svn/corosync/trunk@1266 fd59a12c-fef9-0310-b244-a6a79926bd2f
The instantiaton of the component is performed with some new steps:
1. SU invoke Comp to instantiate
2. Comp multicast a new event
MESSAGE_REQ_EXEC_AMF_COMPONENT_INSTANTIATE
3. Comp receive the new event
MESSAGE_REQ_EXEC_AMF_COMPONENT_INSTANTIATE
4. If the Comp is within the SU hosted on the node. The
component invokes
the clc_cli instantiate script to start the component
and start a timer
to supervise the start and registration of the component.
5. If the instantiation time elapse before the component has
registered himself
Comp is sending a new multicasted event
MESSAGE_REQ_EXEC_AMF_COMPONENT_INSTANTIATE_TMO.
6. Comp receive
MESSAGE_REQ_EXEC_AMF_COMPONENT_INSTANTIATE_TMO event.
7. The Comp presence state is set to
SA_AMF_PRESENCE_INSTANTIATION_FAILED
8. When all Components are in presence state
SA_AMF_PRESENCE_INSTANTIATED or
SA_AMF_PRESENCE_INSTANTIATION_FAILED the start or restart will
continue with
the assignment of load.
This implemntation means that the complete instantiation procedure
never will be endlessly waiting for a register. The
instantiation will
either turn out in a component instantiation failure or a success.
Hardening of the cluster start use case:
1. A clearer separation of the responsibilities between
amf_cluster and
amf_application.
2. A clearer interface and separation between amf_main (amf.c) and
amf_cluster.
3. A clearer interface and separation between amf_cluster
and amf_node.
4. A clearer separation of the responsibilities between amf_node and
amf_application.
git-svn-id: http://svn.fedorahosted.org/svn/corosync/trunk@1251 fd59a12c-fef9-0310-b244-a6a79926bd2f
initially in an order such that at least two nodes start after at least one
node has been started and its SUs has been instantiated.
git-svn-id: http://svn.fedorahosted.org/svn/corosync/trunk@1241 fd59a12c-fef9-0310-b244-a6a79926bd2f
The call to gethostbyaddr() has been removed. This has been replaced by a
protocol where each node multicasts its hostname (obtained with gethostname()).
- "Logical AMF nodes"
The AMF node name is no longer a hostname. The saAmfNodeClmNode
configuration attribute of the AMF node is now the hostname. This config
attribute is now mandatory. The change to amf.conf file shows required changes.
- Some other AMF sync bug fixes
git-svn-id: http://svn.fedorahosted.org/svn/corosync/trunk@1236 fd59a12c-fef9-0310-b244-a6a79926bd2f
2. Improvement of the use case 'Amf node join'
3. Improvement to manage more than one SG within an Application.
4. Improvement to manage an arbitrary number of Csi-assignments associated
to the Csi
git-svn-id: http://svn.fedorahosted.org/svn/corosync/trunk@1232 fd59a12c-fef9-0310-b244-a6a79926bd2f
- One AMF node reads the AMF config file (IMM style)
- One AMF node syncs others AMF nodes
- One AMF object is serialized and sent as one message
- Serialization/deserialization of most objects is trivial (memcpy)
except for component and csi-attributes objects which have variable size
arrays/strings.
- Depth first AMF object tree traversal preserves relations when syncing
- Ordered lists of SUs and SIs
- Constructors/destructor per class
- Serializers/deserializers per class
- Config-change changes sync state
- Sync callbacks executes the sync
- "Use case" tracing for sync using the SYNCTRACE macro (trace6)
- Sync master is initially the winner of a timeout race and if the
master leaves the cluster, the node with the lowest node ID becomes new master.
- amf_malloc implements an AMF central malloc routine with error handling.
git-svn-id: http://svn.fedorahosted.org/svn/corosync/trunk@1200 fd59a12c-fef9-0310-b244-a6a79926bd2f
model). A central header file (amf.h) keeps all the definitions and
prototypes needed.
New things apart from that:
- some doxygen html generated from AMF e.g. each file has a description
- saAmfHAStateGet() now works
- component invoked healthchecks implemented (but not tested)
git-svn-id: http://svn.fedorahosted.org/svn/corosync/trunk@1071 fd59a12c-fef9-0310-b244-a6a79926bd2f
