RCA: There were 2 problems.
1. SGRpt prune expiry didn't create S,G entry with none oil when no other
interfaces were part of the oil.
2. When restarting the timer with new hold value, comparision was missing and
old timer was not stopping.
Fix:
SGRpt Prune pending expiry will put SG entry with none oil if no other
Signed-off-by: Saravanan K <saravanank@vmware.com>
interfaces present. If present we will be deleting the inherited oif from oil.
Deleting the oif in that scenario will take care of changing mroute.
When alone interface expires in SGRpt prune pending state, we shall detect by
checking installed flag. if not installed, install mroute.
1. When a node changes from non-DR to DR in the given topology,
the node was receiving both PIM Join as well as IGMP join.
Since it was already receiving PIM Join previously, ifchannel was
already present. Hence when it becomes DR, the IGMP source flag is not
set due to issue in the code. Hence it never creates S,G entry thinking
that it is not DR.
2. When pim join expires, the pim flag is not reset when ifchannel is not
deleted.
Issue: #7752
Signed-off-by: Mobashshera Rasool <mrasool@vmware.com>
problem :
=========
When (*,G) prune received where we have SGRpt state,
ifchannel goes to NO_INFO state and doesn't get removed.
Root cause :
============
During the processing of (*,G) prune, we are not removing the
ifchannel on PruneTmp or PrunePendingTmp state.
Fix :
=====
In that scenario, stop joinExpiry timer and delete the ifchannel.
issue #7347
Co-authored-by: Saravanan K <saravanank@vmware.com>
Signed-off-by: vishaldhingra <vdhingra@vmware.com>
Remove mid-string line breaks, cf. workflow doc:
.. [#tool_style_conflicts] For example, lines over 80 characters are allowed
for text strings to make it possible to search the code for them: please
see `Linux kernel style (breaking long lines and strings)
<https://www.kernel.org/doc/html/v4.10/process/coding-style.html#breaking-long-lines-and-strings>`_
and `Issue #1794 <https://github.com/FRRouting/frr/issues/1794>`_.
Scripted commit, idempotent to running:
```
python3 tools/stringmangle.py --unwrap `git ls-files | egrep '\.[ch]$'`
```
Signed-off-by: David Lamparter <equinox@diac24.net>
Problem:
When the ifchannel is in SGRpt prune, if we receive a join, we go into no info
state but mroute still present with none oil
Join Prune Expiry timer on the ifchannel was still running when
Prune pending expired. This causes ifchannel not to be deleted and hence mroute.
Fix:
Stop expiry timer when we move into NOINFO state and delete the ifchannel.
Signed-off-by: Saravanan K <saravanank@vmware.com>
RCA: starg join fell in to SGRpt join check and was treated as SGRpt join
so ifchannel state machine moved from prunepending to noinfo.
Fix: Check if it is starg join and process
Signed-off-by: Saravanan K <sarav511@vmware.com>
RCA:
1. Prune processing during join state was putting of join expiry timer
2. Join received during prune pending state was not comparing hold time with remaining expiry timer.
Fix:
Fixed as per RFC 4601/7761
Signed-off-by: Saravanan K <saravanank@vmware.com>
Issue: Client1------LHR-----(int-1)RP(int-2)------client2
Client2 send IGMP join for group G.
Client1 send IGMP join for group G.
verify show ip mroute in RP, will have 2 OIL.
Client2 send IGMP leave.
Verify show ip mroute in RP, will still have 2.
Root cause: When RP receives IGMP join from client2, it creates
a (s,g) channel oil and add the interface int-2 into oil list and
set the flag PIM_OIF_FLAG_PROTO_IGMP to int-2
Client1 send IGMP join, LHR will send a (*,G) join to RP. RP will
add the interface int-1 into the oil list of (s,g) channel_oil and
will set the flag PIM_OIF_FLAG_PROTO_IGMP and PIM_OIF_FLAG_PROTO_PIM
to the int-1 and set PIM_OIF_FLAG_PROTO_PIM to int-2 as well. It is
happening because of the pim_upstream_inherited_olist_decide() and
forward_on() get all the oil and update the flag wrongly.
So now when client 2 sends IGMP prune, RP will not remove the int-2
from oil list since both PIM_OIF_FLAG_PROTO_PIM & PIM_OIF_FLAG_PROTO_IGMP
are set, it just unset the flag PIM_OIF_FLAG_PROTO_IGMP.
Fix: Introduced new flags in if_channel, PIM_IF_FLAG_MASK_PROTO_PIM
& PIM_IF_FLAG_MASK_PROTO_IGMP. If a if_channel is created because of
pim join or pim (s,g,rpt) prune received, then set the flag
PIM_IF_FLAG_MASK_PROTO_PIM. If a if_channel is created becuase of IGMP
join received, then set the flag PIM_IF_FLAG_MASK_PROTO_IGMP.
When an interface needs to be added into the oil list check if
PIM_IF_FLAG_MASK_PROTO_PIM or PIM_IF_FLAG_MASK_PROTO_IGMP is set, then
update oil flag accordingly.
Signed-off-by: Sarita Patra <saritap@vmware.com>
Series of events leading to the problem -
1. (S,G) has been pruned on the rp on downlink-1
2. a (*,G) join is rxed on downlink-1 without the source S. This
results in the (S,G,rpt) prune state being cleared on downlink-1.
As a part of the clear the ifchannel associated with downlink-1
is deleted.
3. The ifchannel_delete handling is expected to add downlink-1
as an inherited OIF to the channel OIL (which it does). However
it is also added in as an immediate OIF (accidentally) as the
ifchannel is still present (in the process of being deleted).
To avoid the problem defer pim_upstream_update_join_desired
evaluation until after the channel is deleted.
Relevant debug logs -
PIM: pim_ifchannel_delete: ifchannel entry (27.0.0.15,239.1.1.106)(downlink-1) del start
PIM: pim_channel_add_oif(pim_ifchannel_delete): (S,G)=(27.0.0.15,239.1.1.106): proto_mask=4 OIF=downlink-1 vif_index=7: DONE
PIM: pimd/pim_oil.c pim_channel_del_oif: no existing protocol mask 2(4) for requested OIF downlink-1 (vif_index=7, min_ttl=1) for channel (S,G)=(27.0.0.15,239.1.1.106)
PIM: pim_upstream_switch: PIM_UPSTREAM_(27.0.0.15,239.1.1.106): (S,G) old: NotJoined new: Joined
PIM: pim_channel_add_oif(pim_upstream_inherited_olist_decide): (S,G)=(27.0.0.15,239.1.1.106): proto_mask=2 OIF=downlink-1 vif_index=7 added to 0x6 >>>>>>>>>>>>>>>>>>
PIM: pim_upstream_del(pim_ifchannel_delete): Delete (27.0.0.15,239.1.1.106)[default] ref count: 2 , flags: 81 c_oil ref count 1 (Pre decrement)
PIM: pim_ifchannel_delete: ifchannel entry (27.0.0.15,239.1.1.106)(downlink-1) del end
Ticket: CM-26732
Signed-off-by: Anuradha Karuppiah <anuradhak@cumulusnetworks.com>
Additional protocols were being set on the OIF proto-mask without
logs. Added logs in that area.
Also added start and end logs to ifchannel_delete to help
identify state machine changes that play out as a part of this
event handling.
Ticket: CM-26732
Signed-off-by: Anuradha Karuppiah <anuradhak@cumulusnetworks.com>
Initially, MLAG Sync is happened at pim_ifchannel, this is mainly to
support even config mismatches(missing configuration of dual active).
But this causes more syncs for each entry.
and also it is not In-line with PIM EVPN. to avoid that moving to
pm_upstream based syncing.
Signed-off-by: Satheesh Kumar K <sathk@cumulusnetworks.com>
There was some code missed during the upstreaming process
due to code squash. Identify and put into a commit
to keep code consistent and correct.
Signed-off-by: Satheesh Kumar K <sathk@cumulusnetworks.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@cumulusnetworks.com>
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
A local membership is created on the vxlan termination device ipmr-lo. This
is done to -
1. Pull multicast vxlan tunnel traffic to the VTEP for termination by
triggering JoinDesired on the BUM multicast group.
2. Include the OIF in the mroute to signal to the dataplane component
that flow needs to be vxlan terminated.
Earlier we were overloading the PIM_UPSTREAM_FLAG_MASK_SRC_IGMP for
this local membership creation but that is creating confusion both in
the state machine and in the show outputs. To avoid that we use the
more apparent PIM_UPSTREAM_FLAG_MASK_SRC_VXLAN_TERM. With this change -
1. We get LHR functionality for VXLAN_TERM mroutes
2. OIF is populated with PIM_OIF_FLAG_PROTO_PIM only
Signed-off-by: Anuradha Karuppiah <anuradhak@cumulusnetworks.com>
It was causing a Join on (S,G) who's prune state was being cleared. This
was an inactive (KAT not running; no immediate OIL) entry that was being
flushed out but because of this incorrect Join (that was being done with
out join-state checks) the source was getting populated repeatedy i.e.
never aged.
Output of "ip monitor mroute"
=============================
(27.0.0.11,239.1.1.102) Iif: lo State: resolved Table: default
Deleted (27.0.0.11,239.1.1.102) Iif: lo State: resolved Table: default
(27.0.0.11,239.1.1.102) Iif: pimreg State: resolved Table: default
(27.0.0.11,239.1.1.102) Iif: uplink-1 State: resolved Table: default
(27.0.0.11,239.1.1.102) Iif: uplink-1 State: resolved Table: default
(27.0.0.11,239.1.1.102) Iif: uplink-1 State: resolved Table: default
(27.0.0.11,239.1.1.102) Iif: lo Oifs: uplink-1 State: resolved Table: default
(27.0.0.11,239.1.1.104) Iif: lo Oifs: pimreg uplink-1 State: resolved Table: default
(27.0.0.11,239.1.1.102) Iif: lo Oifs: pimreg uplink-1 State: resolved Table: default
Deleted (27.0.0.11,239.1.1.102) Iif: lo State: resolved Table: default
(27.0.0.11,239.1.1.102) Iif: pimreg State: resolved Table: default
(27.0.0.11,239.1.1.102) Iif: uplink-1 State: resolved Table: default
(27.0.0.11,239.1.1.102) Iif: uplink-1 State: resolved Table: default
(27.0.0.11,239.1.1.102) Iif: uplink-1 State: resolved Table: default
(27.0.0.11,239.1.1.102) Iif: lo Oifs: uplink-1 State: resolved Table: default
These mroute events (on a no longer existing multicast souce) continue in
a never ending loop.
Triggered joins/prunes MUST only done via state machine transitions i.e.
via pim_upstream_update_join_desired.
Signed-off-by: Anuradha Karuppiah <anuradhak@cumulusnetworks.com>
Dumps while in problem state -
============================
[from "show ip pim state"]
Active Source Group RPT IIF OIL
1 6.0.0.31 239.1.1.111 n swp1 swp4( J * )
[from "show ip pim join"]
Interface Address Source Group State Uptime Expire Prune
swp3 6.0.0.22 6.0.0.31 239.1.1.111 JOIN --:--:-- 03:11 --:--
You can see from the dumps that the pim downstream router has joined on
swp3 but that OIF has not been added to the OIL with flag
PIM_OIF_FLAG_PROTO_PIM. This is because the join was rxed while the
ifchannel was in a prune-pending state.
Relevant logs -
===============
[
PIM: recv_prune: prune (S,G)=(6.0.0.31,239.1.1.111) rpt=1 wc=0 upstream=6.0.0.22 holdtime=210 from 6.0.0.28 on swp3
PIM: pim_upstream_ref(pim_ifchannel_add): upstream (6.0.0.31,239.1.1.111) ref count 3 increment
PIM: pim_upstream_add(pim_ifchannel_add): (6.0.0.31,239.1.1.111), iif 6.0.0.26/0 (swp1) found: 1: ref_count: 3
PIM: pim_ifchannel_add: ifchannel (6.0.0.31,239.1.1.111) is created
PIM: pim_joinprune_recv: SGRpt flag is set, del inherit oif from up (6.0.0.31,239.1.1.111)
PIM: pim_mroute_add(pim_channel_del_oif), vrf default Added Route: (6.0.0.31,239.1.1.111) IIF: swp1, OIFS: swp4
PIM: pim_channel_del_oif(pim_joinprune_recv): (S,G)=(6.0.0.31,239.1.1.111): proto_mask=4 IIF:1 OIF=swp3 vif_index=3
PIM: recv_join: join (S,G)=(6.0.0.31,239.1.1.111) rpt=0 wc=0 upstream=6.0.0.22 holdtime=210 from 6.0.0.28 on swp3
PIM: PIM_IFCHANNEL(swp3): (6.0.0.31,239.1.1.111) is switching from SGRpt(PP) to JOIN
PIM: Sending Request for New Channel Oil Information(6.0.0.31,239.1.1.111) VIIF 1(default)
]
Signed-off-by: Anuradha Karuppiah <anuradhak@cumulusnetworks.com>
When a inherited OIL becomes empty join-desired can go to false. So
we need to re-run join-desired evaluation on any inherited OIL changes.
Signed-off-by: Anuradha Karuppiah <anuradhak@cumulusnetworks.com>
This commit includes the following changes -
1. kat needs to be included when evaluting join desired on a (S,G)
entry.
2. there were cases where we were adding OIF based on joindesired
being true for unrelated reasons (on other OIFs). cleaned up those
cases.
3. make all calls to pim_upstream_switch conditional on the JoinDesired
macro.
Signed-off-by: Anuradha Karuppiah <anuradhak@cumulusnetworks.com>
These logs were printing file name which has little value (is always
pim_oil.c). Instead print the caller.
add_oif/del_oif are being called directly from one too many. Instead OIF
setup needs to be consolidated via the PIM state machine. These
debugs are expected to help in understanding what needs to be cleaned up.
Signed-off-by: Anuradha Karuppiah <anuradhak@cumulusnetworks.com>
When you turn on `debug igmp trace` we are seeing a bunch
of debugs associated with pim processing. This is because we were
using PIM_DEBUG_TRACE which is both `debug igmp trace` and `debug pim trace`
when tracing igmp code it would be nice to only see igmp work.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
We have a zlog_warn that is unguarded ( and really is a debug message )
as that there is nothing the end user can do and nothing to note
here other than a debug message to track refcounts. Change
to an appropriate debug and zlog_debug it instead.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
In a pim-evpn setup (say TORC11<=>TORC12) an mroute can have a mix of
PIM and IGMP joins. The vxlan termination device ipmr-lo is IGMP
joined on termination mroutes and the peerlink-rif can be pim joined
on the same mroute if the MLAG peer (TORC11) loses all its uplinks to
underlay -
root@TORC12:~# net show pim state 239.1.1.101|grep pimreg
1 * 239.1.1.101 uplink-1
pimreg(I ), ipmr-lo( J ), peerlink-3.4094( J )
root@TORC12:~#
When the uplinks come back up on TORC11 it will prune the peerlink-rif
and join the RP (say spine) via the uplinks.
TORC12 is rxing the prune and removing the if_channel
(pim_ifchannel_delete). However it is not removing the OIF from
mfcc_ttl basically leaving behind a leaked OIF in the forwarding
entry. And this is because it is deriving the owner flag from the
parent upstream entry and incorrectly concluding that all OIFs are
IGMP joined.
Thix fix flushes out both PIM and IGMP ownership when the ifchannel is
deleted.
There is a second fix in the commit and that is to set the proto mask
correctly (to STAR) for inherited OIFs. (S,G) entries can inherit the
OIF from the (*, G) entry and this decision can change when the pim/igmp
ifchannel is removed. The earlier code was setting the proto-mask
incorrectly to PIM or IGMP.
Signed-off-by: Anuradha Karuppiah <anuradhak@cumulusnetworks.com>
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
(cherry picked from commit d4d1d968dbbe61347393f7dace8b675496ff1024)
When a link goes down the vifi was being deleted but the OIF stayed
in the OIL with a stale vifi -
oroot@act-7726-03:~# net show pim state
Codes: J -> Pim Join, I -> IGMP Report, S -> Source, * -> Inherited from (*,G), V -> VxLAN
Installed Source Group IIF OIL
1 * 239.1.1.111 swp1s1 pimreg(I ), ipmr-lo( J )
1 6.0.0.28 239.1.1.111 lo pimreg( J ), ipmr-lo( *), swp1s1( J )
root@act-7726-03:~# ip link set swp1s1 down
root@act-7726-03:~# net show pim state
Codes: J -> Pim Join, I -> IGMP Report, S -> Source, * -> Inherited from (*,G), V -> VxLAN
Installed Source Group IIF OIL
1 * 239.1.1.111 swp1s0 pimreg(I ), ipmr-lo( J )
1 6.0.0.28 239.1.1.111 lo ipmr-lo( *), swp1s0( J ), <oif?>( J ) >>>>>>>>
root@act-7726-03:~#
The problem was as a part ifchannel_delete the join state of the channel
was checked to avoid incorrect OIF deletion this was preventing the OIF
from being flushed. Fix is to flip the channel join-state to NOINFO before
deleting it.
Signed-off-by: Anuradha Karuppiah <anuradhak@cumulusnetworks.com>
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
There is no need to check for ALLOC function failures
in the code base. If we cannot get more memory we
assert.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
The pim ifchannel expiry timer was not setting any debug output.
Let's add something in to help us understand what is going on.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Fix: When RP receives a (*, G) join and corresponding (s,g)
is present, then check for OIL is not-empty, then only switch
upstream (s, g) state to JOINED.
Signed-off-by: Sarita Patra <saritap@vmware.com>
It doesn't make much sense for a hash function to modify its argument,
so const the hash input.
BGP does it in a couple places, those cast away the const. Not great but
not any worse than it was.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
So when we remove a ifchannel from the system we should check to
see if we still care about the S,G having it in the OIL still
due to inheritance rules. The deletion does not necessarily
mean it should not be in the OIL for the S,G.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
When we receive a S,G join and the ifchannel is in S,G RPT Prune state,
pim should transition the ifchannel state to JOIN and transition the
pim_upstream state for the S,G stream.
Ticket: CM-24513
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
pim was sending a triggered response on every S,G RPT prune information
read. Suppose we had this in a *,G message:
*,G
S1, G RPT Prune
S2, G RPT Prune
We would send two triggered *,G messages upstream. This leads to over
processing and quickly changing state if S1 or S2 were in different
states.
Modify the code to send just one Triggered *,G upstream after looking
at all S,G state for a *,G.
Ticket: CM-24531
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
When we are displaying S,G string data we already auto
display the string as (S,G) no need to have ((S,G)).
Cleanup some that were found during log look through.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Create a `struct pim_router` and move the thread master into it.
Future commits will further move global varaibles into the pim_router
structure.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
When we fail to add a local membership add some additional debugs
so that we can have a bit more information on when something goes
bad.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
When pimd is getting terminated, pim_upstream_del() gets called as
part of cleaning process. pim_upstream_del() deletes the route and
assigns NULL to the up->channel_oil. It also deletes each if_channel
by calling the function pim_ifchannel_delete().
pim_ifchannel_delete() internally calls the caller function pim_upstream_del(),
if it is the last ifchannel for that upstream. So pim_upstream_del
is getting called twice, which will access the up->channel_oil which
was already set to NULL before. This results in crash.
Fix:
pim_ifchannel_delete() should call pim_upstream_del (caller function)
only if the up->ref_count > 0. Added an assert(up->ref_count > 0) in
the function pim_upstream_del().
Signed-off-by: Sarita Patra <saritap@vmware.com>
