Imagine a situation where a interface is bouncing up/down.
The interface comes up and daemons like pbr will get a nht
tracking callback for a connected interface up and will install
the routes down to zebra. At this same time the interface can
go down. But since zebra is busy handling route changes ( from pbr )
it has not read the netlink message and can get into a situation
where the route resolves properly and then we attempt to install
it into the kernel( which is rejected ). If the interface
bounces back up fast at this point, the down then up netlink
message will be read and create two route entries off the connected
route node. Zebra will then enqueue both route entries for future processing.
After this processing happens the down/up is collapsed into an up
and nexthop tracking sees no changes and does not inform any upper
level protocol( in this case pbr ) that nexthop tracking has changed.
So pbr still believes the nexthops are good but the routes are not
installed since pbr has taken no action.
Fix this by immediately running rnh when we signal a connected
route entry is scheduled for removal. This should cause
upper level protocols to get a rnh notification for the small
amount of time that the connected route was bouncing around like
a madman.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
We can make the Linux kernel send an ARP/NDP request by adding
a neighbour with the 'NUD_INCOMPLETE' state and the 'NTF_USE' flag.
This commit adds new dataplane operation as well as new zapi message
to allow other daemons send ARP/NDP requests.
Signed-off-by: Jakub Urbańczyk <xthaid@gmail.com>
For the sake of Segment Routing (SR) and Traffic Engineering (TE)
Policies there's a need for additional infrastructure within zebra.
The infrastructure in this PR is supposed to manage such policies
in terms of installing binding SIDs and LSPs. Also it is capable of
managing MPLS labels using the label manager, keeping track of
nexthops (for resolving labels) and notifying interested parties about
changes of a policy/LSP state. Further it enables a route map mechanism
for BGP and SR-TE colors such that learned BGP routes can be mapped
onto SR-TE Policies.
This PR does not introduce any usable features by now, it is just
infrastructure for other upcoming PRs which will introduce 'pathd',
a new SR-TE daemon.
Co-authored-by: Renato Westphal <renato@opensourcerouting.org>
Co-authored-by: GalaxyGorilla <sascha@netdef.org>
Signed-off-by: Sebastien Merle <sebastien@netdef.org>
It is causing build failures because of conflicts with netinet.
Instead I have re-defined the MAC-SYNC UAPIs in the re_netlink.c
This is clearly a hack that needs to be re-visited.
Signed-off-by: Anuradha Karuppiah <anuradhak@cumulusnetworks.com>
MAC-IP routes are used for syncing local entries across redundant
switches in an EVPN-MH setup. A path from a peer that has a local
ES as destination is tagged as a SYNC path. The SYNC path results in the
addition of local MAC and/or local neigh entry in zebra and in the
dataplane.
Implementation overview
=======================
1. Three new flags "local-inactive", "peer-active" and "peer-proxy"
are maintained per-local-MAC and per-local-Neigh entry.
2. The "peer-XXX" flags are set and cleared via SYNC path updates
from BGP. Proxy sync paths result in the setting of "peer-proxy" flag
(and non-proxies result in the "peer-active").
3. A neigh entry that has a "peer-XXX" flag set is programmed as
"static" in the dataplane.
4. A MAC entry that has a "peer-XXX" flag set or is referenced by
a sync-neigh entry (that has a "peer-XXX" flags set) is programmed
as "static" in the dataplane.
5. The sync-seq number is used to normalize the MM seq number across
all the redundant switches i.e. the max MM seq number across all
switches is used by each of the switches. This commit also includes
the changes needed for extended MM seq syncing.
6. A MAC/neigh entry has to be local-active or peer-active to sent to
BGP. An entry that is NOT local-active is sent with the proxy flag (so
BGP can "proxy" advertise it).
7. The "peer-active" flag is aged out by zebra by using a hold_timer
(this is instead of being abruptly dropped on SYNC path delete). This
age-out is needed to handle peer-switch restart (procedures are specified
in draft-rbickhart-evpn-ip-mac-proxy-adv). The holdtime needs to be
sufficiently long to allow an external neighmgr daemon or the dataplane
component to independently probe and establish local reachability of a
host. The MAC and neigh hold time values are configurable.
PS: In the future this probing may happen in FRR itself.
CLI changes to display sync info
================================
MAC
===
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
root@torm-11:mgmt:~# net show evpn mac vni 1000
Number of MACs (local and remote) known for this VNI: 6
Flags: N=sync-neighs, I=local-inactive, P=peer-active, X=peer-proxy
MAC Type Flags Intf/Remote ES/VTEP VLAN Seq #'s
00:02:00:00:00:25 local vlan1000 1000 0/0
02:02:00:00:00:02 local PI hostbond1 1000 0/0
02:02:00:00:00:06 remote 03:00:00:00:00:02:11:00:00:01 0/0
02:02:00:00:00:01 local X hostbond1 1000 0/0
00:00:00:00:00:11 local PI hostbond1 1000 0/0
02:02:00:00:00:05 remote 03:00:00:00:00:02:11:00:00:01 0/0
root@torm-11:mgmt:~#
root@torm-11:mgmt:~# net show evpn mac vni 1000 mac 00:00:00:00:00:11
MAC: 00:00:00:00:00:11
ESI: 03:00:00:00:00:01:11:00:00:01
Intf: hostbond1(58) VLAN: 1000
Sync-info: neigh#: 0 local-inactive peer-active >>>>>>>>>>>>
Local Seq: 0 Remote Seq: 0
Neighbors:
No Neighbors
root@torm-11:mgmt:~#
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
neigh
=====
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
root@torm-11:mgmt:~# net show evpn arp vni 1003
Number of ARPs (local and remote) known for this VNI: 4
Flags: I=local-inactive, P=peer-active, X=peer-proxy
Neighbor Type Flags State MAC Remote ES/VTEP Seq #'s
2001:fee1:0:3::6 local active 00:02:00:00:00:25 0/0
45.0.3.66 local P active 00:02:00:00:00:66 0/0
45.0.3.6 local active 00:02:00:00:00:25 0/0
fe80::202:ff:fe00:25 local active 00:02:00:00:00:25 0/0
root@torm-11:mgmt:~#
root@torm-11:mgmt:~# net show evpn arp vni 1003 ip 45.0.3.66
IP: 45.0.3.66
Type: local
State: active
MAC: 00:02:00:00:00:66
Sync-info: peer-active >>>>>>>>>>>>>>>>
Local Seq: 0 Remote Seq: 0
root@torm-11:mgmt:~#
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Signed-off-by: Anuradha Karuppiah <anuradhak@cumulusnetworks.com>
Multihoming support requires a new dataplane feature, MAC-ECMP, to
bridge traffic to remote ESs that are attached to more than one
active VTEP.
As a part of this support indirection has also been added via
L2-NHGs. Using a nexthop group allows for fast failover
of MAC entries when an access port attached to a remote-ES goes
down i.e. instead of updating many MAC entries this becomes a
single NHG update to the dataplane.
Note: Some of the code here needs to be reworked to the new
dataplane model.
Signed-off-by: Anuradha Karuppiah <anuradhak@cumulusnetworks.com>
Added a macro to validate the v4 mapped v6 address.
Modified bgp receive & send updates for v4 mapped v6 address as
nexthop and installing it as recursive nexthop in RIB.
Minor change in fpm while sending the routes for nexthop as
v4 mapped v6 address.
Signed-off-by: Kaushik <kaushik@niralnetworks.com>
Fix the function encoding evpn neighbor. The size of the buffer
for the netlink message wasn't correct and because of that we thought that
the message didn't fit entirely in the buffer and thus we were not sending
the update to the kernel.
Signed-off-by: Jakub Urbańczyk <xthaid@gmail.com>
`force_nhg` is only settable when calling from `fpm`, so if the kernel
was using next hop groups it would override our knob.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
When installing a nexthop-group the recent commit:
commit 0be6e7d75d
reversed the logic for testing if adding data to
the netlink message succeeded and we thought we did
not thus not creating the nexthop group.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
* Use nl_attr_add32 instead of nl_attr_add where it is possible.
* Move common code from build_singlepath() and build_multipath()
to separate function.
Signed-off-by: Jakub Urbańczyk <xthaid@gmail.com>
* Rename netlink utility functions like addattr to be less ambiguous
* Replace rta_attr_* functions with nl_attr_* since they introduced
inconsistencies in the code
* Add helper functions for adding rtnexthop struct to the Netlink
message
Signed-off-by: Jakub Urbańczyk <xthaid@gmail.com>
If you haven't migrated your FPM server to use next hop groups, it is
possible that you want to disable this feature. This commit implements
a toggle to enable/disable next hop groups usage (even if your Linux
kernel is not using it).
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Force off kernel NHG install with netns-based VRFs for
now. There is not really a good solution for allowing
kernel nexthop groups in namespaced based vrfs.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
When installing a nexthop group, dump out the ifindex of the
nexthop being installed as a bit more data for the developer.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
The function rt_netlink.c is using to lookup the vrf by
passed in table id.
I'm also going to pretend that this function is not
so awful to run when we have a large number of routes
incoming.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Replace sprintf with snprintf where straightforward to do so.
- sprintf's into local scope buffers of known size are replaced with the
equivalent snprintf call
- snprintf's into local scope buffers of known size that use the buffer
size expression now use sizeof(buffer)
- sprintf(buf + strlen(buf), ...) replaced with snprintf() into temp
buffer followed by strlcat
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
The netlink_request function takes a `struct nlmsghdr *`
pointer from a common pattern that we use:
struct {
struct nlmsghdr n;
struct fib_rule_hdr frh;
char buf[NL_PKT_BUF_SIZE];
} req;
We were calling it `netlink_request(Socket, &req.n)`
The problem here is that coverity, rightly so, sees that
we access the data after the nlmsghdr in netlink_request and
tells us we have an read beyond end of the structure. While
we know we haven't mangled anything up here because of manual
inspection coverity doesn't have this knowledge implicitly.
So let's modify the code call to netlink_request to pass in the
void pointer of the req structure itself, cast to the appropriate
data structure in the function and do the right thing. Hopefully
the coverity SA will be happy and we can move on with our life.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
* Break lines longer than 80 columns.
* Remove space after '('.
* Use '%pIX' instead of 'inet_ntop'.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Store VNI information in the data plane context so we can use it to
build the FPM netlink update with that information later.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Instead of retuning always `0`, lets return the amount of used bytes for
the message. This will be used by the new FPM interface to know how many
bytes we must reserve for the output buffer.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
* Use `inet_ntop` instead of `inet_ntoa`
* Replace function name with `__func__`
* Inline functions
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Generalize the netlink route message building function so it can be used
in the future by the netlink Forwarding Plane Manager (FPM) interface.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
In some places we log the interface but not the vfr the
interface is in. In others we only output the vrf id, which
can be difficult for human to read. This commit makes zebra
debugs more vrf aware.
Signed-off-by: Jakub Urbańczyk <xthaid@gmail.com>
In the netlink code for determining whether to set
a src on the route, we check if the cmd=NEW_ROUTE
but its not possible for this to ever be anything
but a new route since we do a goto skip further up
if its a DEL_ROUTE cmd.
So remove this unnecessary check.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Determine src based on nexthop data even when we are using
kernel nexthop objects.
Before, we were entirely skipping this step and just sending the
nexthop ID, ignoring src determination.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Abstraction the route src determination from a nexthop in the
netlink code into a function for both singlepath and mutlipath
to call.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Some logging systems are, er, "allergic" to tabs in log messages.
(RFC5424: "The syslog application SHOULD avoid octet values below 32")
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
We currently have netlink_neigh_update_ctx,
netlink_vxlan_flood_update_ctx and netlink_macfdb_update_ctx
all of which do slightly different RTM_NEWNEIGH calls into
the kernel. After this change, there will be one common
function.
Signed-off-by: Jakub Urbańczyk <xthaid@gmail.com>
1) When programming a nhg id to the kernel we had no debug of that
is what we are doing.
2) Add debugs to all nexthop information to allow us to follow
which prefix we are talking about. This is especially
useful when dealing with a large number of routes and
you want to grep out one or two too see what is going on.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Ensure that any weight associated with the next hop is installed for
IPv4 routes with IPv6 next hops too.
Updates: lib, zebra: Allow for installation of a weighted nexthop
Signed-off-by: Vivek Venkatraman <vivek@cumulusnetworks.com>
The return type of is_selfroute function is changed from int to bool.
Also remove the redundant invoking of the is_selfroute function in the
calling function netlink_route_change_read_unicast
Fixes: https://github.com/FRRouting/frr/issues/5984
Signed-off-by: Naveen Naidu <naveennaidu479@gmail.com>
Readd the special MAC that represents the flood (head-end replication) entry
for EVPN-VxLAN upon getting a delete notification for it.
Signed-off-by: Vivek Venkatraman <vivek@cumulusnetworks.com>
Reviewed-by: Chirag Shah <chirag@cumulusnetworks.com>
Ticket: CM-25797
Ticket: CM-26238
Testing Done:
1. evpn-min, evpn-smoke - results summarized in CM-25798
add debug trace in specific neigh request send api
to help debug an issue where synchronous response parse
returns with NLMSG_DONE where there is no ipv6 neigh received.
the count value is set to 1 because the request contained
a spcific neigh.
Signed-off-by: Chirag Shah <chirag@cumulusnetworks.com>
Nexthop groups as a whole do not make sense to have a vrf'ness
As that you can have a arbitrary number of nexthops that point
to separate vrf's.
Modify the code to make this distinction, by clearly delineating
the line between the nhg and the nexthop a bit better.
Nexthop groups having a vrf_id only make sense if you are using
network namespaces to represent them.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Add a config that disables use of kernel-level nexthop ids.
Currently, zebra always uses nexthop ids if the kernel supports
them.
Signed-off-by: Mark Stapp <mjs@voltanet.io>
zebra can catch the kernel's route deletion by netlink.
but current FRR can't delete kernel-route on vrf(l3mdev)
when kernel operator delete the route on out-side of FRR.
It looks problem about kernel-route deletion.
This problem is caused around _nexthop_cmp_no_labels(nh1,nh2)
that checks the each nexthop's member 'vrf_id'.
And _nexthop_cmp_no_labels's caller doesn't set the vrf_id
of nexthop structure. This commit fix that case.
Signed-off-by: Hiroki Shirokura <slank.dev@gmail.com>
We were not setting the RTNH_F_ONLINK flag where appropriate
when creating nexthop objects in the kernel.
Set it on the nhmsg.nh_flags netlink message.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Linux has the idea of allowing a weight to be sent
down as part of a nexthop group to allow the kernel
to weight particular nexthop paths a bit more or less
than others.
See:
http://tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.rpdb.multiple-links.html
Allow for installation into the kernel using the weight attribute
associated with the nexthop.
This code is foundational in that it just sets up the ability
to do this, we do not use it yet. Further commits will
allow for the pass through of this data from upper level protocols.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Replace the existing list of nexthops (via a nexthop_group
struct) in the route_entry with a direct pointer to zebra's
new shared group (from zebra_nhg.h). This allows more
direct access to that shared group and the info it carries.
Signed-off-by: Mark Stapp <mjs@voltanet.io>
This reverts commit 7d5bb02b1a.
Allow zebra to actually maintain the nexthop group in the
linux kernel.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Fix 2 Coverity issues:
1) zebra_nhg.c -> all paths in nhg_ctx_process_finish have
already deref'ed the ctx pointer no need for a test of it
2) the **ifp pointer passed in may be NULL. Prevent an accidental
deref if calling function does not pass in a ifp pointer.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Checkpatch was complaining because this code was extending
beyond 80 characters on a couple lines. Adjusted a conditional
tree to fix that.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Guard against an overflow read when processing
nexthop groups from netlink. Add a check to ensure
we don't try to write passed the array size.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add code for handling nexthop group hash entry encaps
and sending them to the kernel. Add some more debugging
information for the encaps and groups in general.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
There was some code copypasta for mpls stack building in the
netlink install path. Reduced that to a common function.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
When we receive a route delete from the kernel and it
contains a nexthop object id, use that to match against
route gateways with instead of explicit nexthops.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Move the supports_nh bool indicating whether the kernel we are
using supports nexthop objects into the netlink kernel interface
itself. Since only linux and netlink support nexthop object APIs
for now this is fine.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add handling for delete/update nexthop object messages from the
kernel.
If someone deletes a nexthop object we are still using, send it back
down. If the someone updates a nexthop we are using, replace that nexthop
with ours. Routes are referencing this nexthop object ID and we resolved
it ourselves, so we should force the other `someone` to submit to our
will.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
On restart, if we failed to remove any nexthop objects due
to a kill -9 or such event, sweep them if we aren't using them.
Add a proto field to handle this and remove the is_kernel bool.
Add a dupicate flag that indicates this nexthop group is only
present in our ID hashtable. It is a dupicate nexthop we received
from the kernel, therefore we cannot hash on it.
Make the idcounter globally accessible so that kernel updates
increment it as soon as we receive them, not when we handle them.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Give all nhg_hash_entrys we install into the kernel
as nexthop objects a defined proto matching the zebra
rib table one. This makes sense since nhe's are proto-independent
and determined exclusively in zebra.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add the ability to recursively resolve nexthop group hash entries
and resolve them when sending to the kernel.
When copying over nexthops into an NHE, copy resolved info as well.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
We were only setting and checking the ifindex if
the nexthop had an *_IFINDEX type. However, when nexthop
active checking is done, the non-*_IFINDEX types can also
obtain a nexthop with an ifindex and are thus valid too.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
We will use a nhe context for dataplane interaction with
nextho group hash entries.
New nhe's from the kernel will be put into a group array
if they are a group and queued on the rib metaq to be processed
later.
New nhe's sent to the kernel will be set on the dataplane context
with approprate ID's in the group array if needed.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Removing this function since the new paradigm
of everything just being nhg_connected structs
makes it not make a lot of sense.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Put the setting of the ifp on a nexthop group hash
entry into the zebra_nhg_alloc() function. It should
only be added if its not a group/recursive (it doesn't
have any depends) and its nexthop type has an ifindex.
This also provides functionality for proto-side ifp
setting.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
A nexthop group should not have a VRF ID. Only individual
nexthops need to be using a VRF. Fixed this both kernel and
proto side.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Re-organize and expose the nhg_connected functions so that
it can be used outside zebra_nhg.c. And then abstract those
into zebra_nhg_depends_* and zebra_nhg_depenents_* functons.
Switch the ifp struct to use an RB tree for its dependents,
making use of the nhg_connected functions.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Create a nhg_depenents tree that will function as a way
to get back pointers for NHE's depending on it.
Abstract the RB nodes into nhg_connected for both depends and
dependents. This same struct is used for both.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add a helper function to allow us to check if two
nhg_hash_entry's dependency lists are equal.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Make the the kernel debug zlog for nexthop messages from the
kernel more aligned with the route message kernel debug zlog.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Changed our alloc function to just copy the nhg and
nhg_depends. This makes the zebra_nhg_find code a
little bit cleaner, hopefully preventing bugs.
The only issue with this is that it makes us have to loop
over the nexthops in a group an extra time for the copies.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add functionality to allow us to send nexthop groups
to the kernel. It creates a nexthop_grp array based on
the dependency list in the nhg_hash_entry and then shoves
that into the netlink message.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Nexthop groups can have nexthops in different vrf's. So,
let's make the group vrf_id just be VRF_DEFAULT for hash
lookup purposes.
Set vrf_id to be VRF_DEFAULT for every message. If its a new
nextop, set the vrf to be the appropriate thing, otherwise
its a group and can just be left as default.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Simplify the code for nexthop hash entry creation. I made nexthop
hash entry creation expect the nexthop group and depends to always
be allocated before lookup. Before, it was only allocated if it had
dependencies. I think it makes the code a bit more readable to go
ahead an allocate even for single nexthops as well.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add functionality to read in a group from the kernel,
create a hash entry for it, and add its nexthops to
its dependency list.
Further, we create its nhg struct separtely from this,
copying over any nexthops it should reference directly
into it.
Thus, we have two types for representation of the nexthop group:
nhe->nhg_depends->[nhe, nhe, nhe]
nhe->nhg->nexthop->nexthop->nexthop
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
