Replace the existing list of nexthops (via a nexthop_group
struct) in the route_entry with a direct pointer to zebra's
new shared group (from zebra_nhg.h). This allows more
direct access to that shared group and the info it carries.
Signed-off-by: Mark Stapp <mjs@voltanet.io>
Apparently the multipath_num functionatlity has been broken
for a while because we were ignoring the recusive nexthops
when marking them inactive based on it.
This sets them as inactive as well if the parent breaks it.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
We were re-counting the entire group's active number on
every iteration of this nexthop_active_update() loop.
This is not great from a performance perspective but also
it was failing to properly mark things according to the
specified multipath_num.
Since a nexthop is set as active before this check, if its == to
the set ecmp, it gets marked inactive even though if its
under the max ecmp wanted!
ex)
set ecmp to 1.
`/usr/lib/frr/zebra -e 1`
All kernel routes will be marked inactive even with just one nexthop!
K 1.1.1.1/32 [0/0] is directly connected, dummy1 inactive, 00:00:10
K 1.1.1.2/32 [0/0] is directly connected, dummy2 inactive, 00:00:10
K 1.1.1.3/32 [0/0] is directly connected, dummy3 inactive, 00:00:10
K 1.1.1.4/32 [0/0] is directly connected, dummy4 inactive, 00:00:10
K 1.1.1.5/32 [0/0] is directly connected, dummy5 inactive, 00:00:10
K 1.1.1.6/32 [0/0] is directly connected, dummy6 inactive, 00:00:10
K 1.1.1.7/32 [0/0] is directly connected, dummy7 inactive, 00:00:10
K 1.1.1.8/32 [0/0] is directly connected, dummy8 inactive, 00:00:10
K 1.1.1.9/32 [0/0] is directly connected, dummy9 inactive, 00:00:10
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Clean up the relationships between zebra's rib and nexthop-group
headers as prep for adding a nexthop-group pointer to the
route_entry.
Signed-off-by: Mark Stapp <mjs@voltanet.io>
Put the code to free the data held by a nhg_ctx
in nhg_ctx_free() as well. We do it similiarly for
the dplane_ctx.
Let nhg_ctx_fini() be any other routines that need to
be handled before freeing.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
SA warned us lookup could be NULL dereferenced in some
paths. Handle the case where we are passed a NULL
nexthop before we try to copy it.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
We were only checking that two nhg_hash_entry's were equal
based on the active nexthop NUMBER. This is not sufficient in
special cases where whats active with one route using it,
might not be active with the other. We can see this with
routes trying to resolve to themselves.
Ex)
1.1.1.0/24
-> 1.1.1.1 dummy1 (inactive)
-> 1.1.1.2 dummy2
1.1.2.0/24
-> 1.1.1.1 dummy1
-> 1.1.1.2 dummy1 (inactive)
Without checking each nexthop individually, they will
hash to the same group since they have the same number of
active nexthops.
Fix this by looping over every nexthop for each nhe (they should
be sorted) and checking if the NEXTHOP_FLAG_ACTIVE flag's match.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Fix 2 Coverity issues:
1) zebra_nhg.c -> all paths in nhg_ctx_process_finish have
already deref'ed the ctx pointer no need for a test of it
2) the **ifp pointer passed in may be NULL. Prevent an accidental
deref if calling function does not pass in a ifp pointer.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Add a private header file for functions that are internal/special
case like how we do it for `lib/nexthop_group_private.h`.
Remove a bunch of functions from the header file only being used
statically and add some comments for those remaining to indicate
better what their use is.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Re-work the validity setting and checking APIs
for nhg_hash_entry's to make them clearer.
Further, they were originally only beings set
on ifdown and install. Extended their use into
releasing entries and to account for setting
the validity of a recursive dependent.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
The commenting for why we would need to requeue a
group from the kernel to be later processed was not
sufficient. Add a better explanation for the flow
and state of the system.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Change the wording of the flag indicating we have received
a nexthop group from the kernel with a different ID but
is fundamentally identical to one we already have.
It was colliding with a flag of similar name in the nexthop struct.
Change it from NEXTHOP_GROUP_DUPLICATE -> NEXTHOP_GROUP_UNHASHABLE
since it is in fact unhashable.
Also change the wording of functions and comments referencing the same
problem.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
When determining whether to set the nhg_hash_entry as
invalid, we should have been checking the depends, not
the dependents. If its a group and at least one of its
depends is valid, the group is still valid.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Now with this patch we can't use shutdown for cleanup:
```
commit 2fc69f03d2 (pr_5079)
Author: Mark Stapp <mjs@voltanet.io>
Date: Fri Sep 27 12:15:34 2019 -0400
zebra: during shutdown processing, drop dplane results
Don't process dataplane results in zebra during shutdown (after
sigint has been seen). The dplane continues to run in order to
clean up, but zebra main just drops results.
Signed-off-by: Mark Stapp <mjs@voltanet.io>
```
Adjusted nhg uninstall handling to clear data and other
cleanup before sending to the dataplane.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Reduce the api for deleting nexthops and the containing
group to just one call rather than having a special case
and handling it separately.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Check both the nhg and nexthop are not NULL before passing
them to be hashed. Clang SA caught this.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Only used the afi passed into `zebra_nhg_find()` for nexthops
that are blackhole/ifindex. Others should use the type actually declared
in the nexthop struct itself.
Basically, nexthop objects of type blackhole/ifindex in the kernel must
have an address family, they cannot be ambigious and be shared.
This is some requirement in the linux ip core code.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add a mechanism to requeue groups we receive from the
kernel if the IDs are in a weird order (Group ID is lower
than individual nexthop IDs for example).
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
If we get a nexthop group from the kernel with labels
and queue it as a context to process later, we have to
free the label stack we allocated.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add some getters for the nhg_ctx struct. Probably unnecessary
at this point since they are all static but if they ever become
public it will be nice to have them.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Update nhg_hash_entry to use the non-recursive version of
nexthop_group_equal() since it doesn't really need to compare all
of those.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
We were waiting until install time to mark nexthops as duplicate.
Since they are immutable now and re-used, move this marking into
when they are actually created to save a bunch of cycles.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Before checking the equivalence of the whole group itself,
check to see if they contain the same number of non-recursive
active nexthops. This should shorten lookup time for the case of
non-resolved nexthop group creation.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Create any depends only after the initial hash lookup
fails. Should reduce hashing cpu cycles significantly.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Move the supports_nh bool indicating whether the kernel we are
using supports nexthop objects into the netlink kernel interface
itself. Since only linux and netlink support nexthop object APIs
for now this is fine.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add handling for delete/update nexthop object messages from the
kernel.
If someone deletes a nexthop object we are still using, send it back
down. If the someone updates a nexthop we are using, replace that nexthop
with ours. Routes are referencing this nexthop object ID and we resolved
it ourselves, so we should force the other `someone` to submit to our
will.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
On restart, if we failed to remove any nexthop objects due
to a kill -9 or such event, sweep them if we aren't using them.
Add a proto field to handle this and remove the is_kernel bool.
Add a dupicate flag that indicates this nexthop group is only
present in our ID hashtable. It is a dupicate nexthop we received
from the kernel, therefore we cannot hash on it.
Make the idcounter globally accessible so that kernel updates
increment it as soon as we receive them, not when we handle them.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Give all nhg_hash_entrys we install into the kernel
as nexthop objects a defined proto matching the zebra
rib table one. This makes sense since nhe's are proto-independent
and determined exclusively in zebra.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
The kernel does not allow duplicate IDs in the same group, but
we are perfectly find with it internally if two different
nexthops resolve the the same nexthop (default route for instance).
So, we have to handle this when we get ready to install.
Further, pass the max group size in the arguments to ensure we
don't overflow. Don't actually think this is possible due to
multipath checking in nexthop_active_update() but better to be
safe.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
If the nhe was successfully installed, make sure its marked
as valid. Not fully sure how/where the valid flag is going to
be used yet.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
We were setting a group to be recursive if its first depend
was. This is not the case; individual depends of the group
might be recursive but the group itself is not.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Move the resolving and installing of a single nhg_hash_entry
into the install function itself, rather than letting zebra_rib
handle it.
Further, ensure depends are installed/queued before installing
a group. The ordering should be find here since only one thread
will call this API.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Move the installation of an nhe out of nexthop_active_update()
and into the rib install path. So, only install the nhe when
a route using it is being installed.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Switch the nhg_connected tree structures to use the new
RB tree API in `lib/typerb.h`. We were using the openbsd-tree
implementation before.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
We were not setting the NEXTHOP_GROUP_RECURSIVE flag via
the rib find path. Adding a check and set after successful
creation of a new nhg_hash_entry.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
When going through the zebra_nhg_rib_find(), we now handle the
case of if that nexthop has been recursively resolved. A depend
is created and passed along to zebra_nhg_find().
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add a refcnt as soon as depend is connected to mark
that this is being referenced as part of a group or
resolving another one. If the one referencing it
is never used, decrement it.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add some helper functions for finding/creating nexthop
group hash entries and assigning them as a depends for
another one using them in a group or resolving to them.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add some helper functions for ref incrementing and
decrementing the depends of a nexthop group hash entry.
This just abstracts the RB tree manipulation a bit more.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Set the resolved nhg during the find path, rather
than after it has been created. This make more sense
now that we are hashing on the resolved nexthop as well.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Refactor/move around the code for nexthop resolution so
that it occurs only when the nexthop actually changes. Further,
provide a helper function to make the code more readable.
Also, remove the check for NEXTHOPS_CHANGED as this flag is used
specifcially for nexthop tracking and not an appropriate check
here.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
When hashing/creating the NHE, use the nexthops vrf as its
source of data. This is gotten directly from an interface
and should not come from a route.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
When updating a route's referenced NHE, accept a NULL value
as valid and clear out the pointer in the struct.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
When the resolved nexthop changes, we should increment the new
resolved NHE by the refcnt for the unresolved NHE being used
by the routes and decrement the old one by the same amount.
Before, we were simple incrementing by one, causing incorrect refcnts
to occur.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Ignore the cleanup for now until we get the timing
figured out without using the kernel nexthop object API.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
If the nhg_hash_entry is a group, check if its members
are valid before setting it invalid. If even one is valid,
then this group should still be considered valid.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
In zebra_nhg_find(), if we created a nhg_hash_entry, return
true so we know rib-side.
Kernel-side, we don't care since it will always just enqueue
a context to process later.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add the ability to recursively resolve nexthop group hash entries
and resolve them when sending to the kernel.
When copying over nexthops into an NHE, copy resolved info as well.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
We were only setting and checking the ifindex if
the nexthop had an *_IFINDEX type. However, when nexthop
active checking is done, the non-*_IFINDEX types can also
obtain a nexthop with an ifindex and are thus valid too.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
We will use a nhe context for dataplane interaction with
nextho group hash entries.
New nhe's from the kernel will be put into a group array
if they are a group and queued on the rib metaq to be processed
later.
New nhe's sent to the kernel will be set on the dataplane context
with approprate ID's in the group array if needed.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Upon release, call the approprate functions to remove itself
from depends/dependents trees it is in.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add some functions to iterate over the depends/dependents
RB tree and remove themselves from the other's RB tree.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Can't RM_REMOVE directly with a key, you need to actually pass the
data to be removed. So, lookup with a key first to find the node,
then remove it.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Removed a static function that did not need to be
there. The nhg_connected_cmp() function provides
all the needed functionality for comparing ID's
in the RB tree.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Update the zebra_nhg_hash_equal() function to use
the nexthop_group_equal() function in lib/nexthop_group
instead of comparing their depends RB tree.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Removing this function since the new paradigm
of everything just being nhg_connected structs
makes it not make a lot of sense.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Put the setting of the ifp on a nexthop group hash
entry into the zebra_nhg_alloc() function. It should
only be added if its not a group/recursive (it doesn't
have any depends) and its nexthop type has an ifindex.
This also provides functionality for proto-side ifp
setting.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Re-organize and expose the nhg_connected functions so that
it can be used outside zebra_nhg.c. And then abstract those
into zebra_nhg_depends_* and zebra_nhg_depenents_* functons.
Switch the ifp struct to use an RB tree for its dependents,
making use of the nhg_connected functions.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Create a nhg_depenents tree that will function as a way
to get back pointers for NHE's depending on it.
Abstract the RB nodes into nhg_connected for both depends and
dependents. This same struct is used for both.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add function to increment the route reference count for nhg_hash_entry's
and to do so recursively if its a group.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add a helper function to allow us to check if two
nhg_hash_entry's dependency lists are equal.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add helper function to allow us to lookup an ID inside
of a nhg_hash_entry's dependency list.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add a function that allows us to take a single
nexthop struct and look that up or create a group and
nexthop hash entry with it.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Pass a boolean to zebra_nhg_find(), indicating whether the
nhg is being lookedup from the kernel side or not.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Move the id counter further up into zebra_nhg_find() so that
it is still incremented if we receive a duplicate that never
would get allocated. The kernel will still use the dup, so we
have to account for that in our id counter.
Also, if we don't create a new entry, reset the id back to where
it was when zebra_nhg_find() was called.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Changed our alloc function to just copy the nhg and
nhg_depends. This makes the zebra_nhg_find code a
little bit cleaner, hopefully preventing bugs.
The only issue with this is that it makes us have to loop
over the nexthops in a group an extra time for the copies.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Fix a couple functions that were using depends (plural)
rather than depend(singular) in their wording.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add a function to duplicate a nhg dependency linked
list. We will use this for duplicating the dependency
list rather than the linked list dup function in lib/linkedlist.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Simplify the code for nexthop hash entry creation. I made nexthop
hash entry creation expect the nexthop group and depends to always
be allocated before lookup. Before, it was only allocated if it had
dependencies. I think it makes the code a bit more readable to go
ahead an allocate even for single nexthops as well.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add some functions that can be called to free everything that should
have been allocated in a nexthop group hash entry.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add functionality to read in a group from the kernel,
create a hash entry for it, and add its nexthops to
its dependency list.
Further, we create its nhg struct separtely from this,
copying over any nexthops it should reference directly
into it.
Thus, we have two types for representation of the nexthop group:
nhe->nhg_depends->[nhe, nhe, nhe]
nhe->nhg->nexthop->nexthop->nexthop
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
We treat "groups" from the kernel here as a dependency list.
Each hash entry, if its a group from the kernel, has
a list of any other nexthop hash entries that are in its
group. A non-group nexthop from the kernel will have its
dependency list set to NULL.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
The nexthop group hash entries were using the "TMP" memory
type. Declared one for them and updated to use it.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Changed to the wording in the duplicate error message
since its techincally possible we get could try to
create a dupe from somewhere else besides the kernel
in the future.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add an interface pointer for an nexthop group hash entry
when we are getting a rib_add for a new route.
Also, add the interface index to the `show nexthop-group` command.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add functionality to uninstall nexthops we created on shutdown.
To account for this, I added in a function for zebra_router
cleanup in a shutdown event.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
When nexthop entry reference counts hit zero and
we created them, uninstall them from the kernel.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add a function that can handle the results of a dataplane
ctx status, dpending on the operation performed.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add functions for sending a nexthop to be queued on the dataplane
for install/uninstall into the kernel.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
