When using the default CLI mode, the northbound layer needs to create
a separate transaction to process each YANG-modeled command since
they are supposed to be applied immediately (there's no candidate
configuration nor the "commit" command like in the transactional
CLI). The problem is that configuration transactions have an overhead
associated to them, in big part because of the use of some heavy
libyang functions like `lyd_validate()` and `lyd_diff()`. As of
now this overhead is substantial and doesn't scale well when large
numbers of transactions need to be performed in sequence.
As an example, loading 50k prefix-lists using a single transaction
takes about 2 seconds on a modern CPU. Loading the same 50k
prefix-lists using 50k transactions can take more than an hour
to complete (which is unacceptable by any standard). To fix this
problem, some heavy optimization work needs to be done on libyang and
on the FRR northbound itself too (e.g. perform partial configuration
diffs whenever possible). This, however, should be a long term
effort since these optimizations shouldn't be trivial to implement
and we're far from having the performance numbers we need.
In the meanwhile, this commit introduces a simple but efficient
workaround to alleviate the issue. In short, a new back-off timer
was introduced in the CLI to monitor and detect when too many
YANG-modeled commands are being received at the same time. When
a certain threshold is reached (100 YANG-modeled commands within
one second), the northbound starts to group all subsequent commands
into a single large transaction, which allows them to be processed
much faster (e.g. seconds and not hours). It's essentially a
protection mechanism that creates dynamically-sized transactions
when necessary to prevent performance issues from happening. This
mechanism is enabled both when parsing configuration files and when
reading commands from a terminal.
The downside of this optimization is that, if several YANG-modeled
commands are grouped into the same transaction and at least one of
them fails, the whole transaction is rejected. This is undesirable
since users don't expect transactional behavior when that's not
enabled explicitly. To minimize this issue, the CLI will log all
commands that were rejected whenever that happens, to make the
user aware of what happened and have enough information to fix
the problem. Commands that fail due to parsing errors or CLI-level
validations in general are rejected separately.
Again, this proposed workaround is intended to be temporary. The
goal is to provided a quick fix to issues like #6658 while we work
on better long-term solutions.
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
DEFPY_YANG will allow the CLI to identify which commands are
YANG-modeled or not before executing them. This is going to be
useful for the upcoming configuration back-off timer work that
needs to commit pending configuration changes before executing a
command that isn't YANG-modeled.
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
When we have a prefix that has been selected, note that that
particular flag has been set and give that information to the
end user.
eva# show bgp ipv4 uni neighbors 192.168.161.131 prefix-counts
Prefix counts for 192.168.161.131, IPv4 Unicast
PfxCt: 814246
Counts from RIB table walk:
Adj-in: 0
Damped: 0
Removed: 0
History: 0
Stale: 0
Valid: 814246
All RIB: 814246
PfxCt counted: 814246
PfxCt Best Selected: 0
Useable: 814246
eva# show bgp ipv4 uni neighbors 192.168.161.2 prefix-counts
Prefix counts for 192.168.161.2, IPv4 Unicast
PfxCt: 814070
Counts from RIB table walk:
Adj-in: 0
Damped: 0
Removed: 0
History: 0
Stale: 0
Valid: 814070
All RIB: 814070
PfxCt counted: 814070
PfxCt Best Selected: 814070
Useable: 814070
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
nhrp listens for route entries to be deleted, in case some new routes
impact the current routes installed by nhrp. To prevent from
unconfiguring nhrp shortcut route, just prevent nhrp routes to be
processed.
Signed-off-by: Philippe Guibert <philippe.guibert@6wind.com>
Address-sanitizer runs in the CI appear to require more
memory than is available (at present), so skip the top
x32 route_scale testcase when running with <4G of ram.
Signed-off-by: Mark Stapp <mjs@voltanet.io>
Make some changes to the route-scale topotest, in view of
issue #6734. Table-drive the test to eliminate some
repeated code. Assert and fail if a step in the progression
of scale fails. Wait a little longer between checking the show
output - it's costly to generate that output at scale. Add a
memleak testcase.
Signed-off-by: Mark Stapp <mjs@voltanet.io>
The `INET_ADDRSTRLEN` is 16 and is only enough to format an IPv4 address.
So when there is a prefix (`/xx`), the debug output may get truncated.
Use `PREFIX2STR_BUFFER` macro instead to fix the issue.
Signed-off-by: Aaron LI <aly@aaronly.me>
1. evpn_type5_test_topo1 tests started failing in CI for all Ubuntu 18.04 machine,
which are having kernel version: 5.4.0-42-generic
2. We will enable these tests once issue is found and fixed.
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
We were not getting any benefits from attempting to walk all tables at the
same time and it made debugging harder, so lets execute one table walk
per time.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Zebra runs on a different thread than FPM, so we need to synchronize
them by using events. While here, implement completion detection for all
kinds of walk.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Two important fixes:
* `stream_read_try` does a dirty trick and converts the `-1` return to
`-2` when errno is `EAGAIN`, `EWOULDBLOCK` or `EINTR`.
* Don't enable reads until the connection is complete.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
BGP Yang is using sub modules and at present FRR is not processing
submodules in embedded framework yang
Signed-off-by: VishalDhingra <vdhingra@vmware.com>
The `log monitor' command is a no-op and actually
outputs a `this doesn't do anything` warning. Let's remove
this cli line from our tests as that don't do anything and
people will look at these configs for guidance.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
This code was deprecated in 5.0 and removed after a year.
It has not been in the code base and we forgot to update the
doc.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Coverity has noticed that we are using bgp_evpn after
we have already NULL checked it one time. Add an assert
to make Coverity happy here, if we get to this point
something terrible has happened.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Coverity rightly points out that bgp_table_top might return
NULL and immediately deref'ing it might be a problem.
Add a bit of safety.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
I wanted to preserve the old code flow to see what might
be needed in the future in commit:
23ca3269da
Coverity doesn't like dead code. So let's comment it out.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Recent changes to remove PRIu... in commit:
6cde4b4552
causes clang 11 to be unhappy, with length of field warnings.
Modify the offending code to compile properly using that compiler.
I've tested against clang 11 and gcc 9.3
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
The documentation for this command was missing. Add a little
bit of data for people in the future.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
If _force_ is set, then ALL prefixes are counted for maximum instead of
accepted only. This is useful for cases where an inbound filter is applied,
but you want maximum-prefix to act on ALL (including filtered) prefixes.
For instance, we have a configuration like:
neighbor r1 maximum-prefix 10
neighbor r1 prefix-list custom in
!
ip prefix-list custom seq 1 permit 10.0.0.0/24
ip prefix-list custom seq 2 permit 10.0.1.0/24
This will accept only 2 prefixes and discard all others instead of
shutting down the session when 10 is reached.
With this new knob (force), we will count all received prefixes and shutdown
the session when 10 is reached.
The bigger problem is when you have lots of peers with full feed and such a
configuration like in an example.
This is kinda re-ordering of how to treat filter vs. maximum-prefix.
Signed-off-by: Donatas Abraitis <donatas.abraitis@gmail.com>
