Go to file
Renato Westphal b855e95fd3 lib: introduce configuration back-off timer for YANG-modeled commands
When using the default CLI mode, the northbound layer needs to create
a separate transaction to process each YANG-modeled command since
they are supposed to be applied immediately (there's no candidate
configuration nor the "commit" command like in the transactional
CLI). The problem is that configuration transactions have an overhead
associated to them, in big part because of the use of some heavy
libyang functions like `lyd_validate()` and `lyd_diff()`. As of
now this overhead is substantial and doesn't scale well when large
numbers of transactions need to be performed in sequence.

As an example, loading 50k prefix-lists using a single transaction
takes about 2 seconds on a modern CPU. Loading the same 50k
prefix-lists using 50k transactions can take more than an hour
to complete (which is unacceptable by any standard). To fix this
problem, some heavy optimization work needs to be done on libyang and
on the FRR northbound itself too (e.g. perform partial configuration
diffs whenever possible).  This, however, should be a long term
effort since these optimizations shouldn't be trivial to implement
and we're far from having the performance numbers we need.

In the meanwhile, this commit introduces a simple but efficient
workaround to alleviate the issue. In short, a new back-off timer
was introduced in the CLI to monitor and detect when too many
YANG-modeled commands are being received at the same time. When
a certain threshold is reached (100 YANG-modeled commands within
one second), the northbound starts to group all subsequent commands
into a single large transaction, which allows them to be processed
much faster (e.g. seconds and not hours).  It's essentially a
protection mechanism that creates dynamically-sized transactions
when necessary to prevent performance issues from happening. This
mechanism is enabled both when parsing configuration files and when
reading commands from a terminal.

The downside of this optimization is that, if several YANG-modeled
commands are grouped into the same transaction and at least one of
them fails, the whole transaction is rejected. This is undesirable
since users don't expect transactional behavior when that's not
enabled explicitly. To minimize this issue, the CLI will log all
commands that were rejected whenever that happens, to make the
user aware of what happened and have enough information to fix
the problem. Commands that fail due to parsing errors or CLI-level
validations in general are rejected separately.

Again, this proposed workaround is intended to be temporary. The
goal is to provided a quick fix to issues like #6658 while we work
on better long-term solutions.

Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
2020-08-03 15:17:03 -03:00
.github .github: move pr template to subdir 2019-03-29 16:51:58 +00:00
alpine alpine: enable multi-arch builds 2020-06-24 16:33:18 -04:00
babeld *: un-split strings across lines 2020-07-14 10:37:25 +02:00
bfdd *: introduce DEFPY_YANG & friends 2020-08-03 15:17:03 -03:00
bgpd bgpd: Add to neighbor prefix-counts the count of best path selected 2020-07-31 10:06:39 -04:00
debian debian: Add missing daemons to logrotation knowledge 2020-06-25 07:13:50 -04:00
doc doc: Remove Cisco Compatability section 2020-07-27 07:57:13 -04:00
docker docker: don't fail on chown /etc/frr 2020-06-10 00:20:04 -04:00
eigrpd *: introduce DEFPY_YANG & friends 2020-08-03 15:17:03 -03:00
fpm *: Replace sizeof something to sizeof(something) 2020-03-08 21:44:53 +02:00
gdb bgpd: Convert binfo to path 2018-10-09 14:26:30 -04:00
grpc build: add LLVM bitcode targets 2020-05-05 14:39:12 +02:00
include zebra, lib: add an internal API to get relative default nsid in other ns 2020-05-18 14:11:03 +02:00
isisd *: introduce DEFPY_YANG & friends 2020-08-03 15:17:03 -03:00
ldpd ldpd: process pw-status in received order 2020-07-20 09:08:15 -04:00
lib lib: introduce configuration back-off timer for YANG-modeled commands 2020-08-03 15:17:03 -03:00
m4 build: find all future minor versions of python3 2020-07-09 06:47:31 +02:00
mlag zebra: Do not build mlag protobuf support if version 3 is not avail 2019-12-15 09:37:51 -05:00
nhrpd *: un-split strings across lines 2020-07-14 10:37:25 +02:00
ospf6d Merge pull request #6732 from opensourcerouting/printfrr-prep 2020-07-29 14:29:34 -04:00
ospfclient *: Replace s_addr 0 => INADDR_ANY 2020-02-06 09:00:12 +02:00
ospfd ospfd: Fix Zebra route add message truncation issue 2020-07-30 08:16:18 -07:00
pbrd Merge pull request #6730 from wesleycoakley/pbrd-dscp-ecn 2020-07-23 12:08:38 -04:00
pimd *: pim igmp yang registery to appropriate makefile 2020-07-22 23:22:23 -07:00
pkgsrc *: cleanup .gitignore files 2018-09-08 21:30:42 +02:00
python build: fix Makefile rule redefinition 2020-07-17 05:54:54 +02:00
qpb build: add LLVM bitcode targets 2020-05-05 14:39:12 +02:00
redhat redhat: Update logrotate to have knowledge of all daemons 2020-06-25 07:12:59 -04:00
ripd *: introduce DEFPY_YANG & friends 2020-08-03 15:17:03 -03:00
ripngd *: introduce DEFPY_YANG & friends 2020-08-03 15:17:03 -03:00
sharpd lib,sharpd,zebra: initial support for multiple backup nexthops 2020-07-17 13:12:33 -04:00
snapcraft snapcraft: Add vrrpd to the snapcraft package 2019-11-19 10:46:33 +01:00
solaris doc: rename man pages to frr-* 2020-01-15 14:05:11 +01:00
staticd *: introduce DEFPY_YANG & friends 2020-08-03 15:17:03 -03:00
tests tests: Avoid top ecmp route_scale test case when memory limited 2020-07-30 14:23:52 -04:00
tools Merge pull request #6732 from opensourcerouting/printfrr-prep 2020-07-29 14:29:34 -04:00
vrrpd *: introduce DEFPY_YANG & friends 2020-08-03 15:17:03 -03:00
vtysh *: introduce DEFPY_YANG & friends 2020-08-03 15:17:03 -03:00
watchfrr watchfrr: add (network) namespace support 2020-07-22 12:56:04 +02:00
yang yang: route-map model description format 2020-07-31 16:21:45 -07:00
zebra *: introduce DEFPY_YANG & friends 2020-08-03 15:17:03 -03:00
.clang-format clang-format: add FOREACH_SAFI to the ForEachMacros list 2020-08-03 12:18:24 -03:00
.dir-locals.el tools: fix emacs configuration file 2019-11-04 11:45:52 -03:00
.dockerignore docker: Make docker image on CentOS 7 2019-11-26 19:29:30 +00:00
.gitignore python: add graphviz callgraphs 2020-05-05 14:39:12 +02:00
bootstrap.sh autoreconf -i 2007-02-06 19:28:28 +00:00
buildtest.sh config: switch a few references to say FRR 2017-07-12 11:25:33 -05:00
changelog-auto.in debian: cleanly split off from dist tarball 2019-02-19 21:31:18 +01:00
config.version.in build: carry --with-pkg-extra-version into tarballs 2018-10-24 15:11:50 +02:00
configure.ac build: Allow removal of build configs from version string 2020-06-19 11:15:43 -05:00
COPYING *: make consistent & update GPLv2 file headers 2017-05-15 16:37:41 +02:00
COPYING-LGPLv2.1 build: remove LGPL v2.0, add LGPL v2.1 2016-11-15 17:19:38 +09:00
Makefile.am python: add graphviz callgraphs 2020-05-05 14:39:12 +02:00
README.md README.md: Add FRR Icon 2019-10-29 12:42:06 -04:00
stamp-h.in Initial revision 2002-12-13 20:15:29 +00:00

Icon

FRRouting

FRR is free software that implements and manages various IPv4 and IPv6 routing protocols. It runs on nearly all distributions of Linux and BSD as well as Solaris and supports all modern CPU architectures.

FRR currently supports the following protocols:

  • BGP
  • OSPFv2
  • OSPFv3
  • RIPv1
  • RIPv2
  • RIPng
  • IS-IS
  • PIM-SM/MSDP
  • LDP
  • BFD
  • Babel
  • PBR
  • OpenFabric
  • VRRP
  • EIGRP (alpha)
  • NHRP (alpha)

Installation & Use

For source tarballs, see the releases page.

For Debian and its derivatives, use the APT repository at https://deb.frrouting.org/.

Instructions on building and installing from source for supported platforms may be found in the developer docs.

Once installed, please refer to the user guide for instructions on use.

Community

The FRRouting email list server is located here and offers the following public lists:

Topic List
Development dev@lists.frrouting.org
Users & Operators frog@lists.frrouting.org
Announcements announce@lists.frrouting.org

For chat, we currently use Slack. You can join by clicking the "Slack" link under the Participate section of our website.

Contributing

FRR maintains developer's documentation which contains the project workflow and expectations for contributors. Some technical documentation on project internals is also available.

We welcome and appreciate all contributions, no matter how small!

Security

To report security issues, please use our security mailing list:

security [at] lists.frrouting.org