When using the default CLI mode, the northbound layer needs to create
a separate transaction to process each YANG-modeled command since
they are supposed to be applied immediately (there's no candidate
configuration nor the "commit" command like in the transactional
CLI). The problem is that configuration transactions have an overhead
associated to them, in big part because of the use of some heavy
libyang functions like `lyd_validate()` and `lyd_diff()`. As of
now this overhead is substantial and doesn't scale well when large
numbers of transactions need to be performed in sequence.
As an example, loading 50k prefix-lists using a single transaction
takes about 2 seconds on a modern CPU. Loading the same 50k
prefix-lists using 50k transactions can take more than an hour
to complete (which is unacceptable by any standard). To fix this
problem, some heavy optimization work needs to be done on libyang and
on the FRR northbound itself too (e.g. perform partial configuration
diffs whenever possible). This, however, should be a long term
effort since these optimizations shouldn't be trivial to implement
and we're far from having the performance numbers we need.
In the meanwhile, this commit introduces a simple but efficient
workaround to alleviate the issue. In short, a new back-off timer
was introduced in the CLI to monitor and detect when too many
YANG-modeled commands are being received at the same time. When
a certain threshold is reached (100 YANG-modeled commands within
one second), the northbound starts to group all subsequent commands
into a single large transaction, which allows them to be processed
much faster (e.g. seconds and not hours). It's essentially a
protection mechanism that creates dynamically-sized transactions
when necessary to prevent performance issues from happening. This
mechanism is enabled both when parsing configuration files and when
reading commands from a terminal.
The downside of this optimization is that, if several YANG-modeled
commands are grouped into the same transaction and at least one of
them fails, the whole transaction is rejected. This is undesirable
since users don't expect transactional behavior when that's not
enabled explicitly. To minimize this issue, the CLI will log all
commands that were rejected whenever that happens, to make the
user aware of what happened and have enough information to fix
the problem. Commands that fail due to parsing errors or CLI-level
validations in general are rejected separately.
Again, this proposed workaround is intended to be temporary. The
goal is to provided a quick fix to issues like #6658 while we work
on better long-term solutions.
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
Rather than doing a f*gly hack for the RPKI code, let's do an on-exit
hook in cmd_node. Also allows replacing some special-casing in the vty
code.
Signed-off-by: David Lamparter <equinox@diac24.net>
The `--enable-pcreposix` configure option was not actually compiling
properly. Follow pre-existing pattern for inclusion of regex.h
or the pcreposix.h header.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
There's no need to check for removed configuration objects in the
following two cases:
* A private candidate configuration is being edited;
* The startup configuration is being read.
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
This makes printfrr extensions available in most of our format strings.
snprintf() is the obvious exception.
Signed-off-by: David Lamparter <equinox@diac24.net>
Add 'no log commands' cli and at the same time add a
--command-log-always to the daemon startup cli.
If --command-log-always is specified then all commands are
auto-logged and the 'no log commands' form of the command
is now ignored.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
The ability to lock the running configuration to prevent other users
from changing it is a very important one. We already supported
the "configure exclusive" command but the lock was applied to
the CLI users only (other clients like ConfD could still commit
configuration transactions, ignoring the CLI lock). This commit
introduces a global lock for the running configuration that is
shared by all northbound clients, and provides a public API to
manipulate it. This way other northbound clients will also be able
to lock/unlock the running configuration if required (the upcoming
gRPC northbound plugin will have RPCs for that).
NOTE: this is a management-level lock for the running configuration,
not to be confused with low-level locks used to avoid data races.
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
These are necessary to use functions defined in these headers from C++.
Signed-off-by: David Lamparter <equinox@diac24.net>
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
Just copying th const char* of the xpath means that if we
are enqueing multiple changes from a buffer, the last xpath
addedd will overwrite all of the previous references.
Copying the xpath to a buffer simplifies the API when
retrofitting the commands.
Signed-off-by: Emanuele Di Pascale <emanuele@voltanet.io>
Confirmed commits allow the user to request an automatic rollback to
the previous configuration if the commit operation is not confirmed
within a number of minutes. This is particularly useful when the user
is accessing the CLI through the network (e.g. using SSH) and any
configuration change might cause an unexpected loss of connectivity
between the user and the managed device (e.g. misconfiguration of a
routing protocol). By using a confirmed commit, the user can rest
assured the connectivity will be restored after the given timeout
expires, avoiding the need to access the router physically to fix
the problem.
When "commit confirmed TIMEOUT" is used, a new "commit" command is
expected to confirm the previous commit before the given timeout
expires. If "commit confirmed TIMEOUT" is used while there's already
a confirmed-commit in progress, the confirmed-commit timeout is
reset to the new value.
In the current implementation, if other users perform commits while
there's a confirmed-commit in progress, all commits are rolled back
when the confirmed-commit timeout expires. It's recommended to use
the "configure exclusive" configuration mode to prevent unexpected
outcomes when using confirmed commits.
When an user exits from the configuration mode while there's a
confirmed-commit in progress, the commit is automatically rolled
back and the user is notified about it. In the future we might
want to prompt the user if he or she really wants to exit from the
configuration mode when there's a pending confirmed commit.
Needless to say, confirmed commit only work for configuration
commands converted to the new northbound model. vtysh support will
be implemented at a later time.
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
A while ago all FRR configuration commands were converted to use the
QOBJ infrastructure to keep track of configuration objects. This
means the configuration lock isn't necessary anymore because the
QOBJ code detects when someones tries to edit a configuration object
that was deleted and react accordingly (log an error and abort the
command). The possibility of accessing dangling pointers doesn't
exist anymore since vty->index was removed.
Summary of the changes:
* remove the configuration lock and the vty_config_lockless() function.
* rename vty_config_unlock() to vty_config_exit() since we need to
clean up a few things when exiting from the configuration mode.
* rename vty_config_lock() to vty_config_enter() to remove code
duplication that existed between the three different "configuration"
commands (terminal, private and exclusive).
Configuration commands converted to the new northbound model don't
need the configuration lock either since the northbound API also
detects when someone tries to edit a configuration object that
doesn't exist anymore.
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
When editing the candidate configuration, the northbound must ensure
that either all changes made by a command are accepted or none are.
This is done to prevent inconsistent states where only parts of a
command are applied in the event any error happens.
The previous API for converted commands, the nb_cli_cfg_change()
function, required callers to pass an array containing all changes
that needed to be applied in the candidate configuration. The
problem with this API is that it was very inconvenient for complex
commands, which change different configuration options depending
on several factors. This required users to manipulate the array
of configuration changes using low-level primitives, making it
complicated to implement some commands.
To solve this problem, introduce a new API based on the two following
functions:
- nb_cli_enqueue_change()
- nb_cli_apply_changes()
The first function is used to enqueue configuration changes, one
at time. Then the nb_cli_apply_changes() function is used to apply
all the enqueued configuration changes.
To implement this, a static-sized array was allocated in the "vty"
structure, along with a counter of enqueued changes. This eliminates
the need to declare an array of configuration changes in every
converted CLI command, simplifying things quite considerably.
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
When reading in config files and we have failures on multiple
lines actually note the actual failure lines and return them.
This fixes an issue where we stopped counting errors after
the first one and we got missleading line numbers that
did not correspond to the actual problem.
This is fixed:
sharpd@donna ~/frr> sudo /usr/lib/frr/pimd --log=stdout -A 127.0.0.1 -f /etc/frr/pimd.conf
2018/10/11 09:41:01 PIM: VRF Created: default(0)
2018/10/11 09:41:01 PIM: pim_vrf_enable: for default
2018/10/11 09:41:01 PIM: zclient_lookup_sched_now: zclient lookup immediate connection scheduled
2018/10/11 09:41:01 PIM: zclient_lookup_new: zclient lookup socket initialized
2018/10/11 09:41:01 PIM: pimd 6.1-dev starting: vty@2611
2018/10/11 09:41:01 PIM: [EC 100663304] ERROR: No such command on config line 2: inteface lo
2018/10/11 09:41:01 PIM: [EC 100663304] ERROR: No such command on config line 3: ip igmp
2018/10/11 09:41:01 PIM: [EC 100663304] ERROR: No such command on config line 4: ip igmp join 224.1.1.1 13.13.13.2
^C2018/10/11 09:45:09 PIM: Terminating on signal SIGINT
2018/10/11 09:45:09 PIM: VRF Deletion: default(0)
Fixes: #3161
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
When reading the config file add an ability to know
if we have properly read in anything. So that a daemon
can make fallback plans.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Sometimes output would be mangled when filtering with include as a
result of the following bugs:
* Filters were applied per each call to vty_out() instead of buffering
until a line break and then applying
* Long output would sometimes be cut due to using the wrong buffer
pointer
Also remove the trailing \n as it should no longer be necessary to
ensure the vty prompt ends up on a new line.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
* Rewrite pager implementation
* Replace fprintf() with vty_out()
* Modify vty_out() for better vtysh support
* Remove static global outputfile var
* Remove fp argument from many vtysh functions
* Add some docs for stuff along the way
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
This patch adds a CLI preprocessor function that activates when `|` is
found in the command. This is the start of adding support for some text
processing utilities intended for inline use. The first one implemented
here is `| include`, which provides grep-like filtering of command
output.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
vty_frame() can be used to reduce the amount of output produced by "show
running-config" and "write ...". It buffers output in struct vty->frame
(1024 bytes) and outputs it when vty_out is called. If vty_out isn't
called, it can be removed with vty_endframe() later.
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
Multi-Instance OSPF configuration CLI would fail because
first client return error upon seeing qobj_index being 0.
With new marco generate new error code to return from each
instance (vtysh client) and if the command is intended for given
instance, its qobj_index would be nonzero and process the command
and push correct ospf context. Other instance would return the error.
On vtysh end, check all instance return an error log a message to a
file.
Testing Done:
Verfied various MI-OSPF configuration CLI with multi instances.
Signed-off-by: Chirag Shah <chirag@cumulusnetworks.com>
- SIGTSTP appropriately suspends the foreground terminal
- SIGINT causes the daemon to exit, regardless of -d
- SIGQUIT causes the daemon to daemonize, regardless of -d
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
Instead of having an ?: expression embedded in every single caller of
vty_out, just expand \n to \r\n in the vty code if neccessary.
(Deprecation warnings will be enabled in the next commits which will do
the search-and-replace over the codebase.)
[This reverts commit 4d5f445 "lib: add vty_outln()"]
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
* Fix bad format specifier in thread.[ch]
* Move PRINTF_ATTRIBUTE macro to zebra.h
* Use PRINTF_ATTRIBUTE on termtable printers
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
The xml2cli.pl script was useful years ago when the vty code was very
rudimentary. This is not the case anymore, so convert all ldpd CLI
commands to use DEFUNs directly and get rid of the XML interface.
The benefits are:
* Consistency with the other daemons;
* One less build dependency (the LibXML perl module);
* Easier to add new commands.
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
The FSF's address changed, and we had a mixture of comment styles for
the GPL file header. (The style with * at the beginning won out with
580 to 141 in existing files.)
Note: I've intentionally left intact other "variations" of the copyright
header, e.g. whether it says "Zebra", "Quagga", "FRR", or nothing.
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
Centralise read_config/daemonize/dryrun/pidfile/vty_serv into libfrr.
This also makes multi-instance pid/config handling available as part of
the library. It's only wired up in ospfd, but the code is in lib/.
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
This removes the automatic resizing of the vty input buffer and places a
hard size cap of 4096 bytes. It also fixes a potentially unsafe strcpy.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
[cherry-picked from master 2af38873d8]
This removes the automatic resizing of the vty input buffer and places a
hard size cap of 4096 bytes. It also fixes a potentially unsafe strcpy.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
