- Previously was substituting a pointer to local allocated session for the
session_id returned from the FE adapter. This complexity isn't needed.
- Get rid of "%llu" format and the casts that came with it, instead use PRIu64
and the actual (uint64_t) type.
Signed-off-by: Christian Hopps <chopps@labn.net>
When the user specifies `--command-log-always` in CLI arguments then also log
commands executed from loading the config file.
Signed-off-by: Christian Hopps <chopps@labn.net>
When daemons transition to mgmtd they should stop reading their split config
files, and let mgmtd do that, otherwise things can get out of sync.
Signed-off-by: Christian Hopps <chopps@labn.net>
Effectively a massive search and replace of
`struct thread` to `struct event`. Using the
term `thread` gives people the thought that
this event system is a pthread when it is not
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
This is a first in a series of commits, whose goal is to rename
the thread system in FRR to an event system. There is a continual
problem where people are confusing `struct thread` with a true
pthread. In reality, our entire thread.c is an event system.
In this commit rename the thread.[ch] files to event.[ch].
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
This commit contains fixes for the following issues found
- 'mgmt commit check' issued through 'vtysh -f' was actually commtting the changeset.
- On config validation failure backend, mgmtd was not passing the correct error-reason
to frontend.
- 'mgmt rollback ...' was reverting the change on backend, but config on mgmtd daemon
remains intact
Signed-off-by: Pushpasis Sarkar <pushpasis@gmail.com>
This commit introduces the Frontend Interface which can be used
by front-end management clients like Netconf server, Restconf
Server and CLI to interact with new FRR Management daemon (MGMTd)
to access and sometimes modify FRR management data.
This commit includes the following functionalities in the changeset:
1. Add new Frontend server for clients connect to.
2. Add a C-based Frontend client library which can be used by Frontend
clients to communicate with MGMTd via the Frontend interface.
3. Maintain a frontend adapter for each connection from an appropriate
Frontend client to facilitate client requests and track one or more
client sessions across it.
4. Define the protobuf message format for messages to be exchanged
between MGMTd Frontend module and the Frontend client.
5. This changeset also introduces an instance of MGMT Frontend client
embedded within the lib/vty module that can be leveraged by any FRR
daemon to connect to MGMTd's Frontend interface. The same has been
integrated with and initialized within the MGMTd daemon's process
context to implement a bunch of 'set-config', 'commit-apply',
'get-config' and 'get-data' commands via VTYSH
Co-authored-by: Pushpasis Sarkar <pushpasis@gmail.com>
Co-authored-by: Abhinay Ramesh <rabhinay@vmware.com>
Co-authored-by: Ujwal P <ujwalp@vmware.com>
Signed-off-by: Yash Ranjan <ranjany@vmware.com>
Initial commit: 23b2a7ef52
changed the json output of `show bgp <afi> <safi> json` to
not have pretty print because when under a situation where
there are a bunch of routes with a large scale ecmp show
output was taking forever and this commit cut 2 minutes out
of vtysh run time.
Subusequent commit: f4ec52f7cc
changed this back.
When upgrading to latest version the long run time was noticed
due to testing. Let's add back this functionality such that
FRR can have reduced run times with vtysh when it's really
needed.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Some results:
```
====
PCRE
====
% ./a.out "^65001" "65001"
comparing: ^65001 / 65001
ret status: 0
[14:31] donatas-pc donatas /home/donatas
% ./a.out "^65001_" "65001"
comparing: ^65001_ / 65001
ret status: 0
=====
PCRE2
=====
% ./a.out "^65001" "65001"
comparing: ^65001 / 65001
ret status: 0
[14:30] donatas-pc donatas /home/donatas
% ./a.out "^65001_" "65001"
comparing: ^65001_ / 65001
ret status: 1
```
Seems that if using PCRE2, we need to escape outer `()` chars and `|`. Sounds
like a bug.
But this is only with some older PCRE2 versions. With >= 10.36, I wasn't able
to reproduce this, everything is fine and working as expected.
Adding _FRR_PCRE2_POSIX definition because pcre2posix.h does not have
include's guard.
Signed-off-by: Donatas Abraitis <donatas@opensourcerouting.org>
There's a common pattern of "get VRF context for CLI node" here, which
first got a helper macro in zebra that then permeated into pimd.
Unfortunately the pimd copy wasn't quite adjusted correctly and thus
caused two coverity warnings (CID 1517453, CID 1517454).
Fix the PIM one, and clean up by providing a common base macro in
`lib/vty.h`.
Also rename the macros (add `_VRF`) to make more clear what they do.
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
This adds the plumbing necessary to yield back a file descriptor to
vtysh. The fd is passed on the command status code bytes through
AF_UNIX SCM_RIGHTS.
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
... this is copypasted all over the codebase & should've been a helper
to begin with really.
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
These are just used to iterate over active vty sessions, a vector is a
weird choice there.
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
These had no remaining users for a while now. The logging backend has
its own list of receivers.
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
This commit fixes the following problem:
- enter the interface node
- move the interface to another VRF
- try to continue configuring the interface
It is not possible to continue configuration because the XPath stored in
the vty doesn't correspond with the actual state of the system anymore.
For example:
```
nfware# conf
nfware(config)# interface enp2s0
<-- move the enp2s0 to a different VRF -->
nfware(config-if)# ip router isis 1
% Failed to get iface dnode in candidate DB
```
To fix the issue, go through all connected vty shells and update the
stored XPath.
Suggested-by: Renato Westphal <renato@opensourcerouting.org>
Signed-off-by: Igor Ryzhov <iryzhov@nfware.com>
The backoff code assumed that yang operations always completed quickly.
It checked for > 100 YANG modeled commands happening in under 1 second
to enable batching. If 100 yang modeled commands always take longer than
1 second batching is never enabled. This is the exact opposite of what
we want to happen since batching speeds the operations up.
Here are the results for libyang2 code without and with batching.
| action | 1K rts | 2K rts | 1K rts | 2K rts | 20k rts |
| | nobatch | nobatch | batch | batch | batch |
| Add IPv4 | .881 | 1.28 | .703 | 1.04 | 8.16 |
| Add Same IPv4 | 28.7 | 113 | .590 | .860 | 6.09 |
| Rem 1/2 IPv4 | .376 | .442 | .379 | .435 | 1.44 |
| Add Same IPv4 | 28.7 | 113 | .576 | .841 | 6.02 |
| Rem All IPv4 | 17.4 | 71.8 | .559 | .813 | 5.57 |
(IPv6 numbers are basically the same as iPv4, a couple percent slower)
Clearly we need this. Please note the growth (1K to 2K) w/o batching is
non-linear and 100 times slower than batched.
Notes on code: The use of the new `nb_cli_apply_changes_clear_pending`
is to commit any pending changes (including the current one). This is
done when the code would not correctly handle a single diff that
included the current changes with possible following changes. For
example, a "no" command followed by a new value to replace it would be
merged into a change, and the code would not deal well with that. A good
example of this is BGP neighbor peer-group changing. The other use is
after entering a router level (e.g., "router bgp") where the follow-on
command handlers expect that router object to now exists. The code
eventually needs to be cleaned up to not fail in these cases, but that
is for future NB cleanup.
Signed-off-by: Christian Hopps <chopps@labn.net>
When using the default CLI mode, the northbound layer needs to create
a separate transaction to process each YANG-modeled command since
they are supposed to be applied immediately (there's no candidate
configuration nor the "commit" command like in the transactional
CLI). The problem is that configuration transactions have an overhead
associated to them, in big part because of the use of some heavy
libyang functions like `lyd_validate()` and `lyd_diff()`. As of
now this overhead is substantial and doesn't scale well when large
numbers of transactions need to be performed in sequence.
As an example, loading 50k prefix-lists using a single transaction
takes about 2 seconds on a modern CPU. Loading the same 50k
prefix-lists using 50k transactions can take more than an hour
to complete (which is unacceptable by any standard). To fix this
problem, some heavy optimization work needs to be done on libyang and
on the FRR northbound itself too (e.g. perform partial configuration
diffs whenever possible). This, however, should be a long term
effort since these optimizations shouldn't be trivial to implement
and we're far from having the performance numbers we need.
In the meanwhile, this commit introduces a simple but efficient
workaround to alleviate the issue. In short, a new back-off timer
was introduced in the CLI to monitor and detect when too many
YANG-modeled commands are being received at the same time. When
a certain threshold is reached (100 YANG-modeled commands within
one second), the northbound starts to group all subsequent commands
into a single large transaction, which allows them to be processed
much faster (e.g. seconds and not hours). It's essentially a
protection mechanism that creates dynamically-sized transactions
when necessary to prevent performance issues from happening. This
mechanism is enabled both when parsing configuration files and when
reading commands from a terminal.
The downside of this optimization is that, if several YANG-modeled
commands are grouped into the same transaction and at least one of
them fails, the whole transaction is rejected. This is undesirable
since users don't expect transactional behavior when that's not
enabled explicitly. To minimize this issue, the CLI will log all
commands that were rejected whenever that happens, to make the
user aware of what happened and have enough information to fix
the problem. Commands that fail due to parsing errors or CLI-level
validations in general are rejected separately.
Again, this proposed workaround is intended to be temporary. The
goal is to provided a quick fix to issues like #6658 while we work
on better long-term solutions.
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
Rather than doing a f*gly hack for the RPKI code, let's do an on-exit
hook in cmd_node. Also allows replacing some special-casing in the vty
code.
Signed-off-by: David Lamparter <equinox@diac24.net>
The `--enable-pcreposix` configure option was not actually compiling
properly. Follow pre-existing pattern for inclusion of regex.h
or the pcreposix.h header.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
There's no need to check for removed configuration objects in the
following two cases:
* A private candidate configuration is being edited;
* The startup configuration is being read.
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
This makes printfrr extensions available in most of our format strings.
snprintf() is the obvious exception.
Signed-off-by: David Lamparter <equinox@diac24.net>
Add 'no log commands' cli and at the same time add a
--command-log-always to the daemon startup cli.
If --command-log-always is specified then all commands are
auto-logged and the 'no log commands' form of the command
is now ignored.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
The ability to lock the running configuration to prevent other users
from changing it is a very important one. We already supported
the "configure exclusive" command but the lock was applied to
the CLI users only (other clients like ConfD could still commit
configuration transactions, ignoring the CLI lock). This commit
introduces a global lock for the running configuration that is
shared by all northbound clients, and provides a public API to
manipulate it. This way other northbound clients will also be able
to lock/unlock the running configuration if required (the upcoming
gRPC northbound plugin will have RPCs for that).
NOTE: this is a management-level lock for the running configuration,
not to be confused with low-level locks used to avoid data races.
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
These are necessary to use functions defined in these headers from C++.
Signed-off-by: David Lamparter <equinox@diac24.net>
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
Just copying th const char* of the xpath means that if we
are enqueing multiple changes from a buffer, the last xpath
addedd will overwrite all of the previous references.
Copying the xpath to a buffer simplifies the API when
retrofitting the commands.
Signed-off-by: Emanuele Di Pascale <emanuele@voltanet.io>
Confirmed commits allow the user to request an automatic rollback to
the previous configuration if the commit operation is not confirmed
within a number of minutes. This is particularly useful when the user
is accessing the CLI through the network (e.g. using SSH) and any
configuration change might cause an unexpected loss of connectivity
between the user and the managed device (e.g. misconfiguration of a
routing protocol). By using a confirmed commit, the user can rest
assured the connectivity will be restored after the given timeout
expires, avoiding the need to access the router physically to fix
the problem.
When "commit confirmed TIMEOUT" is used, a new "commit" command is
expected to confirm the previous commit before the given timeout
expires. If "commit confirmed TIMEOUT" is used while there's already
a confirmed-commit in progress, the confirmed-commit timeout is
reset to the new value.
In the current implementation, if other users perform commits while
there's a confirmed-commit in progress, all commits are rolled back
when the confirmed-commit timeout expires. It's recommended to use
the "configure exclusive" configuration mode to prevent unexpected
outcomes when using confirmed commits.
When an user exits from the configuration mode while there's a
confirmed-commit in progress, the commit is automatically rolled
back and the user is notified about it. In the future we might
want to prompt the user if he or she really wants to exit from the
configuration mode when there's a pending confirmed commit.
Needless to say, confirmed commit only work for configuration
commands converted to the new northbound model. vtysh support will
be implemented at a later time.
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
A while ago all FRR configuration commands were converted to use the
QOBJ infrastructure to keep track of configuration objects. This
means the configuration lock isn't necessary anymore because the
QOBJ code detects when someones tries to edit a configuration object
that was deleted and react accordingly (log an error and abort the
command). The possibility of accessing dangling pointers doesn't
exist anymore since vty->index was removed.
Summary of the changes:
* remove the configuration lock and the vty_config_lockless() function.
* rename vty_config_unlock() to vty_config_exit() since we need to
clean up a few things when exiting from the configuration mode.
* rename vty_config_lock() to vty_config_enter() to remove code
duplication that existed between the three different "configuration"
commands (terminal, private and exclusive).
Configuration commands converted to the new northbound model don't
need the configuration lock either since the northbound API also
detects when someone tries to edit a configuration object that
doesn't exist anymore.
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
When editing the candidate configuration, the northbound must ensure
that either all changes made by a command are accepted or none are.
This is done to prevent inconsistent states where only parts of a
command are applied in the event any error happens.
The previous API for converted commands, the nb_cli_cfg_change()
function, required callers to pass an array containing all changes
that needed to be applied in the candidate configuration. The
problem with this API is that it was very inconvenient for complex
commands, which change different configuration options depending
on several factors. This required users to manipulate the array
of configuration changes using low-level primitives, making it
complicated to implement some commands.
To solve this problem, introduce a new API based on the two following
functions:
- nb_cli_enqueue_change()
- nb_cli_apply_changes()
The first function is used to enqueue configuration changes, one
at time. Then the nb_cli_apply_changes() function is used to apply
all the enqueued configuration changes.
To implement this, a static-sized array was allocated in the "vty"
structure, along with a counter of enqueued changes. This eliminates
the need to declare an array of configuration changes in every
converted CLI command, simplifying things quite considerably.
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
When reading in config files and we have failures on multiple
lines actually note the actual failure lines and return them.
This fixes an issue where we stopped counting errors after
the first one and we got missleading line numbers that
did not correspond to the actual problem.
This is fixed:
sharpd@donna ~/frr> sudo /usr/lib/frr/pimd --log=stdout -A 127.0.0.1 -f /etc/frr/pimd.conf
2018/10/11 09:41:01 PIM: VRF Created: default(0)
2018/10/11 09:41:01 PIM: pim_vrf_enable: for default
2018/10/11 09:41:01 PIM: zclient_lookup_sched_now: zclient lookup immediate connection scheduled
2018/10/11 09:41:01 PIM: zclient_lookup_new: zclient lookup socket initialized
2018/10/11 09:41:01 PIM: pimd 6.1-dev starting: vty@2611
2018/10/11 09:41:01 PIM: [EC 100663304] ERROR: No such command on config line 2: inteface lo
2018/10/11 09:41:01 PIM: [EC 100663304] ERROR: No such command on config line 3: ip igmp
2018/10/11 09:41:01 PIM: [EC 100663304] ERROR: No such command on config line 4: ip igmp join 224.1.1.1 13.13.13.2
^C2018/10/11 09:45:09 PIM: Terminating on signal SIGINT
2018/10/11 09:45:09 PIM: VRF Deletion: default(0)
Fixes: #3161
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
When reading the config file add an ability to know
if we have properly read in anything. So that a daemon
can make fallback plans.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Sometimes output would be mangled when filtering with include as a
result of the following bugs:
* Filters were applied per each call to vty_out() instead of buffering
until a line break and then applying
* Long output would sometimes be cut due to using the wrong buffer
pointer
Also remove the trailing \n as it should no longer be necessary to
ensure the vty prompt ends up on a new line.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
* Rewrite pager implementation
* Replace fprintf() with vty_out()
* Modify vty_out() for better vtysh support
* Remove static global outputfile var
* Remove fp argument from many vtysh functions
* Add some docs for stuff along the way
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
This patch adds a CLI preprocessor function that activates when `|` is
found in the command. This is the start of adding support for some text
processing utilities intended for inline use. The first one implemented
here is `| include`, which provides grep-like filtering of command
output.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
