LYD_VALIDATE_MULTI_ERROR was added in libyang 2.1.36. The currently
enforced minimum of libyang is 2.0.0. Stick an #ifdef around it.
Fixes: 51a2a4b3f4 ("lib: print all errors when validating a config")
Cc: Igor Ryzhov <iryzhov@nfware.com>
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
`nb_cli_apply_changes` can be called with base xpath which should be
prepended to xpaths of every change in a transaction. This base xpath is
respected by regular northbound CLI but not by mgmtd. This commit fixes
the problem.
Signed-off-by: Igor Ryzhov <iryzhov@nfware.com>
mgmtd frees all non-NULL change->value variables at the end of every
commit. We shouldn't assign change->value with data returned by libyang
to prevent freeing of library-allocated memory.
Signed-off-by: Igor Ryzhov <iryzhov@nfware.com>
This was true when we had only a CLI for configuration. Now mgmtd has a
public frontend interface that can be used by external applications, and
they can send invalid requests that lead to errors.
This is still true for CLI though, so the same comment still stays in
`nb_cli_apply_changes_internal`.
Signed-off-by: Igor Ryzhov <iryzhov@nfware.com>
The code doesn't work at all. It tries to use libyang operation
metadata in a regular (not diff) data tree, and regular data trees
don't provide this data. Also, for destroy operations, it searches
for nodes in the running config, which may not have the deleted nodes
if we're not using implicit commits.
Signed-off-by: Igor Ryzhov <iryzhov@nfware.com>
Previously was using an API that returned the root of the data tree given the
users input xpath value, and then used it like it was the leaf node (last not
first). So basically this CLI command only worked when one requested the root
node of the model.
Signed-off-by: Christian Hopps <chopps@labn.net>
Add a wrapper around lys_find_xpath which has an unfortunate API
returning an allocated set of schema nodes when we only ever expect and
want one.
Another libyang function `lys_find_path` returns a single node; however,
that function can assert/abort on invalid path values so is unsuitable
for user input.
Replace previous uses of `lys_find_path` with new API when dealing with
possible invalid path values (i.e., from a user).
Signed-off-by: Christian Hopps <chopps@labn.net>
This commit introduces the MGMT Transaction framework that takes
management requests from one (or more) frontend client sessions,
translates them into transactions and drives them to completion
in co-oridination with one (or more) backend client daemons
involved in the request.
This commit includes the following functionalities in the changeset:
1. Introduces the actual Transaction module. Commands added related to
transaction are:
a. show mgmt transaction all
2. Adds support for commit rollback feature which stores upto the 10
commit buffers. Each commit has a commit-id which can be used to
rollback to the exact configuration state.
Commands supported for this feature are:
a. show mgmt commit-history
b. mgmt rollback commit-id COMMIT_ID
3. Add hidden commands to enable record various performance metrics:
a. mgmt performance-measurement
b. mgmt reset-statistic
Co-authored-by: Pushpasis Sarkar <pushpasis@gmail.com>
Co-authored-by: Abhinay Ramesh <rabhinay@vmware.com>
Co-authored-by: Ujwal P <ujwalp@vmware.com>
Signed-off-by: Yash Ranjan <ranjany@vmware.com>
This commit introduces the MGMT Backend Interface which can be used
by back-end management client daemons like BGPd, Staticd, Zebra to
connect with new FRR Management daemon (MGMTd) and utilize the new
FRR Management Framework to let any Frontend clients to retrieve any
operational data or manipulate any configuration data owned by the
individual Backend daemon component.
This commit includes the following functionalities in the changeset:
1. Add new Backend server for Backend daemons connect to.
2. Add a C-based Backend client library which can be used by daemons
to communicate with MGMTd via the Backend interface.
3. Maintain a backend adapter for each connection from an appropriate
Backend client to facilitate client requests and track one or more
transactions initiated from Frontend client sessions that involves
the backend client component.
4. Add the following commands to inspect various Backend client
related information
a. show mgmt backend-adapter all
b. show mgmt backend-yang-xpath-registry
c. show mgmt yang-xpath-subscription
Co-authored-by: Pushpasis Sarkar <pushpasis@gmail.com>
Co-authored-by: Abhinay Ramesh <rabhinay@vmware.com>
Co-authored-by: Ujwal P <ujwalp@vmware.com>
Signed-off-by: Yash Ranjan <ranjany@vmware.com>
Features added in this commit:
1. Bringup/shutdown new management daemon 'mgmtd' along with FRR.
2. Support for Startup, Candidate and Running DBs.
3. Lock/Unlock DS feature using pthread lock.
4. Load config from a JSON file onto candidate DS.
5. Save config to a JSON file from running/candidate DS.
6. Dump candidate or running DS contents on the terminal or a file in
JSON/XML format.
7. Maintaining commit history (Full rollback support to be added in
future commits).
8. Addition of debug commands.
Co-authored-by: Yash Ranjan <ranjany@vmware.com>
Co-authored-by: Abhinay Ramesh <rabhinay@vmware.com>
Co-authored-by: Ujwal P <ujwalp@vmware.com>
Signed-off-by: Pushpasis Sarkar <pushpasis@gmail.com>
Add a hash_clean_and_free() function as well as convert
the code to use it. This function also takes a double
pointer to the hash to set it NULL. Also it cleanly
does nothing if the pointer is NULL( as a bunch of
code tested for ).
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Pass context argument by value on initialization to be clear that the
value is used/saved but not a pointer to the value. Previously the
northbound code was incorrectly holding a pointer to stack allocated
context structs.
However, the structure definition also had some musings (ifdef'd out
code) and a comment that might be taken to imply that user data could
follow the structure and thus be maintained by the code; it won't; so it
can't; so get rid of the disabled misleading code/text from the
structure definition.
The common use case worked b/c the transaction which cached the pointer
was created and freed inside a single function
call (`nb_condidate_commit`) that executed below the stack allocation.
All other use cases (grpc, confd, sysrepo, and -- coming soon -- mgmtd)
were bugs.
Signed-off-by: Christian Hopps <chopps@labn.net>
Firstly, *keep no change* for `hash_get()` with NULL
`alloc_func`.
Only focus on cases with non-NULL `alloc_func` of
`hash_get()`.
Since `hash_get()` with non-NULL `alloc_func` parameter
shall not fail, just ignore the returned value of it.
The returned value must not be NULL.
So in this case, remove the unnecessary checking NULL
or not for the returned value and add `void` in front
of it.
Importantly, also *keep no change* for the two cases with
non-NULL `alloc_func` -
1) Use `assert(<returned_data> == <searching_data>)` to
ensure it is a created node, not a found node.
Refer to `isis_vertex_queue_insert()` of isisd, there
are many examples of this case in isid.
2) Use `<returned_data> != <searching_data>` to judge it
is a found node, then free <searching_data>.
Refer to `aspath_intern()` of bgpd, there are many
examples of this case in bgpd.
Here, <returned_data> is the returned value from `hash_get()`,
and <searching_data> is the data, which is to be put into
hash table.
Signed-off-by: anlan_cs <vic.lan@pica8.com>
State-only and configuration presence-containers need to be treated
differently when iterating over YANG operational data. Currently the
get_elem() callback is used to know when a state-only p-container
exists or not, and configuration p-containers are assumed to always
exist, which is clearly wrong. Fix this by checking the running
configuration to know whether a rw p-container exists or not.
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
The backoff code assumed that yang operations always completed quickly.
It checked for > 100 YANG modeled commands happening in under 1 second
to enable batching. If 100 yang modeled commands always take longer than
1 second batching is never enabled. This is the exact opposite of what
we want to happen since batching speeds the operations up.
Here are the results for libyang2 code without and with batching.
| action | 1K rts | 2K rts | 1K rts | 2K rts | 20k rts |
| | nobatch | nobatch | batch | batch | batch |
| Add IPv4 | .881 | 1.28 | .703 | 1.04 | 8.16 |
| Add Same IPv4 | 28.7 | 113 | .590 | .860 | 6.09 |
| Rem 1/2 IPv4 | .376 | .442 | .379 | .435 | 1.44 |
| Add Same IPv4 | 28.7 | 113 | .576 | .841 | 6.02 |
| Rem All IPv4 | 17.4 | 71.8 | .559 | .813 | 5.57 |
(IPv6 numbers are basically the same as iPv4, a couple percent slower)
Clearly we need this. Please note the growth (1K to 2K) w/o batching is
non-linear and 100 times slower than batched.
Notes on code: The use of the new `nb_cli_apply_changes_clear_pending`
is to commit any pending changes (including the current one). This is
done when the code would not correctly handle a single diff that
included the current changes with possible following changes. For
example, a "no" command followed by a new value to replace it would be
merged into a change, and the code would not deal well with that. A good
example of this is BGP neighbor peer-group changing. The other use is
after entering a router level (e.g., "router bgp") where the follow-on
command handlers expect that router object to now exists. The code
eventually needs to be cleaned up to not fail in these cases, but that
is for future NB cleanup.
Signed-off-by: Christian Hopps <chopps@labn.net>
lyd_merge_tree replaces dest siblings with source siblings, not what we
want. Instead lyd_merge_siblings to keep both. Instead lyd_merge_siblings
to keep both.
Signed-off-by: Christian Hopps <chopps@labn.net>
Compile with v2.0.0 tag of `libyang2` branch of:
https://github.com/CESNET/libyang
staticd init load time of 10k routes now 6s vs ly1 time of 150s
Signed-off-by: Christian Hopps <chopps@labn.net>
This parametrized use of flog with variable EC and priority doesn't mesh
particularly well with the xref code & there isn't really much reason to
not use fixed/constant calls like this.
Signed-off-by: David Lamparter <equinox@diac24.net>
Back when I put this together in 2015, ISO C11 was still reasonably new
and we couldn't require it just yet. Without ISO C11, there is no
"good" way (only bad hacks) to require a semicolon after a macro that
ends with a function definition. And if you added one anyway, you'd get
"spurious semicolon" warnings on some compilers...
With C11, `_Static_assert()` at the end of a macro will make it so that
the semicolon is properly required, consumed, and not warned about.
Consistently requiring semicolons after "file-level" macros matches
Linux kernel coding style and helps some editors against mis-syntax'ing
these macros.
Signed-off-by: David Lamparter <equinox@diac24.net>
Example:
```
show yang operational-data /frr-routing:routing/control-plane-protocols/control-plane-protocol[type='frr-staticd:staticd'][name='staticd'][vrf='default'] staticd
```
Signed-off-by: Igor Ryzhov <iryzhov@nfware.com>
Make it possible to load YANG modules outside the main northbound
initialization. The primary use case is to support YANG modules
that are specific to an FRR plugin. Example: only load the PCEP
YANG module when the corresponding FRR plugin is loaded. Other use
cases might arise in the future.
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
Combine yang_snodes_iterate_module() and yang_snodes_iterate_all()
into an unified yang_snodes_iterate() function, where the first
"module" parameter is optional. There's no point in having two
separate YANG schema iteration functions anymore now that they are
too similar.
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
The only safe way to iterate over all schema nodes of a given YANG
module is by iterating over all schema nodes of all YANG modules
and filter out the nodes that belong to other modules.
The original yang_snodes_iterate_module() code did the following:
1 - Iterate over all top-level schema nodes of the given module;
2 - Iterate over all augmentations of the given module.
While that iteration strategy is more efficient, it does't handle
well more complex YANG hierarchies containing nested augmentations
or self-augmenting modules. Any iteration that isn't done on the
resolved YANG data hierarchy is fragile and prone to errors.
Fixes regression introduced by commit 8a923b4851 where the
gen_northbound_callbacks tool was generating duplicate callbacks
for certain modules.
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
Display human readable error message in northbound rpc
transaction failure. In case of vtysh nb client, the error
message will be displayed to user.
Testing:
bharat# clear evpn dup-addr vni 1002 ip 11.11.11.11
Error type: generic error
Error description: Requested IP's associated MAC aa:aa:aa:aa:aa:aa is still
in duplicate state
Signed-off-by: Chirag Shah <chirag@nvidia.com>
When calling yang_snodes_iterate_subtree we don't care about
the return code. So explicitly say we don't care so that
SA tools can be on the same page as us.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Change the way the YANG schema node iteration functions work so that
the northbound layer won't have issues with more complex YANG modules
that contain multiple levels of YANG augmentations or modules that
augment themselves indirectly (by augmenting groupings).
Summary of the changes:
* Change the yang_snodes_iterate_subtree() function to always follow
augmentations and add an optional "module" parameter to narrow down
the iteration to nodes of a single module (which is necessary in
some cases). Also, remove the YANG_ITER_ALLOW_AUGMENTATIONS flag
as it's no longer necessary.
* Change yang_snodes_iterate_all() to do a DFS iteration on the resolved
YANG data hierarchy instead of iterating over each module and their
augmentations sequentially.
Reported-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
Whenever libyang loads a module that contains a leafref, it will
also implicitly load the module of the referring node if it's
not loaded already. That makes sense as otherwise it wouldn't be
possible to validate the leafref value correctly.
The problem is that loading a module implicitly violates the
assumption of the northbound layer that all loaded modules
are implemented (i.e. they have a northbound node associated
to each schema node). This means that loading a module that
isn't implemented can lead to crashes as the "priv" pointer
of schema nodes is no longer guaranteed to be valid. To fix this
problem, add a few null checks to ignore data nodes associated
to non-implemented modules.
The side effect of this change is harmless. If a daemon receives
configuration it doesn't support (e.g. BFD peers on staticd),
that configuration will be stored but otherwise ignored. This can
only happen when using a northbound client like gRPC, as the CLI
will never send to a daemon a command it doesn't support. This
minor problem should go away in the long run as FRR migrates to
a centralized management model, at which point the YANG-modeled
configuration of all daemons will be maintained in a single place.
Finally, update some daemons to stop implementing YANG modules
they don't need to (i.e. revert 1b741a01c and a74b47f5).
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
In case of config rollback is enabled,
record northbound transaction based on a control flag.
The actual frr daemons would set the flag to true via
nb_init from frr_init.
This will allow test daemon to bypass recording
transacation to db.
Signed-off-by: Chirag Shah <chirag@nvidia.com>
During the prep phase to apply a northbound commit, if no changes were
detected make sure we fill the error message buffer to explain this.
Signed-off-by: Quentin Young <qlyoung@nvidia.com>
While a configuration transaction can't be rejected once it reaches
the APPLY phase, we should allow NB callbacks to generate error
or warning messages when a configuration change is being applied.
That should be useful, for example, to return warnings back to
the user informing that the applied configuration has some kind of
inconsistency or is missing something in order to be effectively
activated. The infrastructure for this was already present, but the
northbound layer was ignoring all errors/warnings generated during
the apply/abort phases instead of returning them to the user. This
commit changes that.
In the gRPC plugin, extend the Commit() RPC adding a new
"error_message" field to the response type. This is necessary to
allow errors/warnings to be returned even when the commit operation
succeeds (since grpc::Status::OK doesn't support error messages
like the other status codes).
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
All userdata pointers need to be rekeyed to their new xpaths, not just
the one associated with the dnode being moved.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
